Summary:
Ref T13603. This removes the file attachment edge types, which no longer have readers or writers.
It does not delete the underlying data.
This indirectly removes API access to this edge. As far as I know, this was only used by one customer (see D21480) who has migrated away from Phabricator.
Test Plan: Grepped for edge constants.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21823
Summary: Ref T13603. Removes all reads and writes to the other half of the legacy edge.
Test Plan:
- Verified I could still see file content accessible only through object attachment.
- This fixes a unit test broken by removal of only half the edge access in the previous change.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21820
Summary: Ref T13603. Migrate all code which interacts with the "ObjectHasFile" edge to use the "Attachments" table instead.
Test Plan:
- Edited a paste's view policy, confirmed associated file secret was scrambled.
- Verified I could still view paste content as a user who could not naturally view the underlying file.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21819
Summary: Ref T13603. New transactional file attachment extracts PHIDs earlier than the older mechanism did. Allow either pathway to save content and extract a PHID.
Test Plan:
- Created and modified a paste.
- Confirmed both pathways do old and new storage writes.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21817
Summary: Ref T13603. This adds a second write to new "attachment" storage to all writers except one in Paste, which creates the file inline.
Test Plan:
- Updated a macro image, confirmed a write to "attachment" storage (transaction pathway).
- Updated a blog profile image, confirmed a write to "attachment" storage (legacy pathway).
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21816
Summary:
Ref T13603. Currently, files are sometimes detached from objects. For example, when you change the image for a Macro, the old image is detached.
This is wrong: the image should remain attached so users who can view the macro can view the complete "alice change the image from X to Y" transaction. To be able to understand the change that was applied, you need to be able to view both files.
All workflows which currently detach files aren't conistent with the modern way applications behave, except maybe one callsite in a unit test, and that one's kind of moot.
Get rid of this stuff and just use PHID extraction to perform file attachment in all cases.
Test Plan: Created and edited macros, verified files were properly attached and remained attached across edits.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21815
Summary: Ref T13603. Prepare to move this relationship out of edge storage into dedicated storage so it is easier to formalize better in the UI.
Test Plan: Ran `bin/storage upgrade`.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21813
Summary: Ref T13603. When processing transactions that include remarkup blocks, stop extracting file references from them for attachment.
Test Plan: Submitted a remarkup block with a file reference, no longer saw automatic file attachment.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21812
Summary: Ref T13588. This allows "arc unit --everything" to more or less run cleanly on PHP 8.1.
Test Plan: Ran "arc unit --everything".
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21821
Summary:
Ref T13603. Allow "VersionedDraft" to persist remarkup comment area metadata from stacked actions controls.
When files are dragged and dropped, record them as explicit uploads in comment metadata.
Test Plan: Dragged and dropped files into Remarkup stacked action text areas (e.g., in Maniphest), reloaded page, saw metadata persist across reloads.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21828
Summary:
Ref T13681. This was introduced in D593, never used, and doesn't make sense or have any uses in modern Phabricator.
It also does some pretty direct writes that can technically do things that at least //look// like they violate policies, so remove it.
Test Plan:
- Checked the API console, no longer saw "feed.publish".
- Grepped for "feed.publish", no hits.
Maniphest Tasks: T13681
Differential Revision: https://secure.phabricator.com/D21826
Summary:
Ref T13679. In D16983, global settings objects were given an exception to let logged-out users see them, even on installs with no "public" user role.
This exception is too broad and grants everyone all capabilities, not just "CAN_VIEW". In particular, it incorrectly grants "CAN_EDIT", so any user can edit global settings defaults.
Restrict this grant to "CAN_VIEW".
Test Plan:
- As a non-administrator, tried to edit global settings.
- Before: could.
- After: could not.
Maniphest Tasks: T13679
Differential Revision: https://secure.phabricator.com/D21811
Summary: Ref T13677. This was an accidental change in D21807: when reclaiming a resource, wait for it to be completely destroyed before allowing a lease to reclaim another resource.
Test Plan: Reverts accidental behavior change in D21807.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13677
Differential Revision: https://secure.phabricator.com/D21809
Summary:
Ref T13677. Drydock has a hard-coded and fairly arbitrary limit which prevents a resource pool from growing more than 25% at once.
This is vaguely reasonable for resources which allocate quickly, but suffocating for slower resources. It's also wholly arbitrary, and the "one per lease" limit introduced in D21807 should do a better job of covering the same ground while generally being more reasonable, expected, and predictable.
Test Plan: Ran Drydock allocations without the throttle, saw faster pool growth.
Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13677
Differential Revision: https://secure.phabricator.com/D21808
Summary:
Ref T13677. Currently, one lease may cause multiple resources to allocate simultaneously if it starts allocating one, then wakes up from a yield later on and still sees no available resources.
This is never desired -- or, at least, produces desirable behavior only entirely by accident. Normally, it causes an excess of resources to allocate.
This is not a catastrophic problem: the extra resources usually get used sooner or later or cleaned up; and the total amount of badness is limited by overall resource allocation limits.
However, this behavior is also suppressed by an artificial "25% of current pool size" growth limit throttle which I intend to remove. Removing this throttle without fixing the allocator behavior could make this "too many resources" problem worse.
Change the allocator so that a lease that has started allocating a resource won't allocate another resource until the first resource leaves the "pending" state.
This also fixes some general oddness with the allocator and attempts to simplify the structure.
Test Plan:
- Ran 8 taskmasters.
- Destroyed all resources and leases.
- Leased 4 working copies.
- Saw exactly 4 resources build and lease, all simultaneously.
- Destroyed all resources and leases.
- Leased 32 working copies.
- Saw exactly 32 resources build and lease, approximately 8 at a time (limited by taskmasters).
- Destroyed all leases (but not resources).
- Leased 32 working copies, saw them satisfied by existing resources.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13677
Differential Revision: https://secure.phabricator.com/D21807
Summary:
Ref T13677. Track which resources a given lease has begun allocating or reclaiming in a more formal way, and add logging for waiting actions.
The "allocating" mechanism is new. This will replace an existing similar "reclaiming" mechanism in a future change.
Test Plan: See followup changes.
Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13677
Differential Revision: https://secure.phabricator.com/D21806
Summary: Ref T13677. These flags increase the convenience of testing in a development environment.
Test Plan: Used new "--all" flags to release all resources and leases.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13677
Differential Revision: https://secure.phabricator.com/D21805
Summary:
Ref T13676. The 3-minute grace period when a resource can not be reclaimed after its leases are released currently doesn't work reliably because the Resource object usually isn't actually updated when a lease is released.
Add an additional check for recently-destroyed leases, and extend the grace period if we find any.
Test Plan:
- See T13676. Ran reproduction sequence there, observed immediate resource reclamation.
- Applied patch.
- Ran sequence again, observed repository B wait 3 minutes to reclaim a repository A resource.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13676
Differential Revision: https://secure.phabricator.com/D21803
Summary:
Ref T13676. Currently, "bin/drydock lease" just creates a lease that permits any blueprint.
To prepare for "use specific blueprint X", unify the logic between this workflow and LeaseUpdateWorker so we select only blueprints which at least have coarse compatibility (e.g., if we're leasing a host, only select enabled blueprints of classes that can allocate hosts).
Test Plan: Used `bin/drydock lease` to try to lease a nonsense type, got sensible error. Leased a host.
Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13676
Differential Revision: https://secure.phabricator.com/D21801
Summary: Ref T13676. This makes it easier to create resource pressure without juggling a big pile of terminals.
Test Plan: Used `bin/drydock lease --count 5 ...` to acquire 5 leases.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13676
Differential Revision: https://secure.phabricator.com/D21800
Summary: Ref T13676. When the required "repositories.map" attribute is omitted, `bin/drydock lease` currently fatals in an unhelpful way when trying to lease a working copy.
Test Plan:
Ran `bin/drydock lease --type working-copy` with no attributes, after following steps in T13676.
```
<Allocation Failed> One or more blueprints promised a new resource, but failed when allocating: [PhutilAggregateException] All blueprints failed to allocate a suitable new resource when trying to allocate lease ("PHID-DRYL-orbtwtlinksm3xqpyhmw").
- Exception: Working copy lease is missing required attribute "repositories.map".
Attribute "repositories.map" should be a map of repository specifications.
```
Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13676
Differential Revision: https://secure.phabricator.com/D21796
Summary: Ref T13676. Ref T13588. Fix some issues that prevent "bin/phd" and "bin/drydock" from executing under PHP 8.1, broadly because `null` is being passed to `strlen()`.
Test Plan: Ran `bin/phd debug task` and `bin/drydock ...` under PHP 8.1.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13676, T13588
Differential Revision: https://secure.phabricator.com/D21795
Summary:
Ref T5479. Ref T13658. This was a contributed application from the early days of Phabricator which never had customers or users in the wild. The contributor moved on from the project many years ago.
Any capabilities in this general role would look different today. It also has one or two product name literal strings, so this is as good a time as any to remove it.
This change does not remove storage; I'll issue upgrade guidance and do that separately after some time.
Test Plan: Grepped for "phragment", got no relevant hits.
Subscribers: hach-que, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13658, T5479
Differential Revision: https://secure.phabricator.com/D21793
Summary:
Ref T9530. Ref T13658. The "Releeph" application was never useful outside of Facebook and any application providing release support would not resemble it much.
It has some product name literal strings, so now is as good a time as any to get rid of it.
This application never left prototype and I'm not aware of any install in the wild that uses it (or has ever used it).
I did not destroy the database itself. I'll issue upgrade guidance and destroy the database in some future release, just in case.
Test Plan: Grepped for "releeph", found no relevant/removable hits.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13658, T9530
Differential Revision: https://secure.phabricator.com/D21792
Summary: Ref T13658.
Test Plan: Static checks only, these are all reachable only by breaking your install in weird ways.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21781
Summary: Ref T13658.
Test Plan: Static checks only, these are mostly obscure or require breaking the install in unusual ways.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21780
Summary: Ref T13658.
Test Plan:
This is non-exhaustive.
- Started a JIRA auth provider creation workflow.
- Same for LDAP.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21779
Summary: Ref T13658.
Test Plan:
This is non-exhuastive.
- Poked around "Auth": edited a provider, toggled a provider.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21778
Summary: Ref T13658.
Test Plan:
This test plan is non-exhaustive.
- Ran `bin/storage databases`.
- Viewed Badges UI exmaples page.
- Used eval rule for `strings.platform.server.name`, got "Phabricator".
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21773
Summary: Ref T13658.
Test Plan:
This test plan is non-exhaustive.
- Viewed "remarkup.process" Conduit method API page.
- Viewed URIs in a Diffusion repository.
- Viewed editor protocol configuration in Settings.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21772
Summary: Ref T13658.
Test Plan:
This test plan is non-exhaustive.
- Ran `bin/mail`.
- Uninstalled and reinstalled an application.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21770
Summary: Ref T13658
Test Plan:
This test plan is non-exhaustive.
- Looked at some of the config.
- Looked at guides.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21769
Summary: Ref T13658.
Test Plan:
This is non-exhaustive.
- Looked at some (most?) of the config values in Config.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21768
Summary: Ref T13658.
Test Plan:
This test plan is non-exhaustive.
- Used Auth to start an "another copy of Phabricator" OAuth server workflow.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21766
Summary: Ref T13658. I used the linter in D21763 to identify these and `split` them into arbitrary groups of 10 files.
Test Plan:
This test plan is non-exhaustive, because some of these strings are difficult to reach.
- Looked at "Create Service" in Almanac.
- Used "bin/auth" to go through a one-time auth workflow (not all related strings can be hit on a single workflow).
- Started the "Generate Keypair" worfklow in "SSH Public Keys".
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21765
Summary:
Ref T13667. The Herald "content added" rule (and other similar rules) do not correctly extract content from empty commits.
When we load an empty raw diff, return an empty changed content map.
Ref T13588. Also fix some PHP8.1 null/string stuff
Test Plan:
- Ran "bin/repository reparse --publish <commit>", with an empty commit hash and a nonempty commit hash.
- Reviewed Herald transcripts for general sanity.
Maniphest Tasks: T13667, T13588
Differential Revision: https://secure.phabricator.com/D21761
Summary:
Ref T13673. After the changes in that task, we may execute "git config -l ..." as a user other than the user we used to write this temporary file.
Use "--file -" to pass the data instead, avoiding use of temporary files. This makes us agnostic to filesystem permissions.
Test Plan: Viewed a Git repository with submodules in Diffusion with "ssh.user" configured as a user relatively isolated from the webserver user.
Maniphest Tasks: T13673
Differential Revision: https://secure.phabricator.com/D21759
Summary: Ref T13588. Fix a couple of argument parsing issues here.
Test Plan: Ran "bin/auth recover" under PHP 8.1.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21757
Summary:
See T13673. Recent versions of Git (and older versions with backported security patches) now refuse to run Git commands if the top-level repository directory is not owned by the user running the command.
Currently, we "sudo" to that user only when performing writes, so upgrading Git can aggressively break a Phabricator system by knocking out essentially all Diffusion/Conduit read pathways.
As an immediate mitigation, just "sudo" in all cases where a daemon user is available. This fixes the problem, and seems like the least-bad approach. The downside is that the web user may theoretically have fewer privileges than the daemon user and this could reduce the number of layers an attacker armed with some other Git vulnerability might have to get through to do something dangerous (e.g., perform a write on a pathway where only reads are expected), but any separation between the web and daemon accounts is essentially theoretical and has never been enforced.
Test Plan: Applied patch to impacted Phacility shard, saw Diffusion work properly again.
Differential Revision: https://secure.phabricator.com/D21756
Summary: Ref T13661. Fix up some UI and language so it's more clear that this is about disabling blog comments.
Test Plan: Viewed UIs, saw some more usable strings.
Maniphest Tasks: T13661
Differential Revision: https://secure.phabricator.com/D21755
Summary: Ref T13661. This allows posts to have comments disabled (or restricted) on a per-post basis, and makes them inherit the containing blog policy by default.
Test Plan: Locked a post by editing its policy explicitly; locked a post by editing the containing blog policy.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13661
Differential Revision: https://secure.phabricator.com/D21754
Summary: Ref T13661. This ancient "RemarkupInterface" constant has no references.
Test Plan: Grepped for references, found none. Viewed a blog, viewed a post.
Maniphest Tasks: T13661
Differential Revision: https://secure.phabricator.com/D21753
Summary: Ref T13588. This field may be "null" (and is probably never the empty string, but that's a more ambitious fix).
Test Plan: Ran unit tests, got a pass.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21752
Summary:
Ref T13661.
I'm fairly sure these policies don't actually do anything (you can't "interact" with a blog) but the primarily support a Phame Post object policy of "Same as Parent Blog", which is the "natural" interact policy for a post.
Most of this is infrastructure support for mutable interact policies: today, only Maniphest has interact mutability and only via indirect effects (locking tasks), not through a directly mutable "Can Interact" policy.
Test Plan:
Ran storage upgrade, edited interact policy of a blog, saw appropriate persistence and transactions.
Created and edited a task to make sure there's no weird fallout from increasing what can be done with interact policies.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13661
Differential Revision: https://secure.phabricator.com/D21751
Summary: Ref T13661. The Phame blog main page has a double breadcrumb border in non-live mode.
Test Plan: Viewed a blog page in internal and live modes, saw single border in both cases (previously: double border for internal mode).
Maniphest Tasks: T13661
Differential Revision: https://secure.phabricator.com/D21750
Summary: Ref T13661. These constants are no longer used.
Test Plan:
For a blog with a nonempty description, viewed the "Manage" page and the home page; saw properly rendered description.
Grepped for removed constants, no hits.
Maniphest Tasks: T13661
Differential Revision: https://secure.phabricator.com/D21749
Summary: Ref T6203. Ref T13661. These policies are incorrectly nullable, although it's likely that no pathway exists in the application to write NULL into them. Fix the schema.
Test Plan: Ran `bin/storage upgrade`.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13661, T6203
Differential Revision: https://secure.phabricator.com/D21748
Summary:
Ref T13588. This configuration value may not be set.
Also fix an issue in `bin/storage` and whatever else I hit between now and this diff actually uploading.
Also fix a MySQLi report mode difference, beginning in PHP 8.1.
Also update a bunch of "static" property usage in Lisk.
Test Plan: Ran `bin/files ...` locally under PHP 8.1.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21744
Summary:
Ref T13669. See that task for discussion. Based on a disheartening interaction in June 2020, discourage use of Mailgun.
Also update the default Postmark inbound address list, since they've added a new address.
Test Plan: Read documentation, grepped for "mailgun".
Maniphest Tasks: T13669
Differential Revision: https://secure.phabricator.com/D21738
Summary:
Ref PHI2149. This provides the trivial Query class required for the daemons to publish Almanac Interface transactions.
(Publishing these doesn't do anything interesting, but currently leaves an error in the daemon logs.)
Test Plan:
- Stopped the daemons.
- Edited the port of an Interface in Almanac.
- Ran `bin/worker execute --active --class PhabricatorApplicationTransactionPublishWorker` to publish the transaction.
- Before: fatal on missing class, "Unable to load query for transaction object...".
- After: transaction publishes cleanly.
Differential Revision: https://secure.phabricator.com/D21726
Summary: Ref T13666. See T13666#256253. The order of these parameters is incorrect (introduced in D18817, which was trying to turn the revset "A or B" into "B or A"), but when "commit" is the single head of "branch" (which is common), the revset is functionally equivalent.
Test Plan:
An easy reproduction case is to make a "diffusion.historyquery" call manually against a Mercurial repository, with a valid "branch" name and some non-head "commit".
- Before patch, queried "head^" (by commit hash, not by symbol) of "default" and got "head" too (wrong).
- Applied patch to `secure001.phacility.net` (my local `hg` web environment is iffy right now).
- Re-ran the same query, saw "head^" as the first result rather than "head" (all hashes rather than symbols, as above), which is desired.
Reviewers: cspeckmim
Reviewed By: cspeckmim
Maniphest Tasks: T13666
Differential Revision: https://secure.phabricator.com/D21722
Summary:
Found in a change submitted to Phorge (https://we.phorge.it/D25018#inline-231), this fixes a typo in populating search the `contributorPHIDs` constraint in the Legalpad search engine.
Currently when trying to search legalpad documents by contributor an error is encountered:
```lang=console
Array for %Ls conversion is empty. Query: contributor.dst IN (%Ls)
```
Test Plan: I searched for legalpad documents based on a contributor and got back correct results.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D21719
Summary:
When previously known commits have been destroyed in a Mercurial repository, Phabricator does not end up marking the commits as unreachable. This results in daemon tasks which continuously fail and retry.
This updates `PhabricatorRepositoryDiscoveryEngine` and `PhabricatorManagementRepositoryMarkReachableWorkflow` to include support of marking commits as unreachable for Mercurial repositories.
The `PhabricatorMercurialGraphStream` also needed updated to support a stream with no starting commit.
Refs T13634
Test Plan:
1. I set up a hosted Mercurial repository.
2. I removed the head commit from the on-disk repository state.
3. I attempted to load the repository page and saw an exception due to a missing commit.
4. I went to `/manage` for the repository and scheduled an update of the repository.
5. After an updated performed, I went to the repository main page and saw there was no exception and the history view properly did not have the commit I had removed.
6. I checked the phd logs and verified there were no exceptions related to the repository.
7. I ran the `./bin/repository mark-reachable` command on the Mercurial repository and it reported that it marked the commit as unreachable.
8. I pushed the same commit back upstream and verified that the commit was found and displayed in the history view of the repository page and `mark-unreachable` did not identify it as being unreachable.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T13634
Differential Revision: https://secure.phabricator.com/D21715
Summary:
The history query for the repository page isn't actually used to display any content. It looks like it was previously used to display the last user which modified a file however this looks to be removed in D21404. This removes the history query from happening as well as updates `DiffusionBrowseTableView` to remove the parameters for passing this information in, resulting in also updating `DiffusionBrowseController` to no longer need to put this information together.
Refs T13666
Test Plan:
1. I removed commits from a repository on the local state.
2. I navigated to the repository's landing page and saw that the landing page attempted to render content and only failed to load the browse files section.
3. I navigated to the history tab and verified that it showed an exception about failing to query commit information.
4. I restored the repository working state to function properly.
5. I navigated to a repository's landing page and verified it loaded properly, including showing the last modified date for each file.
6. I navigated to the Code, Branches, Tags, and History tabs to verify each tab page loaded properly.
7. I verified on the Code tab that the last modified date for each file displayed properly.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T13666
Differential Revision: https://secure.phabricator.com/D21717
Summary:
Depends on D21720. Ref T13666. See D21720 for additional discussion.
Use "MethodCallFuture", introduced in D21720, so that exceptions raised in "execute()" are thrown when the future is resolved, not when the future is created.
This makes exception behavior for clustered and non-clustered setups consistent, and chooses the intended (clustered) behavior in both cases, which currently deals with errors better.
Test Plan:
- Applied both parts of the patch in T13666 (break history queries, force immediate futures) to reproduce the issue in T13666.
- Loaded a Diffusion landing page, reproduced the error described in that task.
- Applied this patch, verified landing page works again.
- Removed the "break history queries" change, verified landing page works in forced-immediate mode.
- Removed the "force immediate" change, verified landing page works in "actual future" mode.
Reviewers: cspeckmim
Reviewed By: cspeckmim
Maniphest Tasks: T13666
Differential Revision: https://secure.phabricator.com/D21721
Summary:
This diff adds conduit methods for searching for legalpad documents and signatures. This is very helpful for auditing who's actually signed a document. It also fixes the "contributorPHIDs" constraint in the existing search engine.
In order to expose legalpad signatures through Conduit, this adds a `phid` column to the `legalpad_documentsignature` table. It includes a migration (in the style of many previous phid-adding migrations) to actually populate the column.
Test Plan: We run this on my company's internal fork and it seems to work okay. I don't think any other conduit methods anywhere have tests (???), but if you can point me at one I'm glad to write a unit test!
Reviewers: O1 Blessed Committers, speck
Reviewed By: O1 Blessed Committers, speck
Subscribers: 20after4, speck, tobiaswiese
Differential Revision: https://we.phorge.it/D25018
Summary: Fixes T13663. `supportsSubtypes` tries to create an editable object, but this isn't always valid for `PhabricatorCalendarImport`. Use `instanceof` instead.
Test Plan:
- Edited calendar import, tasks (2 different subtypes), and projects (2 different subtypes).
- Changed task subtypes using {nav Change Subtype} action and batch editor.
- Changed task and project subtypes using Conduit.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Maniphest Tasks: T13663
Differential Revision: https://secure.phabricator.com/D21714
Summary:
Fixes T13662. Phriction currently passes a map as a "context object", but this code is ancient and predates the modern meaning of a "context object". In modern code, context objects should be real objects.
Provide a real object as a context object. We do this by either loading the actual document or constructing a synthetic version of it.
Test Plan:
- Edited an existing document, observing the preview:
- Used a mention rule, saw a preview.
- Used `[[ a ]]` and `[[ ./a ]]` absolute and relative reference rules, saw accurate previews.
- Edited a new document, observing the preview:
- Used a mention rule, saw a preview.
- Used absolute/relative references, saw accurate previews.
- Grepped for other references to the removed properties (`phriction.isPreview` and `phriction.slug`), found none remaining.
Reviewers: 0
Reviewed By: 0
Maniphest Tasks: T13662
Differential Revision: https://secure.phabricator.com/D21709
Summary:
Ref T13662. I ran into this while trying to reproduce the mention issue discussed there.
Currently, the root document (with slug "/") attempts to preview using the URI `/phriction/preview//` (with two `//` at the end). This is collapsed into "/phriction/preview/" by Apache if "MergeSlashes On" is configured, which is the default behavior. The route then 404s.
Instead, just use "/phriction/preview/?slug=/" so this endpoint functions properly regardless of the "MergeSlashes" configuration.
Test Plan:
- Configured Apache with "MergeSlashes On" (which is the default behavior).
- Tried to preview a content edit of the root document in Phriction, which didn't work and generated 404s for "/phriction/preview//" in the console log.
- Applied patch.
- Previwed content in Phriction (which now worked properly).
- Accessed `/a//b///c////` and similar with "MergeSlashes On" and "MergeSlashes Off", confirmed that this option controls whether PHP receives a URI with or without merged slashes in "__path__" after rewriting.
Reviewers: 0
Reviewed By: 0
Maniphest Tasks: T13662
Differential Revision: https://secure.phabricator.com/D21708
Summary:
See PHI498. This should be initialized to "self::ATTACHABLE" like other attachable properties, but is currently initialized to "array()".
Initialize it the normal way and try to catch all code paths which may have accessed it without actually loading and attaching it.
Also, remove UI for the very old "excuse" property, which "arc" has not written for well more than a year.
Test Plan: Grepped for affected symbols, loaded various revision pages. Somewhat tricky to be 100% sure that every pathway is caught, but it should be obvious if I missed anything once someone hits the code path.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D21710
Summary:
Fixes T13648. If a package recipient has been destroyed, this query currently fails to return an expanded recipient value.
Instead, make sure all input PHIDs get an output. For destroyed packages, the output will just be an empty list.
Test Plan:
- Added a package to a revision as a reviewer.
- Destroyed the package.
- Commented on the revision.
- Processed the publishing worker with `bin/worker execute`.
- Before: fatal after expanding the destroyed package.
- After: clean publish.
Maniphest Tasks: T13648
Differential Revision: https://secure.phabricator.com/D21707
Summary: Ref T13072. This exception is now raised by all of the message-sending code. Pretty straight find/replace.
Test Plan: Grepped for old class name, no hits.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21699
Summary: Ref T13072. Expand the role of "harbormaster.sendmessage" and allow it to send control messages to Builds and Buildables.
Test Plan: Read documentation, sent commands to Builds and Buildables, hit a bunch of error cases, will deploy to catch full-lifecycle Build Target use cases.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21698
Summary: Ref T13072. Make large Conduit doc pages a bit more navigable. This prepares for updating "harbormaster.sendmessage" to support sending messages to builds.
Test Plan: Viewed various Conduit API documentation pages, clicked links.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21696
Summary: Ref T13072. These don't do anything useful yet, but get the skeletons in.
Test Plan: Loaded documentation pages without fataling.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21695
Summary: Ref T13072. Trivially convert this into a modular transaction type.
Test Plan: Issued commands to a buildable.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21694
Summary: Ref T13072. This transaction type has no writers and is mooted by EditEngine.
Test Plan: Grepped for transaction constant.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21693
Summary: Ref T13072. Update the last few constant references to this class and remove it.
Test Plan: Grepped for "HarbormasterBuildCommand", got no hits.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21692
Summary: Ref T13072. Push nearly all Harbormaster build message logic into the new per-message transaction classes.
Test Plan:
- Issued every message to Buildables.
- Issued every message to Builds.
- Looked at a big pile of error messages, couldn't find any typos.
- Grepped for affected symbols, etc.
- Ran `bin/harbormaster restart ...`.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21691
Summary:
Ref T13072. Further modularize build messages by applying each one in a separate transaction type.
This makes it easier to add new types of messages (although I have no particular plans to do this, offhand) and reduces the amount of switch-boilerplate.
This will probably also simplify validating "harbormaster.sendmessage".
Test Plan:
- Applied all commands.
- Ran migration, saw transactions render properly
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21690
Summary: Ref T13072. Update "HarbormasterBuild" to use modern modular transactions.
Test Plan:
- Aborted, restarted, paused, and resumed a build.
- Used `bin/harbormaster restart`.
- Grepped for use of old "::TYPE_COMMAND" constant, didn't find any hits.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21689
Summary: Ref T13072. No callers currently generate these transactions, and they probably never have. Remove them.
Test Plan: Grepped for "HarbormasterBuildTransaction::TYPE_CREATE" and "self::TYPE_CREATE" in the class, found no hits.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21688
Summary:
Ref T13072. Currently, Harbormaster builds react to messages by applying a transaction inline (which can race) that has no real effect.
Later, the BuildEngine picks up the mesasge and applies a real effect, but this isn't transactional.
This is backwards, and makes it more difficult to transition to ModularTransaction and EditEngine. The desired workflow is:
- sending a message //just// writes to the message table (and queues a worker to process the message);
- the BuildEngine processes the message and applies effects in a transactional way.
Force this into at least roughly the right sequence of behaviors. This paves the way for porting to ModularTransaction, which should allow further cleanup.
Test Plan: Paused, resumed, aborted, and restarted a build. Ran BuildWorkers to process the commands, saw builds update appropriately.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21687
Summary:
Ref T13072. Currently, Builds have basic states (like "passed" and "failed") and pending states where a command has been issued but not yet executed (pausing, resuming, restarting, and aborting).
These are handled in a bit of an ad-hoc way, and not everything treats them the same way. In particular, the build page can concurrently report a build as "Aborting" and "Pausing", with different icons and colors.
Make everything use the same logic so that a Build can only be in exactly zero or one pending state, and use the same icons and colors.
Also tighten up which transitions are allowed: for example, it doesn't make sense to pause an aborting build.
The tighter rules don't all produce great UX right now (like "You can't pause this build.", when it would be better as "You can't pause a build which is already aborting." or similar), but just leave that alone for now.
Test Plan: Viewed builds, applied various state changes, ran BuildWorker to effect the state changes, grepped for affected methods, tried to issue various out-of-sequence build commands.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21685
Summary:
Ref T13072. These two similar tables don't make sense to keep separate. Instead, make Build a valid receiver for BuildMessage objects.
These tables are practically the same, so this is straightforward: just copy the rows in and then drop the old table.
(This table was trivial and ephemeral anyway, so I'm not bothering to do the usual "keep it around for a couple years just in case".)
Test Plan:
- Populated BuildCommand table, ran migration, saw Builds end up in the proper transitional state (e.g., pausing, aborting, restarting) with appropriate queued messages.
- Queued new messages by clicking UI buttons.
- Ran BuildWorkers, saw them process messages and mark them as consumed.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21684
Summary:
Ref T13072. Rename various "command" properties to "message" properties, to prepare for merging "HarbormasterCommand" into "HarbormasterMessage".
This change only renames variables and methods and should not affect program behavior.
Test Plan: Grepped for affected symbols, found no unmodified hits.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13072
Differential Revision: https://secure.phabricator.com/D21683
Summary: Ref T13660. Clean up callsites to "PhutilExecPassthru->execute()" to prepare to deprecate it.
Test Plan:
- Grepped for "PhutilExecPassthru" and looked for callsites.
- Ran `GIT_SSH=.../ssh-connect git ls-remote origin` to execute the "ssh-connect" code.
- The two passthru future methods have no callers and could possibly be removed, but I'm just letting sleeping dogs lie for now.
Reviewers: cspeckmim
Reviewed By: cspeckmim
Maniphest Tasks: T13660
Differential Revision: https://secure.phabricator.com/D21703
Summary:
Refer to discussion on D21677#275541
Refs D21681 (arcanist changes)
Phabricator has several uses of the `--debug` flag being used with Mercurial. Use of this flag causes additional output to be added which Phabricator needs, however the behavior of `--debug` is not guaranteed to be stable, and in newer versions of Mercurial there have been additional output that has caused Phabricator to choke on parsing the output. This change removes several uses of `--debug` in favor of using `--template` with the `hg log` or `hg annotate` commands in combination with the `{p1.node}` or `{p2.node}` template format.
The use of `{p1node}` format in templates was added in [[ https://www.mercurial-scm.org/wiki/WhatsNew/Archive#Mercurial_2.4_.282012-11-01.29 | Mercurial 2.4 (2012) ]]. This format was deprecated in [[ https://www.mercurial-scm.org/wiki/WhatsNew#Mercurial_4.9_.282019-02-01.29 | Mercurial 4.9 (2019) ]] in favor of using `{p1.node}` format which is unclear when this new format was added (presumably earlier than Mercurial 4.9).
The use of `--template` with `hg annotate` is only officially supported in [[ https://www.mercurial-scm.org/wiki/Release4.6 | Mercurial 4.6 (2018) ]], though does appear to work in 4.5 but is not documented.
Since the `{p1node}` format was introduced in 2.4 this bumps the required version of `hg` to 2.4 (from 1.9). Since the `annotate --template` feature wasn't added until 4.6 (which is still fairly recent), the use of it is gated on a capability test, but still preferred for use where possible to avoid extraneous output from `--debug` flag.
Test Plan:
I verified I could do the following in a mercurial repository, while having mercurial 5.8 installed:
1. Navigate and view files in Diffusion under e.g. `/source/test-repo/`.
2. While viewing a file in Diffusion verified that I could view the blame of the file and the history/annotations looked accurate for the files I was browsing.
3. From the blame sidebar, select to view a commit which loaded and displayed changes properly.
4. View the history of the repository under e.g. `/source/test-repo/history/default/`. I verified the history looked correct and the tree-like structure showing relationship of commits also looked accurate.
I setup mercurial to run version 4.4, created a new repository, added some commits, and verified all the above behavior still works properly.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D21679
Summary:
Ref T13658. This just scrubs some of the simple references from the codebase.
Most of what's left is in documentation which won't be relevant for a fork and/or which I need to separately revise (or more-or-less delete) at some point anyway.
I removed the "install RHEL" and "install Ubuntu" scripts outright since I don't have any reasonable way to test them and don't plan to maintain them.
Test Plan: Grepped for "phacility", "epriestley"; ran unit tests.
Reviewers: cspeckmim
Reviewed By: cspeckmim
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21678
Summary:
With newer versions of Mercurial come newer debug messages which need filtered out.
1. In the scenario of Phabricator observing a hosted Mercurial repository which exists on a server in a multi-user environment it's possible that a repository computes branch cache at a tip revision which is not present. When this happens Mercurial will include in the debug output this information. This message indicates that the cache is going to be re-computed. See https://www.mercurial-scm.org/pipermail/mercurial/2014-June/047239.html.
2. Likely in some version with added or improved support for `pager` the debug info seems to indicate when a pager is being invoked for a command. This seems to print out regularly despite piping the stdout.
3. If the repository on Phabricator ever had the `largefiles` extension enabled then some additional details about "updated patterns" will print out.
Test Plan:
I verified an observed repository's history could be browsed, specifically the history of files which previously resulted in "Undefined offset: 1".
Added a unit test to check the results of `filterMercurialDebugOutput()`.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D21677
Summary: Found a few typos which could be updated.
Test Plan:
I tested the Configuration page change by navigating to `/config` and verifying the page title set in the browser as well as the page title text on the page
|Before|After|
|---|---|
|{F9013208}|{F9013210}|
|{F9013300}|{F9013301}|
I verified the Conduit error message by navigating to `/auth/start/?__conduit__=1`
{F9013289}
The CircleCI error message was not verified due to the involvement of testing with CircleCI however the change is very minor and has very little risk of impacting any functionality.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21675
Summary: See T13657. An install has "watcher" packages which should not allow owners to "Force Accept" other packages.
Test Plan:
- Created package A, which I own, on "/", with "Weak" authority.
- Created package B, which I do not own, on "/src".
- Created a revision which touches "/src" and added package B as a reviewer.
- Attempted to accept the revision...
- Before patch: permitted to "Force Accept" for package B.
- After patch: not allowed to "Force Accept" for package B.
- Verified that setting package "A" back to "Strong" authority allows a force-accept for package B.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D21674
Summary: This makes the set of hooks easily extensible, as a first step toward integrating more 3rd party CI in phorge.
Test Plan: Send requests to `/harbormaster/hook/circleci/` and `/harbormaster/hook/buildkite/` and check they run the proper handler.
Reviewers: O1 Blessed Committers, Matthew
Reviewed By: O1 Blessed Committers, Matthew
Subscribers: Matthew, speck, tobiaswiese
Maniphest Tasks: T15018
Differential Revision: https://we.phorge.it/D25005
Summary: Ref T13614. Provide "bin/repository lock" to temporarily lock repositories for manual maintenance.
Test Plan:
- Read instructions.
- Used `bin/repository lock` according to the instructions.
- Saw Storage tab in Diffusion report lock held during maintenance, released after it completes.
- Saw "maintenance" push log generated and repository version bump.
- Tried to lock some invalid repositories.
Maniphest Tasks: T13614
Differential Revision: https://secure.phabricator.com/D21671
Summary:
Ref T13614. When a script holds the write lock but modifies the repository directly (rather than by pushing), the repository version won't change when the script releases the write lock. Thus, the writes may not propagate to other nodes (it depends which node lucks out and accepts the next write).
To guarantee that writes propagate, allow these scripts to pretend they pushed the repository. These are bare-bones valid events flagged as "Maintenance".
Test Plan:
- Wrote a script to hold the write lock, wait (or pretend to do something), then release the write lock.
- Applied patches, modified script to use new APIs ("newMaintenanceEvent()").
- Ran script, saw repository verison bump and relevant push logs:
{F8814923}
Maniphest Tasks: T13614
Differential Revision: https://secure.phabricator.com/D21670
Summary:
Ref T13614. When an omnipotent user calls "synchronizeWorkingCopyBeforeWrite()", we record a WorkingCopyVersion record with a null "userPHID". The UI then renders this as "Unknown Object (????)".
Improve this behavior:
- When no PHID is available, just render nothing in the UI (this doesn't seem meaningfully different from no version existing at all).
- Allow callers to provide an acting user PHID, similar to Editor.
There's currently no way to perform this kind of write legitimately in the upstream, but T13614 is providing one.
Test Plan:
- Wrote a script that calls "synchronizeWorkingCopyBeforeWrite()" as the omnipotent user.
- Ran script, saw "Unknown Object (????)" in the UI.
- Applied UI fix, saw empty UI.
- Applied "acting as" fix, modified script to act as the Diffusion application, ran script, saw "Diffusion" attribution in UI.
{F8814806}
Maniphest Tasks: T13614
Differential Revision: https://secure.phabricator.com/D21669
Summary:
Ref T13650. Currently, viewing the API console help page for this method fatals because it constructs a generic, untyped panel.
As a step toward improving this, generate a concrete panel type instead. This isn't the best possible fix, see T13650 for discussion.
Test Plan: Viewed "dashboard.panel.edit" API page, now saw a usable page.
Maniphest Tasks: T13650
Differential Revision: https://secure.phabricator.com/D21668
Summary:
As backstory: I accidentally added the subscriber `PHID-USER-abcd` to `T1` on this install by calling `maniphest.edit`. I intended to edit `T1` on my local install.
This edit is permitted for messy technical reasons, described in T13429. It's not valid, but it's hard to prevent.
The state we reach is also possible even if the edit is rejected (i.e., someone can go manually update the database).
Regardless of how we get into this state, the state (a non-user subscriber) breaks the UI on the task page when it attempts to test if the subscriber can see the task.
To prevent this, only claim that a Handle can have capabilities if the handle is complete. If the handle is incomplete (an invalid or restricted object), it either can't be meaningfully tested for capabilities or the viewer isn't allowed to know them.
Test Plan:
Viewed `T1` on this install, saw a fatal. Applied the same edit to `T1` locally, got the same fatal. Applied patch, no more fatal. Now saw "Unknown Object (User)" in subscriber curtain.
Specifically, the fatal is:
> Attempting to test capability "view" for handle of type "USER", but this capability has not been attached.
Differential Revision: https://secure.phabricator.com/D21662
Summary:
Ref T13559. If you create comments A and B, then delete comments A and B, then undo the deletion of A, the UI undoes the deletion of B instead.
This is becasue the undo rows are shipped down with a static scalar metadata reference. When copied multiple times to create multiple undo rows, they reference the same data object.
Preventing this in the general case is a problem with greater scope. For now, just avoid rendering these rows with any metadata so they don't alias a single data object.
Test Plan:
- Created comments A, B.
- Deleted comments A, B.
- Clicked "Undo" on A.
- Before: Deletion of "B" undone.
- After: Deletion of "A" undone.
Maniphest Tasks: T13559
Differential Revision: https://secure.phabricator.com/D21650
Summary:
Ref T13647. The ref discovery process prunes commits that no longer exist in the repository before executing "git log <new heads> --not <old heads>" to identify newly published commits.
If we don't do this, the "git log" command will fail if any old head has been pruned from the repository.
Currently, this test for missing commits starts with a call to "git for-each-ref" to attempt to resolve symbols as tag or branch names, but:
- this is painfully slow in repositories with many refs; and
- this is incorrect (not consistent with "git" behavior) for 40-character hex strings, which Git will never resolve as symbolic names.
Instead, when a symbol is a 40-character hex string, skip "git for-each-ref" and jump directly to "git cat-file --batch-check".
Test Plan:
- Ran `bin/repository update` in a repository with 65K refs and extra debugging info.
- Before: took ~30s, three calls to `git for-each-ref`.
- After: took ~20s, two calls to `git for-each-ref`. Same resolution result on queries.
Maniphest Tasks: T13647
Differential Revision: https://secure.phabricator.com/D21658
Summary: Ref T13641. These conditions are swapped, and "activeBindings" loads more data than necessary while "bindings" doesn't load enough.
Test Plan: Called method with each attachment, got good results instead of an exception.
Maniphest Tasks: T13641
Differential Revision: https://secure.phabricator.com/D21657
Summary:
Ref T13644. Ref T13638.
- Double-encode the symbol that is used as a path component, similar to Diffusion.
- Fix an outdated reference to ".path", which provided context for symbol lookup.
- Prevent command-clicking headers from looking up the path as a symbol.
Test Plan:
- Command-clicked headers, no longer got a symbol.
- Command-clicked stuff with "/", saw it double-encoded and decoded properly.
- Command-clicked normal symbols, saw "path" populate correctly.
Maniphest Tasks: T13644, T13638
Differential Revision: https://secure.phabricator.com/D21641
Summary:
Ref T9764. These stars are inconsistent, not accessible, and generally weird. They predate icons.
Update them to use icons instead.
Test Plan:
{F8545721}
{F8545722}
{F8545723}
Maniphest Tasks: T9764
Differential Revision: https://secure.phabricator.com/D21640
Summary:
Ref T9764. These "star" icons are unclear, inconsistent, and not friendly to colorblind users.
They date from a time long ago when the product didn't have icons.
Modernize them and make them more consistent with the similar statuses in Harbormaster.
Test Plan:
{F8545690}
{F8545691}
{F8545692}
Maniphest Tasks: T9764
Differential Revision: https://secure.phabricator.com/D21639
Summary: Ref T9499. When using the manual "Update Diff" workflow on the web, the "Repository" field isn't pre-filled properly. This can lead to revisions losing their repository after a manual update.
Test Plan: Did a manual update of a revision with a repository, saw it stick.
Maniphest Tasks: T9499
Differential Revision: https://secure.phabricator.com/D21638
Summary:
See PHI2032. The Conduit UI shows some "Example Custom Constraints" that are intended to be generic, but use of "statuses" is misleading since many objects have a status and most of them don't support these specific values.
Make it more clear that these are generic values.
Test Plan: Read new text.
Differential Revision: https://secure.phabricator.com/D21637
Summary: Ref T13641. Now that we can provide an "Active Devices" query, provide it and make it the default.
Test Plan: Viewed Almanac devices, got a list of active devices. Clicked "All Devices" to get all devices.
Maniphest Tasks: T13641
Differential Revision: https://secure.phabricator.com/D21636
Summary:
Ref T13641. Phabricator sometimes makes intracluster requests that authenticate as a device.
Forbid these requests from authenticating as a disabled device.
Test Plan:
- Ran `bin/ssh-exec --phabricator-ssh-device ...` as an enabled/disabled device (worked; sensible error).
- Made Conduit calls as an enable/disabled device (worked; sensible error).
Maniphest Tasks: T13641
Differential Revision: https://secure.phabricator.com/D21635
Summary: Ref T13065. Migrate "mailKey" and drop the column.
Test Plan: Ran "bin/storage upgrade", got a clean upgrade, saw migrated values in mail properties table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13065
Differential Revision: https://secure.phabricator.com/D21634
Summary: Ref T13065. Migrate "mailKey" and drop the column.
Test Plan: Ran "bin/storage upgrade", got a clean report, saw migrated data in mail properties table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13065
Differential Revision: https://secure.phabricator.com/D21633
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.
Test Plan: Ran "bin/storage upgrade", got a clean report, saw keys migrate in mail properties table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13641, T13065
Differential Revision: https://secure.phabricator.com/D21632
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.
Test Plan: Ran "bin/storage upgrade", got a clean report, saw mail keys migrated to mail properties table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13641, T13065
Differential Revision: https://secure.phabricator.com/D21631
Summary: Ref T13065. Ref T13641. Migrate "mailKey" and drop the column.
Test Plan: Ran "bin/storage upgrade", got a clean report and saw binding mail keys in the mail properties table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13641, T13065
Differential Revision: https://secure.phabricator.com/D21630
Summary: Ref T13641. Ref T13065. Migrate and drop the onboard "mailKey" column for Almanac Services.
Test Plan: Ran storage migration, got clean report, saw migrated value in mail properties table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13641, T13065
Differential Revision: https://secure.phabricator.com/D21629
Summary: Ref T13641. Make "active bindings" a real query and make callers that only care about active bindings only query for active bindings.
Test Plan:
- Queried for "bindings" and "activeBindings" via Conduit.
- Disabled/enabled devices, saw binding status update in UI.
- Loaded Diffusion cluster layout.
- Grepped for `needBindings()`, `getActiveBindings()`, etc.
Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13641
Differential Revision: https://secure.phabricator.com/D21628
Summary:
Ref T13641. Add a "status" property with most of the relevant support code.
This currently has no impact on use of the device or bindings by Diffusion or Drydock: they ignore the status of devices bound to services.
Test Plan:
- Created a new device.
- Changed the status of a device via web and API.
- Queried for devices via API.
- Searched for active and disabled devices.
- Viewed UI in list view, detail view.
- Used typeahead to add a new binding to an interface on a disabled device, got disabled hint in typeahead UI.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13641
Differential Revision: https://secure.phabricator.com/D21627
Summary: Ref T13641. Provide minor modernizations before adding a "Disabled" state.
Test Plan: Browsed devices, created a new device.
Maniphest Tasks: T13641
Differential Revision: https://secure.phabricator.com/D21626
Summary: Ref T13065. See similar changes attached to that task.
Test Plan: Ran migration, got a clean database state, saw mail keys populate in mail property table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13065, T13641
Differential Revision: https://secure.phabricator.com/D21625
Summary:
Ref T13395. "libphutil/" was stripped for parts, but some documentation still references it. This is mostly minor corrections, but:
- Removes "Javelin at Facebook", long obsolete.
- Removes "php FPM warmup", which was always a prototype and is obsoleted by PHP preloading in recent PHP.
Test Plan: `grep` / reading
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D21624
Summary: Ref T13639. Move Diffusion to use the new API and get rid of the old API now that it no longer has any callers.
Test Plan:
Grepped for remaining callers.
{F8539335}
Maniphest Tasks: T13639
Differential Revision: https://secure.phabricator.com/D21620
Summary:
Ref T13639. There's currently a hard-to-hit bug where editing the "Repository" of a revision doesn't update this index.
Instead: update the index on repository change, not just diff update.
Test Plan:
- Updated a revision, used debug view to see index update.
- Changed repository on a revision, used debug view to see index update.
Maniphest Tasks: T13639
Differential Revision: https://secure.phabricator.com/D21618
Summary:
Ref T13639. Make schema changes:
- Make repositoryID nullable, for revisions with no repository.
- Remove "epoch", which has no readers and no clear use.
- Change the ordering of the key, since "pathID" has more unique values and no queries ever issue without it.
Test Plan:
- Ran `bin/storage upgrade`, got a clean schema.
- Reindexed all revisions with an external script.
- Reviewed index via debug UI, saw appropriate index for non-repositoy revisions.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13639
Differential Revision: https://secure.phabricator.com/D21617
Summary:
Ref T13639. Move operations related to updating the "AffectedPath" index to a dedicated class.
This change has no functional effect and only moves code.
Test Plan: Used an external script to rebuild every revision index; destroyed a revision with `bin/remove destroy`.
Maniphest Tasks: T13639
Differential Revision: https://secure.phabricator.com/D21616
Summary:
Ref T13639. The "Affected Path" table is currently hard to inspect: there's no UI, and using MySQL just gives you a bunch of IDs.
Add a simple UI and a debug-mode link to it.
Test Plan:
{F8539098}
{F8539099}
Maniphest Tasks: T13639
Differential Revision: https://secure.phabricator.com/D21615
Summary:
Ref T13639. Updating the affected path table has a peculiar side effect from D19426, which is a simplification of a peculiar side effect from earlier.
Don't condition Owners behavior on path index behavior.
Test Plan: Created a revision.
Maniphest Tasks: T13639
Differential Revision: https://secure.phabricator.com/D21614
Summary: See T13639. This change simplifies providing a more modern approach to querying this data via "differential.revision.search".
Test Plan: Called "differential.query" with paths (got an error) and without paths (got a valid query result).
Differential Revision: https://secure.phabricator.com/D21613
Summary:
Ref T13639. In D17754, this:
> OPEN REVISIONS
> Recently updated open revisions affecting this file.
...was simplified into:
> RECENTLY OPEN REVISIONS
This is a bit misleading, since the panel doesn't contain "recently open" results. Use "Recent Open" instead, which is a bit more consistent with other product text. This is still slightly misleading, but probably close enough.
Test Plan: Read text.
Maniphest Tasks: T13639
Differential Revision: https://secure.phabricator.com/D21612
Summary:
Ref T13631. This option has a behavior other than the behavior implied by the name and documented.
Document the correct behavior, at least. This can likely be removed after T10574.
Test Plan: Read config option help in Config.
Maniphest Tasks: T13631
Differential Revision: https://secure.phabricator.com/D21610
Summary: Ref T13623. In D21603, I made the "partial object" this query returns a raw row, which paging keys can no longer be extracted from properly.
Test Plan: Paged notifications to page 2, no longer saw an error.
Maniphest Tasks: T13623
Differential Revision: https://secure.phabricator.com/D21609
Summary: Ref T13636. Add routing for "/robots.txt", "favicon.ico", and "/status/" on the ShortSite and BlogSite.
Test Plan: Visted all resources (and 404 pages) on Short and Blog sites, and Platform site.
Maniphest Tasks: T13636
Differential Revision: https://secure.phabricator.com/D21607
Summary:
Fixes T12919. Fixes T13636. Prior to this change, some well-known resource paths don't route on sites like ResourceSite.
- `/robots.txt`: Make it route on ResourceSite and just deny the whole site.
- `/favicon.ico`: Make it route on ResourceSite.
- `/status/`: Make it route on ResourceSite.
- 404: Make it render a 404 on ResourceSite.
Test Plan:
- Visited all URIs on ResourceSite, got sensible responses.
- Visited all URIs on main site.
- Visited 404 while logged out, got login page.
Maniphest Tasks: T13636, T12919
Differential Revision: https://secure.phabricator.com/D21606
Summary:
Ref T13635. Currently, the JSON DocumentEngine uses "phutil_json_decode()", but this can confuse "{}" and "[]".
Be more careful about how the JSON value is decoded, to preserve the distinction.
Test Plan: {F8520479}
Maniphest Tasks: T13635
Differential Revision: https://secure.phabricator.com/D21605
Summary:
Ref T13623. The change in D21577 could lead to a case where we try to access stories the user can't see.
Move the story-loading piece to "willFilterPage()" to make our way thorugh this.
Test Plan:
- Made FeedStory return nothing to simulate invisible notifications, loaded page.
- Before: index access fatal.
- After: clean "no notifications".
- Loaded notifications normally, saw normal notifications.
Maniphest Tasks: T13623
Differential Revision: https://secure.phabricator.com/D21603
Summary: Ref T13632. Users searching for `__FILE__`, etc., almost certainly mean to perform a substring search.
Test Plan: Added tests and made them pass. Searched for various tokens, saw compiler interpretation in UI.
Maniphest Tasks: T13632
Differential Revision: https://secure.phabricator.com/D21602
Summary:
Ref T13631. This supports a more robust version of "poll for updates by using dateModified window queries" that uses transactions as a logical clock.
This is particularly relevant for commits, since they don't have a "dateModified" at time of writing.
Test Plan:
- Queried for transactions by type and object.
- Issued various invalid transaction queries, got appropriate errors.
Maniphest Tasks: T13631
Differential Revision: https://secure.phabricator.com/D21601
Summary: Ref T13631. These strings were a little inconsistent; make them more consistent.
Test Plan: Called `diffusion.commit.search` with the appropriate attachment, saw slightly more consistent statuses.
Maniphest Tasks: T13631
Differential Revision: https://secure.phabricator.com/D21600
Summary:
Ref T13631. Move "PhabricatorAuditStatusConstants" to a more modern object ("PhabricatorAuditRequestStatus").
Expose the status value via Conduit.
Test Plan:
- Ran `bin/audit delete`.
- Viewed a commit with auditors in the web UI.
- Grepped for affected symbols.
- Called Conduit with the "auditors" attachment, saw auditor statuses.
Maniphest Tasks: T13631
Differential Revision: https://secure.phabricator.com/D21599
Summary:
Ref T13631. See that task for discussion.
- "NONE": Probably never used?
- "CC": Obsoleted by subscribers.
- "AUDIT_NOT_REQUIRED": For Owners packages, obsoleted by edges.
- "CLOSED": For "Close Audit", obsoleted by "Request Verification".
Test Plan:
- Grepped for constants, browsed Diffusion.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13631
Differential Revision: https://secure.phabricator.com/D21598
Summary: Ref T13631. For now, this only shows the auditor PHID. The current status constants could use some cleanup before they're exposed.
Test Plan: Queried with "auditors" attachment, saw basic auditor information.
Maniphest Tasks: T13631
Differential Revision: https://secure.phabricator.com/D21597
Summary:
Ref T13629.
- Allow files to have custom alt text.
- If a file doesn't have alt text, try to generate a plausible default alt text with the information we have.
Test Plan:
- Viewed image files in DocumentEngine diffs, files, `{Fxxx}` embeds, and lightboxes.
- Saw default alt text in all cases, or custom alt text if provided.
- Set, modified, and removed file alt text. Viewed timeline and feed.
- Pulled alt text with "conduit.search".
Maniphest Tasks: T13629
Differential Revision: https://secure.phabricator.com/D21596
Summary:
Ref T13628. When creating a revision, hide the recently introduced "Author" field.
It seems exceedingly unlikely that anyone would ever want to adjust this field when creating a revision, and this workflow already has a lot of fields and complexity, so this likely adds more noise than signal on the balance.
Test Plan:
- Created revisions via copy/paste on the web workflow, no longer saw "Author".
- Used "Edit Revision" to adjust the author normally.
Maniphest Tasks: T13628
Differential Revision: https://secure.phabricator.com/D21595
Summary:
See PHI2015. Diffusion attempts to prevent a commit's author from being made an auditor, but currently uses an out-of-date method for identifying the author.
Use the modern ("Repository Identity" aware) method instead.
Test Plan:
- Authored a commit as user "X", mapped to my account.
- Pushed/imported/discovered it.
- Changed the identity mapping for "X" from my account to a different account.
- Tried to add myself as an auditor.
- Before: error, "author can't be an auditor".
- After: succeeds.
- Tried to add the newly mapped user as an auditor. This correctly fails with the "author can't be an auditor" error.
It's possible to put commits into a wonky state by remapping the author identity to a user who is already an auditor, but I think that isn't important and we can't do much about it, realistically.
Differential Revision: https://secure.phabricator.com/D21594
Summary:
See PHI2015. Currently, "resigned" reviewers/auditors get different icons in Differential and Diffusion.
The Diffusion icon is exceptionally poor and confusing: it does not communicate "resigned" and it is similar to other icons.
For clarity and consistency, use the Differential icon (a grey "X") in both applications.
Test Plan: {F8492303}
Differential Revision: https://secure.phabricator.com/D21593
Summary:
Ref T13628. If you "Foist Upon", then delete the value in the tokenizer, then scroll down to the preview, you currently get an ugly "Unknown Object (???)" rendering.
This is technically correct in some vague sense and the transaction won't apply, but provide a more aesthetic rendering instead.
Test Plan: {F8487050}
Maniphest Tasks: T13628
Differential Revision: https://secure.phabricator.com/D21592
Summary:
Ref T13628. Currently, Differential has a "Commandeer" action, but no way to edit the author otherwise.
This is largely archaic: there is no reason to prevent editing the author, and this makes it difficult to undo mistakes if you commandeer by accident.
Instead, provide a normal "Author" field and a "Foist Upon" action, similar to the "Owner" and "Claim/Assign" fields in Maniphest.
Test Plan:
- Applied author transactions as the old author ("foisted"), the new author ("commandeerd"), and an arbitrary third party ("changed author").
- Tried to unassign author, etc.
- Viewed stories in feed and transaction timeline.
- Saw sensible automatic reviewer changes.
- Used existing "Commandeer" action, which is unchanged.
- Called "transaction.search" and saw reasonable transaction values.
Maniphest Tasks: T13628
Differential Revision: https://secure.phabricator.com/D21591
Summary: T13578
Test Plan:
This method uses the existing transaction. As such, most of the testing focused on the integration between the workflow and transaction. The only change made to the transaction was to allow an omnipotent user to make the change in addition to an admin.
Other than that, I removed the "approved" flag from the user, then ran the command-line utilty until the user was successfully approved.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T13578
Differential Revision: https://secure.phabricator.com/D21587
Summary: Ref T13627. This makes the API harder to misuse: setting an external connection on a held lock isn't a meaningful operation. Prevent it.
Test Plan: Added a failing test, made it pass.
Maniphest Tasks: T13627
Differential Revision: https://secure.phabricator.com/D21584
Summary:
Ref T13625. See that task for discussion.
Currently, the Viewer when performing revision updates in response to commits may be an arbitrary low-privilege user (an Application, a disabled User, a bot, a mailing list, etc).
Today, this leads to an exception when trying to make API calls.
Ideally, we probably would not perform the update in these cases. However, performing the update isn't a policy violation and is generally less surprising than not performing it, so continue performing it for now: just use the omnipotent user to interact with the API.
Test Plan:
- Authored a commit as a bot user without permission to view the repository or revision.
- Commented out a couple of caches, and used `bin/repository reparse --publish ...` to republish the commit.
- Before: exception when trying to interact with the API.
- After: clean publish.
Maniphest Tasks: T13625
Differential Revision: https://secure.phabricator.com/D21582
Summary:
Ref T13623. When paginating notifications, we may currently construct a query which:
- loads from non-unique rows; and
- returns multiple results.
In particular, `chronologicalKey` isn't unique across the whole table (only for a given viewer). We can get away with this because no user-facing view of notifications is truly "every notification for every viewer" today.
One fix would be to implicitly force the paging query to include `withUserPHIDs(viewerPHID)`, but puruse a slightly more general fix:
- Load only unique stories.
- Explictly limit the pagination subquery to one result.
Test Plan:
- Set page size to 1, inserted duplicate notifications of all stories for another user, clicked "Next", got the GROUP BY error.
- Applied the "only load unique stories" part of the change, got a "expected one row" error instead.
- Applied the "limit 1" part of the change, got a second page of notifications.
Maniphest Tasks: T13623
Differential Revision: https://secure.phabricator.com/D21577
Summary: Ref T13624. Depends on D21578. In "sshd" subprocess contexts, use "PhutilErrorLog" to direct errors to both stderr and, if configured, a logfile on disk.
Test Plan:
- Confiugured an error log.
- Forced `ssh-auth` to fatal.
- Saw errors on stderr and in log.
Maniphest Tasks: T13624
Differential Revision: https://secure.phabricator.com/D21579
Summary: Ref T13611. This property worked correctly when implemented in D19357. The behavior was broken by D20775, which tested node-level routing but did not specifically re-test the "writable" property. This was difficult to spot because ref query outcomes weren't observable in the UI, and the ref itself had the correct property value.
Test Plan:
See D21575. After this change, the UI shows the correct state, rather than showing a read-only service ref as writable:
{F8465865}
Maniphest Tasks: T13611
Differential Revision: https://secure.phabricator.com/D21576
Summary: Ref T13611. Currently, the "writable" property on service bindings has no effect because of a trivial bug. Provide more information in the UI to make this kind of problem observable.
Test Plan:
Viewed "Storage" section of management UI, saw a more-obvious problem with ref management (a non-writable ref is listed as writable).
{F8465851}
Maniphest Tasks: T13611
Differential Revision: https://secure.phabricator.com/D21575
Summary:
Ref T13620.
- Make generic edge stories render links with hovercards. Other story types (like subscriptions) already do this so I'm fairly certain this is just old code from before hovercards.
- Include a longer commit message snippet in hovercards.
Test Plan: {F8465645}
Maniphest Tasks: T13620
Differential Revision: https://secure.phabricator.com/D21574
Summary:
Ref T13617. When an inline comment is added inside a block of added lines, it currently ends up off-by-one when porting forward.
This is a disagreement between the mapping engine and the display engine about what "offset" means. Choose the simpler of the two interpretations.
Test Plan:
- Created a revision with the diff in T13617.
- Added an inline in the middle of the added block.
- Updated the revision with the same diff.
- Before: inline incorrectly moves up by one line.
- After: inline maps correctly.
Maniphest Tasks: T13617
Differential Revision: https://secure.phabricator.com/D21572
Summary: Ref T13586. In the footsteps of D21563, make Herald rule results more formal and structured to support meaningful exception reporting.
Test Plan:
Ran various Herald rules and viewed transcripts, including rules with recursive dependencies and condition exceptions.
{F8447894}
Maniphest Tasks: T13586
Differential Revision: https://secure.phabricator.com/D21565
Summary: Ref T13586. Lift the behavioral core of "HeraldConditionResult" into a new abstract base "HeraldTranscriptResult", with the intent to introduce a "HeraldRuleResult".
Test Plan:
- Ran Herald rules, reviewed transcripts.
- This change should have no behavioral effect.
Maniphest Tasks: T13586
Differential Revision: https://secure.phabricator.com/D21564
Summary:
Ref T13586. Currently, Herald condition logs encode "pass" or "fail" robustly, "forbidden" through a sort of awkward side channel, and can not properly encode "invalid" or "exception" outcomes.
Structure the condition log so results are represented unambiguously and all possible outcomes (pass, fail, forbidden, invalid, exception) are clearly encoded.
Test Plan:
{F8446102}
{F8446103}
Maniphest Tasks: T13586
Differential Revision: https://secure.phabricator.com/D21563
Summary: Ref T13586. The Herald transcript page has become more and more complicated over time, and recently added "Transactions" and "Profiler" sections. Split these across separate navigation tabs to limit the maximum complexity of any single view and make it easier to navigate to particular sections, like the profiler section.
Test Plan: Viewed various transcripts, saw nice digestible sections.
Maniphest Tasks: T13586
Differential Revision: https://secure.phabricator.com/D21493
Summary: Ref T13615. This property was removed from the Facebook API at some point, perhaps November 2020. Stop relying no it.
Test Plan: Created a local Facebook OAuth app, registered a new account locally.
Maniphest Tasks: T13615
Differential Revision: https://secure.phabricator.com/D21571
Summary: Ref T13609. Add the Object PHID (object being built), Container PHID (container of the object being built), Build PHID, and Buildable PHID to Harbormaster build variables.
Test Plan:
{F8448191}
{F8448192}
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13609
Differential Revision: https://secure.phabricator.com/D21569
Summary:
Ref T13587. D21495 has significant changes to the ngram indexer, which might possibly contain bugs.
Make it easier to reindex a subset of documents (based on the date when the index was built, and/or the software version which generated the index).
This is in addition to the existing versioning, which is focused on object versions.
Test Plan: Ran `bin/search index` with various old and new arguments. Spot-checked the `IndexVersion` table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13587
Differential Revision: https://secure.phabricator.com/D21560
Summary:
Ref T13587. Currently, when a document is reindexed by Ferret, the old document is completely discarded and a new version is inserted to replace it.
This approach is simple to implement, but can lead to exhaustion of the ngram AUTO_INCREMENT id column in reasonable circumstances.
Conceptually, this approach "should" be fine and this exhaustion is an awkard implementation detail. However, since it's easy to be less wasteful when performing document updates and all the other approaches are awkward or leaky in other ways that are probably worse, use a more complex implementation to avoid executing unnecessary INSERT statements.
Test Plan:
- Created and indexed a new document, searched for it.
- Updated a document, indexed it with `bin/search index ... --force --trace`, saw only modifications updated in the index.
- Searched for newly added terms (got hits) and removed terms (no longer got hits) to verify add/delete index behavior.
Maniphest Tasks: T13587
Differential Revision: https://secure.phabricator.com/D21495
Summary: Ref T13607. Add some time-oriented constraints to this API method to support compiling build statistics.
Test Plan:
- Called "harbormaster.target.search" with all new constraints.
- Viewed documentation in API console.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13607
Differential Revision: https://secure.phabricator.com/D21559
Summary: Ref T13605. Support selecting a diff's changesets (to get a list of affected file paths) via the API.
Test Plan: Called API with no arguments, diffPHIDs, PHIDs, IDs. Got sensible output.
Maniphest Tasks: T13605
Differential Revision: https://secure.phabricator.com/D21558
Summary:
Ref T13605. Changesets currently have no PHID, which limits their ability to use standard API infrastructure.
Give them a PHID, since there's no reason they don't have one other than their age.
Test Plan:
- Ran migrations, saw PHIDs populated.
- Created new changesets, saw PHIDs.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13605
Differential Revision: https://secure.phabricator.com/D21557
Summary: Ref T13602. Warn when a reivison has at least one human reviewer, no non-human reviewers, and no human reviewers can view it.
Test Plan: {F8430683}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21556
Summary: Ref T13602. Similar to subscriber and mention treatments, make it clear when a user doesn't have view permission.
Test Plan: {F8430595}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21555
Summary:
Ref T13602. When rendering a user hovercard, pass the object on which the reference appears. If the user can't see the object, make it clear on the hovecard.
Restyle the "nopermission" markup in mentions to make it more obvious what the style means: instead of grey text, use red with an explicit icon.
Test Plan: {F8430398}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21554
Summary:
Ref T13602. Currently, Hovercards are functions only of the object they represent (and the viewer, etc).
Recent changes to how users who can't see an object are rendered motivate making them a function of both the object they represent //and// the context in which they are being viewed. In particular, this enables a hovecard for a user to explain "This user can't see the thing you're lookign at right now.", so visual "exiled" markers can have a path forward toward discovery.
Test Plan:
- This change isn't expected to affect any behavior.
- Viewed hovercards, moused over/out, resized windows, viewed standalone cards, viewed debug cards, saw no behavioral changes.
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21553
Summary:
Ref T13602. Currently, timeline comment rendering does not (by default) propagate the context object to the rendering layer.
This means that `@mentions` of users who can't see the object aren't rendered properly (currently: they show up as blue, but should show up as grey).
Pass the context down the stack and into the remarkup engine.
Test Plan: {F8382905}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21548
Summary:
Ref T13602. When a subscriber can't see an object, it's currently hard to figure it out.
Show this status clearly in the curtain UI.
Test Plan: {F8382865}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21547
Summary:
Ref T13602. Currently, the policy framework can not execute "test if many users can see one object" particluarly efficiently. This test must be executed more broadly to implement the changes in T13602.
To avoid making this any worse than it already is, lift this block into a wrapper class that has a bulk queue + fetch API and could eventually be optimized.
Test Plan: Viewed a task with an `@mention` of a user without permission to see it in the summary, saw it rendered in a disabled style.
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21546
Summary:
Ref T13600. When migrating observed repositories between cluster services, impact can be better controlled by fetching a copy of the repository on the target host before clusterizing it.
In particular, in the Phacility cluster, migrations are generally from one shared shard to one dedicated shard. It's helpful to perform these migrations synchronously without waiting for the cluster to sync in the background (helpful in the sense that there are fewer steps and fewer commands to run).
This supports an "--observe" mode to the internal "bin/services load-repository" workflow, which transfers repository data by refetching it from the remote rather than by getting it from the older host. This fetch occurs before cluster configuration is adjusted.
Test Plan: Ran locally as a sanity check, will apply in production.
Maniphest Tasks: T13600
Differential Revision: https://secure.phabricator.com/D21544
Summary:
See <https://discourse.phabricator-community.org/t/i-cant-create-almanac-space/4424/>.
Almanac namespaces have never really had property support, but they implemented the interface in the original implementation.
At the time, this had no effect. Later changes integrated properties into the edit flows and broke this no-op integration.
Remove the interface for now. They could be given property support later, but need a bit of support code.
This feature is very rarely used and primarily useful for Phacility instances.
Test Plan: Created new namespaces and edited namespaces, browsed namespace UI.
Differential Revision: https://secure.phabricator.com/D21543
Summary:
Ref T13588. This has never been meaningful, but a "final private" method is specifically forbidden in PHP8.
Remove meaningless "final" from these methods, per new lint checks.
Test Plan: Ran `arc lint --everything` to identify affected methods, then `... | xargs -n1 arc lint --apply-patches`.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21540
Summary: Ref T13591. This is a minor consistency change to use PHIDs instead of IDs in the commit import processing pipeline. PHIDs are generally more powerful in more contexts and it would be unusual for a modern worker to use an ID here.
Test Plan:
- Made the "accept either ID or PHID" part of the change only.
- Pushed a commit, parsed and reparsed it step by step (this tests that "commitID" tasks can still process normally).
- Made the "write PHIDs" part of the change.
- Pushed a commit, parsed and reparsed it step by step.
- Looked at the task row in the database, saw PHID data.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21533
Summary:
Ref T13591. Improve how parameters are passed between commit worker tasks:
- Always pass "via", to track where tasks came from.
- Always provide "objectPHID" (with the commit PHID).
- Always provide "containerPHID" (with the repository PHID).
Test Plan:
- Pushed a new commit.
- Ran `bin/repository pull` + `bin/repository discover`, saw commit with all parameters.
- Ran `bin/worker execute ...`, saw a Change worker and then a Publish worker with appropriate parameters.
- Ran `bin/repository reparse ... --background`, saw workers queue with appropriate parameters.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21532
Summary:
Ref T13596. See that task for discussion. Executing "INSERT ... SELECT" at default isolation levels requires more locking than executing "SELECT" + "INSERT" separately.
Decompose this "INSERT ... SELECT" into "SELECT + INSERT", and reformat it to execute a minimal set of changes instead of wiping everything out and then writing all of it back. In most cases, this means we write 1 row instead of `O(number of project members)` rows.
Test Plan:
- Created a project. Added and removed members, looked at database and saw a consistent membership/materialization list.
- Created a subproject. Added and removed members, looked at database and saw a consistent membership/materialization list.
I wasn't successful in reproducing the LOCK WAIT issue locally by trying various concurrent SELECT / INSERT / INSERT ... SELECT strategies. It may depend on the "DELETE + INSERT ... SELECT" structure used here, or versions/config/etc, so we'll have to see how that fares in production.
Maniphest Tasks: T13596
Differential Revision: https://secure.phabricator.com/D21527
Summary:
See PHI1983. Ref T13599. Ref T13589. Currently, if you browse to a path browse URI in Diffusion without a trailing slash (`/browse/master/src`), you get a nonsensical view (the directory as a single item).
Be more precise in how "git ls-tree" arguments are constructed.
Test Plan: Visited files and directories in the browse view, with and without trailing slashes. Saw improved behavior for directories with no trailing slash and reasonable behavior in all other cases.
Maniphest Tasks: T13599, T13589
Differential Revision: https://secure.phabricator.com/D21528
Summary:
Ref T13590. By default, PHP kills execution after web scripts run for 30 seconds. If this occurs in the locked section of a repository write while we're holding the durable write lock, the lock will get stuck.
Use "set_time_limit(0)" to prevent this mechanism from interrupting execution while the durable lock is held.
Test Plan:
- Added "set_time_limit(1)" before the lock and "while (1);" in the critical section of the lock.
- Pushed, got the lock stuck.
- Cleared the lock, applied this patch, pushed.
- Got an infinite hang instead. (Normally, we expect the script to take more than 30 seconds to execute because there is a large push that executes in finite time, not because there's an infinte loop.)
Maniphest Tasks: T13590
Differential Revision: https://secure.phabricator.com/D21526
Summary:
Ref T13590. Currently, errors arising from cluster locking (like the "stuck write lock" exception) are not caught and converted into VCS responses on the HTTP VCS workflow.
Catch a broader range of exceptions and convert them into appropriate responses.
Test Plan:
- Forced a "stuck write lock" exception, pushed to a Git repository over HTTP.
- Before: generic fatal.
- After: VCS-specific fatal with a useful message in the "X-Phabricator-Message" response header.
Maniphest Tasks: T13590
Differential Revision: https://secure.phabricator.com/D21525
Summary:
Ref T13590. User objects have some inline caches that don't do readthrough generation by default because it may be indicative of high-impact performance problems in code.
During a VCS request, these caches are normally unnecessary, but they may be hit on some unusual pathways (like error handling).
Flag VCS users as okay for inline generation. This does not indicate a performance problem and access to these caches is very rare, at least today.
Test Plan:
- Executed a Git HTTP request which hit an unhandled exception (stuck write lock).
- Before: got a second-level exception while handling the first exception, when trying to access user preferences to render a standard uncaught exception page.
- After: no second-level exception.
Maniphest Tasks: T13590
Differential Revision: https://secure.phabricator.com/D21524
Summary:
Ref T13590. Currently, when you encounter a HTTP error in Git, there is no apparent way to make the client show any additional useful information. In particular, the response body is ignored.
We can partially get around this by putting the information in an "X-Phabricator-Message: ..." HTTP header, which is visible with "GIT_CURL_VERBOSE=1 git ...". Users won't normally know to look here, but it's still better than nothing.
Test Plan:
- Ran "GIT_CURL_VERBOSE=1 git fetch" against a Phabricator HTTP URI that returned a HTTP/500 error.
- Before: no clue what happened on the client.
- After: client shows useful message in the "X-Phabricator-Message" header in debug output.
Maniphest Tasks: T13590
Differential Revision: https://secure.phabricator.com/D21523
Summary:
Ref T13593. The commit cache in this Engine has a maximum fixed size (currently 65,535 entries).
If we execute discovery in a repository with more refs than this (e.g., 180K), we get fast lookups for the first 65,535 refs and slow lookups for the remaining refs.
Instead, divide the refs into chunks no larger than the cache size, and perform an explicit cache fill before each chunk is processed.
Test Plan:
- Created a repository with 1K refs. Set cache size to 256. Ran discovery.
- Before patch: saw one large cache fill and then ~750 single-gets.
- After patch: saw four large cache fills.
- Compared `bin/repository discover ... --verbose` output before and after patch for overall effect; saw no differences.
Maniphest Tasks: T13593
Differential Revision: https://secure.phabricator.com/D21521
Summary:
Ref T13595. See that task for discussion.
D21511 renamed the iteration variable here (previously "$path") but did not rename this use of it.
Test Plan:
- In Diffusion, browsed a directory with a submodule.
- Before: "setExternalURI()" fatal in conduit call.
- After: directory listing including submodule.
Maniphest Tasks: T13595
Differential Revision: https://secure.phabricator.com/D21520
Summary:
Ref T13589. The output for "git ls-tree commit:path" (the old invocation) and "git ls-tree commit -- path" (the new invocation) differs: the latter emits absolute paths.
Update the code to account for this difference in behavior.
Test Plan:
- Browsed a non-root directory in a Git repository in Diffusion.
- Before: saw absolute paths.
- After: saw relative paths.
Maniphest Tasks: T13589
Differential Revision: https://secure.phabricator.com/D21519
Summary: Ref T13591. Fixes a few issues with the recent updates here discovered in more thorough testing.
Test Plan:
- Stopped the daemons.
- Created a new copy of Phabricator in Diffusion.
- Pulled it with `bin/repository pull ...`.
- Got 17,278 commits on disk with `git log --all --format=%H`.
- Set permanent refs to "master".
- Discovered it with `bin/repository discover ...`.
- This took 31.5s and inserted 17,278 tasks.
- Verified that all tasks have priority 4,000 (PRIORITY_IMPORT).
- Observed that 16,799 commits have IMPORTED_PERMANENT and 479 commits do not.
- This matches `git log master --format=%H` exactly.
- Ran `bin/repository refs ...`. Expected no changes and saw no changes.
- Ran `bin/worker execute --active` for a minute or two. It processed all the impermanent changes first (since `bin/worker` is LIFO and these are supposed to process last).
- Ran `bin/repository refs`. Expected no changes and saw no changes.
- Marked all refs as permanent.
- Starting state: 16,009 message tasks, all at priority 4000.
- Ran `bin/repository refs`, expecting 479 new tasks at priority 4000.
- Saw count rise to 16,488 as expected.
- Saw all the new tasks have priority 4000 and all commits now have the IMPORTED_PERMANENT flag.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21518
Summary:
Ref T13591. There are currently two pathways to queue an import task for a commit: via repository discovery, or via a ref becoming permanent.
These pathways duplicate some logic and have behavioral differences: one does not set `objectPHID` properly, one does not set the priority correctly.
Unify these pathways, make them both set `objectPHID`, and make them both use the same priority logic.
Test Plan:
- Discovered refs.
- See later changes in this series for more complete test cases.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21516
Summary:
Ref T13591. Currently, the "IMPORTED_PERMANENT" flag (previously "IMPORTED_CLOSEABLE", until D21514) flag is set by using the result of "shouldPublishRef()".
This method returns the wrong value for the flag when there is a repository-level reason not to publish the ref (most commonly, because the repository is currently importing).
Although it's correct that commits should not be published in an importing repository, that's already handled in the "PublishWorker" by testing "shouldPublishCommit()". The "IMPORTED_PERMANENT" flag should only reflect whether a commit is reachable from a permanent ref or not.
- Move the relevant logic to a new method in Publisher.
- Fill "IMPORTED_PERMANENT" narrowly from "isPermanentRef()", rather than broadly from "shouldPublishRef()".
- Deduplicate some logic in "PhabricatorRepositoryRefEngine" which has the same intent as the logic in the Publisher.
Test Plan:
- Ran discovery on a new repository, saw permanent commits marked as permanent from the beginning.
- See later changes in this patch series for additional testing.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21515
Summary:
Ref T13591. This is an old flag with an old name, and there's an import bug because the outdated concept of "closable" is confusing two different behaviors.
This flag should mean only "is this commit reachable from a permanent ref?". Rename it to "IMPORTED_PERMANENT" to make that more clear.
Rename the "Unpublished" query to "Permanent" to make that more clear, as well.
Test Plan:
- Grepped for all affected symbols.
- Queried for all commmits, permament commits, and impermanent commits.
- Ran repository discovery.
- See also further changes in this change series for more extensive tests.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21514
Summary:
Ref T13591. Since D8781, this flag does not function correctly in Git and Mercurial repositories, since ref discovery pre-fills the cache.
Move the "don't look at the database" behavior the flag enables into the cache lookup. D8781 should have been slightly more aggressive and done this, it was just overlooked.
Test Plan:
- Ran `bin/repository discover --help` and read the updated help text.
- Ran `bin/repository discover --repair` in a fully-discovered Git repository.
- Before: no effect.
- After: full rediscovery.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21513
Summary:
Ref T13589. In D21510, not every ref selector got touched, and this isn't a valid construction in Git:
```
$ git ls-tree ... -- ''
```
Thus:
- Disambiguate more (all?) ref selectors.
- Correct the construction of "git ls-tree" when there is no path.
- Clean some stuff up: make the construction of some flags and arguments more explicit, get rid of a needless "%C", prefer "%Ls" over acrobatics, etc.
Test Plan: Browsed/updated a local Git repository. (This change is somewhat difficult to test exhaustively, as evidenced by the "ls-tree" issue in D21510.)
Maniphest Tasks: T13589
Differential Revision: https://secure.phabricator.com/D21511
Summary: Ref T13589. See that task for discussion.
Test Plan: Executed most commands via "bin/conduit" or in isolation.
Maniphest Tasks: T13589
Differential Revision: https://secure.phabricator.com/D21510
Summary: Ref T13575. Particularly with the new Apple silicon, I think there are enough domain collisions for `M1`, `M2`, `P1`, etc., to justify adding them to the default ignore list.
Test Plan: Created a mock, then wrote a comment referencing an object on the list (`M1`) and an object not on the list (`T1`). Got text and a link respectively.
Maniphest Tasks: T13575
Differential Revision: https://secure.phabricator.com/D21507
Summary: Without this change, the landing page for the Packages app is https://secure.phabricator.com/packages, which is a 404. There's probably a better way to fix this, but this was the fewest characters.
Test Plan: doitlive
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21494
Summary: Ref T13585. Provide a minimal but technically functional "harbormaster.step.edit" API method.
Test Plan: Used the web console to modify the URI for a "Make HTTP Request" build step.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13585
Differential Revision: https://secure.phabricator.com/D21489
Summary: Ref T13585. This isn't particularly useful (notably, it does not include custom field values and isn't searchable by build plan PHID) but get the basics into place.
Test Plan: Used the web UI to make API calls, reviewed results.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13585
Differential Revision: https://secure.phabricator.com/D21488
Summary:
Ref T2312. Numeric strings are read out of arrays as integers, and modern PHP raises appropriate warnings when they're then treated as strings.
For now, cast the keys to strings explicitly (we know we inserted only strings). In the future, introduction of a `StringMap` type or similar might be appropriate.
Test Plan:
- Added "abc.12345.xyz" to the blocklist, changed my VCS password.
- Before: fatal when trying to "strpos()" an integer.
- After: password change worked correctly.
Maniphest Tasks: T2312
Differential Revision: https://secure.phabricator.com/D21487
Summary:
Changes the heuristic method by which non-zero exit statuses from git-http-backend are found to be due to packfile negotiation during shallow fetches, etc.
Instead of checking git-http-backend stderr for a generic "hung up" error message, see if the pack-result response contains a terminating flush packet ("0000"). This should give a greater assurance that the request was handled correctly and the response is complete.
Test Plan: Run `GIT_CURL_VERBOSE=1 git fetch --depth 1 https://host.example/source/repo.git HEAD` to ensure it completes and includes two successful POST requests during packfile negotiation (the last one actually receives the packfile).
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, dzduvall
Tags: #diffusion
Differential Revision: https://secure.phabricator.com/D21484
Summary:
See <https://discourse.phabricator-community.org/t/mail-details-view-broken/4315>. The change in D21400 detects a missing "GROUP BY" in some variations of this query.
Specifically, we may join multiple recipient rows (since mail may have multiple recipients) and then fail to group the results.
Fix this by adding the "GROUP BY". Additionally, remove the special-cased behavior when no authors or recipients are specified -- it's complicated and not entirely correct (e.g., may produce a "no object" instead of a policy error when querying by ID), and likely predates overheating.
Test Plan:
- Disabled `metamta.one-mail-per-recipient` in Config.
- Generated a message to 2+ recipients.
- Viewed the message detail; queried for the message by specifying 2+ recipients.
- Viewed the unfiltered list of messages, saw the query overheat.
Differential Revision: https://secure.phabricator.com/D21486
Summary: See PHI1912. Ref T13491. "arc" now requires "--" when stdin is not a TTY; provide this argument for users.
Test Plan: Viewed example in console, saw "--". Executed example.
Maniphest Tasks: T13491
Differential Revision: https://secure.phabricator.com/D21482
Summary:
See PHI1900. Recent changes to how commit graphs are drawn made the height automatic in most cases, but it fails in Differential because the element isn't initially visible so the computed height is 0.
Just give them an explicit height so they show up again.
Test Plan: Viewed graphs in Maniphest, Differential, and Diffusion; saw them all render properly.
Differential Revision: https://secure.phabricator.com/D21481
Summary:
See PHI1901. An install would like improved support for identifying files related to an object (like a task or revision) for retention/archival/backup/migration/snapshotting purposes.
The "attachment" edge is not really user-level: it just means "if you can see the object, that allows you to see the file". This set includes files that users may not think of as "attached", like thumbnails and internal objects which are attached for technical reasons.
However, this is generally an appropriate relationship to expose for retention purposes.
Test Plan: Used "edge.search" to find files attached to a revision and objects attached to a file.
Differential Revision: https://secure.phabricator.com/D21480
Summary: Ref T13583. To improve support for making it harder to improperly mix data retention policies, allow Herald to act on comment content.
Test Plan:
- Wrote comment content Herald rules in Maniphest and Differential.
- Submitted non-matching comments (no action) and matching comments (Herald action).
- In Differential, triggered rules by submitting non-matching main content and a matching inline comment.
Maniphest Tasks: T13583
Differential Revision: https://secure.phabricator.com/D21479
Summary:
Ref T13552. The behavior of "RepositoryQuery" with ambiguous identifiers under "withRepositoryPHIDs()" is tricky. This leads to failure to load commits in Subversion in some cases.
Use "withRepository()", which gives us the correct identifier resolution behavior.
Test Plan: Viewed a subversion repository history in Diffusion, saw commit details after change.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21469
Summary:
Ref T13552. The Herald field "Accepted Differential revision" (and similar fields) depend on the task/revision update steps running before Herald executes.
Herald currently executes first, so it never sees associated revisions. Swap this order.
Test Plan: Published a commit, got a clean parse/import. Will test with production rules ("Cowboy Commits").
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21468
Summary: See PHI1885. Repository operations are queryable by state and author, but neither column has a usable key. Add usable keys.
Test Plan: Ran EXPLAIN on a state query. Ran `bin/storage upgrade`. Ran EXPLAIN again, saw query go from a table scan to a `const` key lookup.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D21465
Summary:
Ref T13581. Currently, unexpected exceptions inside Conduit calls are passed to the client, but not logged on the server.
These exceptions should generally be unexpected, and producing a server-side trace is potentially useful.
Test Plan: Simulated a during-execution exception, saw it get logged on the server.
Maniphest Tasks: T13581
Differential Revision: https://secure.phabricator.com/D21464
Summary:
Ref T13581. If you query for revisions by hash and provide multiple hashes (A, B) which match a single revision (e.g., older and newer diffs for that revision), the query omits a GROUP BY clause but should contain one.
Add a GROUP BY clause in this case.
Test Plan:
With a working copy that has multiple hashes corresponding to a single revision, ran `arc branches` before and after the change. Before, got this error:
```
[2020-09-15 17:02:07] EXCEPTION: (ConduitClientException) ERR-CONDUIT-CORE: Rows passed to "loadAllFromArray(...)" include two or more rows with the same ID ("130"). Rows must have unique IDs. An underlying query may be missing a GROUP BY. at [<arcanist>/src/conduit/ConduitFuture.php:65]
```
After, clean execution.
Maniphest Tasks: T13581
Differential Revision: https://secure.phabricator.com/D21462
Summary:
Ref T13552. When a previously discovered commit becomes reachable from a permanent ref, we re-queue workers to update it. However, the commit may already be marked as "published", so the publish worker may do nothing.
It would perhaps be simpler to not mark the commit as published when it isn't reachable from a permanent ref, but this is tricky because the flag is also part of the "imported / all steps" state (see T13580).
Until that can be cleaned up, just clear the flag.
Test Plan:
- Pushed a commit with "fixes X" to a non-permanent branch.
- Pushed it to a permanent branch.
- Before change: task failed to close.
- After change: task closes properly.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21460
Summary:
Ref T13552. Now that these steps can build their own "CommitRef" object from storage on the "CommitData" object, move them from the "Message" step to the "Publishing" step.
This should resolve the root issue in T13552, where a commit moved from a non-permanent branch to a permanent branch does not publish closures properly.
Test Plan: Used "bin/repository reparse --publish ..." to republish changes.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21450
Summary:
Ref T13552. Turn "CommitData" into an application-level layer on top of the repository-level "CommitRef" object.
For older commits which will not have a "CommitRef" record on disk, build a synthetic one at runtime. This could eventually be migrated.
Test Plan: Ran "bin/repository reparse --message", browsed Diffusion.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21449
Summary: Ref T13552. Currently, various callers read raw properties off "CommitData" directly. Wrap these in accessors to support storage changes which persist "CommitRef" information instead.
Test Plan:
- Ran "diffusion.querycommits", saw the same data before and after.
- Looked at a commit, saw authorship information and date.
- Viewed tags in a repository, saw author information.
- Ran "rebuild-identities", saw no net effect.
- Grepped for callers to "getCommitDetail(...)".
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21448
Summary: Ref T13552. The internal caller for this now uses "internal.commit.search", which is always authority-reading. No legitimate external caller should rely on the behavior of "bypassCache"; no-op it to simplify behavior.
Test Plan: Called "diffusion.querycommits", saw the same data as before.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21447
Summary: Ref T13552. Swap the call we're using to build "CommitRef" objects here to the recently-introduced "internal.commit.search" method.
Test Plan: Used "bin/repository reparse --message ..." to reparse commits, added "var_dump()" to inspect results. Saw sensible CommitRef and CommitData objects get built.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21446
Summary:
Ref T13552. This is one of two callsites to "diffusion.querycommits". It's an old debugging workflow which I haven't used in years and which is likely obsoleted by identities and other changes.
I believe the root problem here was also ultimately user error (a user has misconfigured their local Git author email as another user).
Test Plan: Grepped for "lookup-users", got no hits.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21444
Summary:
Ref T13552. Commit parsers currently invoke a special mode of "diffusion.querycommits", which is an older frozen method.
The replacement, "diffusion.commit.search", is not really appropriate for low-level access. This mode of having a single method which operates in "cache" or "non-cache" modes also ends up in a lot of unnecessary field shuffling.
Provide "internal.commit.search" as a modern equivalent that returns a "DiffusionCommitRef"-compatible structure.
Test Plan: Executed "internal.commit.search", got sensible low-level commit results.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21443
Summary:
Ref T13552. Some Diffusion conduit calls may only be served by a node which hosts a working copy on disk, so they're proxied if received by a different node.
This capability is currently bound tightly to "DiffusionRequest", which is a bundle of context parameters used by some Diffusion calls. However, call proxying is not fundamentally a Diffusion behavior.
I want to perform proxying on a "*.search" call which does not use the "DiffusionRequest" parameter bundle. Lift proxying to the root level of Conduit.
Test Plan: Browsed diffusion in a clusterized repsository.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21442
Summary: Ref T13552. Neither "$hashes" or "$user" are used, and constructing them has no side effects.
Test Plan: Searched for these symbols.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21441
Summary:
Ref T13570. Fixes T13235. In most cases, we use modern (v4) signatures for almost all AWS API calls, and have for several years.
However, sending email via SES currently uses an older piece of external code which uses the older (v3) signature method.
AWS is retiring v3 signatures on October 1 2020, so this pathway will stop working.
Update the pathway to use `PhutilAWSFuture`, which provides v4 signatures.
T13235 discusses poor error messages from SES. Switching to Futures fixes this for free, as they have more useful error handling.
Test Plan:
- Configured an SES mailer, including the new `region` parameter.
- Used `bin/mail send-test` to send mail via SES.
- Sent invalid mail (from an unverified address); got a more useful error message.
- Grepped for removed external, no hits.
Maniphest Tasks: T13570, T13235
Differential Revision: https://secure.phabricator.com/D21461
Summary: Ref T13577. After the fix in D21453, lint identifies additional static errors in Phabricator; fix them.
Test Plan: Ran `arc lint`; these messages are essentially all very obscure.
Subscribers: hach-que, yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13577
Differential Revision: https://secure.phabricator.com/D21457
Summary:
Ref T13573. Using the browser "Print" feature on pages produces "Thu, Aug 4, 12:22" timestamps which require context to interpret precisely (they don't have a year and don't have a timezone).
Instead, retain these timestamps in "screen" contexts but use "YYYY-MM-DD HH:MM:SS (UTC+X)" timestamps when printing.
Test Plan: Printed Maniphest tasks and other pages in Safari and Chrome using "?__print__=1" and "Print to PDF", saw absolute timestamps after this chagne in the printed documents.
Maniphest Tasks: T13573
Differential Revision: https://secure.phabricator.com/D21451
Summary: Ref T13552. Provide a richer handle/status list item for commit lists.
Test Plan: Viewed commits in various interfaces, saw richer information.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21431
Summary:
Ref T13552. Build the "commit list" elements so that the menu action items collapse under the element on mobile.
Also change the mobile breakpoint to 512px because my Safari window can't go any narrower than 508px. Future changes to responsive design will be more content-aware anyway.
Test Plan: Looked at commits in various interfaces, at desktop and mobile widths.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21430
Summary:
Ref T13552. The current layout doesn't work particularly well on desktops or devices.
We have some device/desktop table layout code, but it isn't generic. We also have property list layout code, but it isn't generic either.
Provide generic layout elements ("Fuel", from "Phabricator UI Layout" to "PHUIL"?) and narrowly specialize their display behavior. Then swap the ListItemView stuff to use it.
Test Plan:
Saw slightly better responsive behavior:
{F7637457}
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21418
Summary:
Ref T13552. In unifying the various Graph/List/Table commit views, some information was dropped -- particularly, audit status.
Restore most of it. The result isn't very pretty, but has most of the required information.
Test Plan: {F7637411}
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21417
Summary: Ref T13552. Some of the CSS can be removed or simplified now that essentially all lists of commits are on a single rendering pathway.
Test Plan: Grepped for affected CSS, viewed commit graph.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21416
Summary: Ref T13552. Remove yet another way to render a list of commits, and unify it with "CommitGraphView".
Test Plan:
- Viewed commit search results.
- Viewed owners package detail page.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21415
Summary:
Ref T13552.
Currently, the "Browse" page shows a snippet of unmerged changes if you're looking at a non-default branch. Remove this for consistency with the simplified main "Browse" page. This is reachable via "Compare".
Update the "Compare" page to use the new "CommitGraphView".
Test Plan:
- Looked at the "Browse" page of "stable".
- Looked at the "Compare" page for "stable vs master".
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21414
Summary: Ref T13552. When viewing a merge commit, merged changes are currently shown inline. Update this view to use the new "GraphView" rendering pipeline.
Test Plan:
- Viewed a merge commit, saw merges.
- Viewed history, profile page, etc.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21413
Summary:
Ref T13552. This older view mostly duplicates other code and has only two callsites:
- The "Commits" section of user profile pages.
- The "Ambiguous Hash" page when you visit a commit hash page which is an ambiguous prefix of two or more commit hashes.
Replace both with "DiffusionCommitGraphView".
Test Plan:
- Visited profile page, clicked "Commits".
- Visited an ambiguous hash page (`rPbd3c23`).
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21412
Summary: Ref T13552. In the new combined "table/list" graph view, tidy up the graph rendering.
Test Plan: {F7633504}
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21411
Summary:
Ref T13552. Currently, commit lists are sometimes rendered as an object list and sometimes rendered as a table. There are two separate views for table rendering.
Add a fourth view ("list, with a graph") with the eventual intent of unifying all the other views. For now, this only replaces "HistoryListView" -- and needs some more work to really be a convincing replacement.
Test Plan:
- Looked at "History" in Diffusion, saw an ugly view with all the information we want.
- Grepped for "HistoryListView", no hits.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21410
Summary:
Ref T13552. Currently, Diffusion has two effectively identical history views, the "Graph" view and the "History" view.
These arose out of product uncertainty about the importance of the graph, but I think we can just put the graph on the "object item list" view and merge these views.
Test Plan: Looked at repositories in Diffusion, no longer saw a "Graph" tab. Grepped for "graph"-related symbols.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21409
Summary:
Ref T13552. Currently, the repository landing page has a panel with recent commits. This is accessible by clicking "History" and usually below the fold, so it's not clearly useful.
Since I'm consolidating this code anyway to fix an issue with the import pipeline, just get rid of this history view.
Test Plan: Viewed a repository landing page, no longer saw a history panel.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21408
Summary: Ref T13552. This older class has no callers; tag and branch listings were replaced with an "ObjectList" view.
Test Plan: Grepped for "DiffusionTagTableView", got no hits.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21407
Summary:
Ref T13552. I'm trying to reduce the number of direct callers to commit authorship metadata. This header seems low-value enough to simply remove; this information is shown more clearly and prominently in the "Provenance" UI.
In particular, commits have multiple dates (authored, committed, pushed) but this header shows only one. It currently shows the author identity and the commit date, which isn't entirely correct. And it potentially uses an "Identity" as a timeline actor, which is conceptually fine but not entirely firm ground.
Test Plan: Viewed a commit, saw no more subheader.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21406
Summary:
Ref T13552. Give "Commit" objects a more modern, identity-aware way to render author and committer information.
This uses handles in a more modern way and gives us a single read callsite for raw author and committer names.
Test Plan:
- Grepped for callers to the old methods, found none. (There are a lot of "renderAuthor()" callers in transactions, but this call takes no parameters.)
- Viewed some commits, saw sensible lists of authors and committers.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21405
Summary:
Ref T13552. When viewing a directory in Diffusion, we make an Ajax call to get the last commit for each path.
This call currently pulls author information, since an older version of this UI showed author information.
The current UI does not show author information, so this parameter is unused. Delete the code which builds it.
Test Plan: Grepped for `'author'` and references to the "pull-lastmodified" behavior. This behavior is invoked in only one place, which never generates an author placeholder.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21404
Summary:
See PHI1834. Currently, the inline adjustment engine can sometime "adjust" an inline off the end of a diff. If it does, we lay it out on an invalid display line here and never render it.
Instead, make sure that layout never puts a comment on an invalid line, so the UI is robust against questionable decisions by the adjustment engine: no adjustment should be able to accidentally discard an inline.
Test Plan:
- Created a two diff revision, where Diffs 1 and 2 have "alphabet.txt" with A-Z on one line each. The file is unchanged across diffs; some other file is changed.
- Added a comment to lines P-Z of Diff 1.
- Before: comment is adjusted out of range on Diff 2 and not shown in the UI.
- After: comment is still adjusted out of range internally, but now corrected into the display range and shown.
Differential Revision: https://secure.phabricator.com/D21435
Summary:
Ref PHI1835. Generally, Jupyter notebooks in the wild may store source and markdown content as either a single string or a list of strings.
Make the renderer read these formats more consistently. In particular, this fixes rendering of code blocks stored as a single string.
This also fixes an issue where cell labels were double-rendered in diff views.
Test Plan:
Created a notebook with a code block represented on disk as a single string, rendered a diff from it.
{F7696071}
Differential Revision: https://secure.phabricator.com/D21434
Summary:
See PHI1839. Currently, the "No newline at end of file" text is dropped in the 1-up diff view for changes that affect a file with no trailing newline.
Track it through the construction of diff primitivies more carefully.
Test Plan: {F7695760}
Differential Revision: https://secure.phabricator.com/D21433
Summary:
Although I'm not entirely thrilled about doing flow control like this (as an actual action in a build plan), I believe this build step works correctly and there's no fancy replacement mechanism on the immediate horizon, and this didn't send us down a slippery slope of Turing-complete builds encoded without real structure or context. Just kick it out of prototype.
(Other approaches which might be better in the long run are things like "this is a top-level behavior on the build plan itself" and/or "build plans are written in a DSL, not a Javascript UI".)
Test Plan: Added a new build step, saw this as an option in the "Flow Control" section.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D21432
Summary:
Reverts D21419. See PHI1814. Previously, I used "user-select: all" to group sequences of spaces for selection.
However, this has a side effect: the sequence is now selected with a single click. I didn't read the docuementation on the CSS property thoroughly and missed this in testing, since I was focused on drag-selection behavior.
This behavior is enough of a net negative that I think we're in a worse state overall; revert it.
Test Plan: Straight revert.
Differential Revision: https://secure.phabricator.com/D21429
Summary:
Ref T13556. These options are very old and effectively obsoleted by "bin/phd debug [--trace]". I haven't used either option diagnostically in many years, and they aren't mentioned in the documentation.
Remove them to simplify configuration, and because "phd.trace" doesn't work anyway and likely hasn't for a long time -- it has specific issues with TTY detection (see T13556).
Test Plan: Grepped for "phd.trace" and "phd.verbose". Ran "bin/phd debug [--trace]" and saw verbose/trace output.
Maniphest Tasks: T13556
Differential Revision: https://secure.phabricator.com/D21426
Summary: See PHI1819. This structure may have `null` elements.
Test Plan: Will confirm user reproduction case.
Differential Revision: https://secure.phabricator.com/D21420
Summary:
Ref T2495. See PHI1814. Currently, Phabricator replaces tabs with spaces when rendering diffs.
This may or may not be the best behavior in the long term, but it gives us more control over expansion of tabs than using tab literals.
However, one downside is that you can use your mouse cursor to select "half a tab", and can't use your mouse cursor to distinguish between tabs and spaces. Although you probably shouldn't be doing this, this behavior is less accurate/correct than selecting the entire block as a single unit.
A specific correctness issue with this behavior is that the entire block is copied to the clipboard as a tab literal if you select any of it, so two different visual selection ranges can produce the same clipboard content.
This particular behavior can be improved with "user-select: all", to instruct browsers to select the entire element as a single logical element. Now, selecting part of the tab selects the whole thing, as though it were really a tab literal.
(Some future change might abandon this approach and opt to use real tab literals with "tab-size" CSS, but we lose some ability to control alignment behavior if we do that and it doesn't have any obvious advantages over this approach other than cursor selection behavior.)
Test Plan:
- In Safari and Firefox, dragged text to select a whitespace-expanded tab literal. Saw browsers select the whole sequence as though it were a single tab.
- In Chorme, this also mostly works, but there's some glitchiness and flickering. I think this is still a net improvement, it's just not as smooth as Safari and Firefox.
Maniphest Tasks: T2495
Differential Revision: https://secure.phabricator.com/D21419
Summary:
See PHI1810. In situations where:
- An author submits an urgent change for review.
- The author pings reviewers to ask them to look at it.
...the reviewers may not be able to move the review forward if the review is currently a "Draft". They can only "Commandeer" or ask the author to "Request Review" as ways forward.
Although I'm hesitant to support review actions (particularly, "Accept") on draft revisions, I think there's no harm in allowing reviewers to skip tests and promote the revision out of draft as an explicit action.
Additionally, lightly specialize some of the transaction strings to distinguish between "request review from draft" and other state transitions.
Test Plan:
- As an author, used "Request Review" to promote a draft and to return a change to reviewers for consideration. These behaviors are unchanged, except "promote a draft" has different timeline text.
- As a non-author, used "Begin Review" to promote a draft.
Differential Revision: https://secure.phabricator.com/D21403
Summary:
Currently, adding subscribers to a draft revision raises a warning that they won't get an email/notification.
This warning has some false positives:
- it triggers on any subscriber change, including removing subscribers; and
- it triggers if you're only adding yourself as a subscriber.
Narrow the scope of the warning so it is raised only if you're adding a subscriber other than yourself.
Test Plan:
- Added a non-self subscriber, got the warning as before.
- Added self as a subscriber, no warning (previously: warning).
- Removed a subscriber, no warning (previously: warning).
Differential Revision: https://secure.phabricator.com/D21402
Summary:
See PHI1810. Build toward support for "Request Review" by non-authors on drafts, to forcefully pull a revision out of draft.
Currently, some action strings can't vary based on revision state or the current viewer, so this "pull out of draft" action would have to either: say "Request Review"; or be a totally separate action.
Neither seem great, so allow the labels and messages to vary based on the viewer and revision state.
Test Plan: Grepped for affected symbols, see followup changes.
Differential Revision: https://secure.phabricator.com/D21401
Summary:
Modern Mercurial may emit some more patterns under "--debug".
This whole list is gross and can likely now be eliminated by increasing the minimum required Mercurial version (as `arc` has), but just paper over it for now.
Test Plan:
Locally, saw some views return to functional behavior that weren't previously working on a modern version of Mercurial.
The reproduction case is likely something in the vein of "repository is not writable by webserver, look at history view".
Differential Revision: https://secure.phabricator.com/D21398
Summary:
See PHI1809. This query may join the "slug" table, but each project may have multiple slugs, and the query does not "GROUP BY" when this join occurs.
This may lead to partial result sets and unusual paging behavior.
This could likely be caught categorically in `loadAllFromArray()`; I'll adjust this in a followup.
Test Plan:
A minimal reproduction case is something like:
- Give project P slugs: a, b, c.
- Give project Q slugs: d.
- Query for slugs: a, b, c, d; with limit 2.
- Order the query so P returns first.
- Expect: P and Q.
- Actual: P generates 3 raw rows and the final result is just P with no pagination cursor.
Differential Revision: https://secure.phabricator.com/D21399
Summary:
A handful of Phacility production shards have run into memory pressure issues recently. Although there's no smoking gun, and at least two other plausible contributors, one possible concern is that the Fact daemon was written before hibernation and can not currently hibernate. Even if there's no memory leak, this creates unnecessary memory pressure by holding the processes in memory.
Allow the Fact daemon to hibernate, like other daemons do.
Test Plan: Ran "bin/phd debug fact", saw the Fact daemon hibernate.
Subscribers: yelirekim
Differential Revision: https://secure.phabricator.com/D21389
Summary: Ref T13546. Companion change to D21372. Move URI normalization code to Arcanist to we can more-often resolve remote URIs correctly.
Test Plan: Grepped for affected symbols.
Maniphest Tasks: T13546
Differential Revision: https://secure.phabricator.com/D21373
Summary:
See PHI1794, which reports an issue where a large number of queued webhook calls led to connection exhaustion. To make this easier to reproduce and test, add "--count" and "--background" flags to "bin/webhook call".
This primarily supports "bin/webook call ... --background --count 10000" to quickly fill the queue with a bunch of calls.
Test Plan: Ran `bin/webhook call` in foreground and background modes, with and without counts. Saw appropriate console and queue behavior.
Differential Revision: https://secure.phabricator.com/D21368
Summary:
The "Export Data" workflow incorrectly uses the "Policy Favorites" setting to choose a default export format. This is just a copy/paste error; the correct setting exists and is unused.
If the setting value is an array (as the "Policy Favorites" value often is), we try to use it as an array index. This generates a runtime exception after D21044.
```
[2020-06-16 06:32:12] EXCEPTION: (RuntimeException) Illegal offset type in isset or empty at [<arcanist>/src/error/PhutilErrorHandler.php:263]
#0 <#2> PhutilErrorHandler::handleError(integer, string, string, integer, array) called at [<phabricator>/src/applications/search/controller/PhabricatorApplicationSearchController.php:460]
```
- Use the correct setting.
- Make sure the value we read is a string.
Test Plan:
- Used "Export Data" with a nonempty, array-valued "Policy Favorites" setting.
- Before: runtime exception.
- After: clean export.
- Used "Export Data" again, saw my selection from the first time persisted.
Differential Revision: https://secure.phabricator.com/D21361
Summary: Ref T13546. This makes some "arc" tasks a little easier, and will make them more correct if "arc" ever switches to using SSH.
Test Plan: Ran "harbormaster.buildable.search" from the web UI, saw URIs in the result set.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13546
Differential Revision: https://secure.phabricator.com/D21346
Summary:
Currently, Phortune attempts to prevent users from removing themselves as account managers. It does this by checking that the new list includes them.
Usually this is sufficient, because you can't normally edit an account unless you're already a manager. However, we get the wrong result (incorrect rejection of the edit) if the actor is omnipotent and the acting user was not already a member.
It's okay to edit an account into a state which doesn't include you if you have permission to edit the account and aren't already a manager.
Specifically, this supports more formal tooling around staff modifications to billing accounts, where the actor has staff-omnipotence and the acting user is a staff member and only used for purposes of leaving a useful audit trail.
Test Plan: Elsewhere, ran staff tooling to modify accounts and was able to act as "alice" to add "bailey", even though "alice" was not herself a manager.
Differential Revision: https://secure.phabricator.com/D21288
Summary:
Ref T13513. An inline is not considered empty if it has a suggestion, but some of the shared transaction code doesn't test for this properly.
Update the shared transaction code to be aware that application comments may have more complex emptiness rules.
Test Plan:
- Posted an inline with only an edit suggestion, comment went through.
- Tried to post a normal empty comment, got an appropriate warning.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21287
Summary:
Ref T13541. The passthru future does not have time limit behavior, so if we reach this code we currently fail.
Phabricator never reaches this code normally, but this code is reachable during debugging if you try to foreground a slow fetch to inspect it.
Passthru commands generally only make sense to run interactively, and the caller or control script can enforce their own timeouts (usually by pressing "^C" with their fingers).
Test Plan: Used a debugging script to run ref-by-ref fetches in the foreground.
Maniphest Tasks: T13541
Differential Revision: https://secure.phabricator.com/D21284
Summary:
See PHI1752.
- Early exit of document layout can cause us to fail to populate available rows.
- Some Jupyter documents have "markdown" cells with plain strings, apparently.
Test Plan: Successfully rendered example diff from PHI1752.
Differential Revision: https://secure.phabricator.com/D21285
Summary:
Ref PHI1749. Instead of opening files to the last unchanged line on either side of the change, open files to the "simple" line number of the selected block.
For inlines, this is the inline line number.
For blocks, this is the first new-file line number, or the first old-file line number if no new-file line number exists in the block.
This may not always be what the user is hoping for (we can't know what the state of their working copy is) but should produce more obvious behavior.
Test Plan:
- In Diffusion, used "Open in Editor" with and without line selections. Saw same behavior as before.
- Used "n" and "r" to leave an inline with the keyboard, saw same behavior as before.
- Used "\" and "Open in Editor" menu item to open a file with:
- Nothing selected or changeset selected (line: 1).
- An inline selected (line: inline line).
- A block selected (line: first line in block, per above).
Differential Revision: https://secure.phabricator.com/D21282
Summary: Ref T13276. Ref T13513. All readers and writers were removed more than a year ago; clean up the last remnants of this table.
Test Plan: Grepped for table references, found none.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13513, T13276
Differential Revision: https://secure.phabricator.com/D21281
Summary:
Ref T13513. Syntax highlighting is potentially expensive, and the changeset rendering pipeline can cache it. However, the cache is currently keyed ONLY by Differential changeset ID.
Destroy the existing cache and rebuild it with a more standard cache key so it can be used in a more ad-hoc way by inline suggestion snippets.
Test Plan: Used Darkconsole, saw cache hits and no more inline syntax highlighting for changesets with many inlines.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21280
Summary: Ref T13513. Inline comment context information is somewhat expensive to construct and can be cached. Add a readthrough cache on top of it.
Test Plan: Loaded a source code changeset with many inline comments, used Darkconsole to inspect query activity. Saw caches get populated. Updated cache key, saw caches regenerate. Browsed Diffusion, nothing looked broken.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21279
Summary: Ref T13513. For now, I'm not supporting inline edit suggestions in Diffusion, although it's likely not difficult to do so in the future. Clean up some of the code so that plain ol' inlines work the same way they did before.
Test Plan:
- Created, edited, reloaded, submitted inlines in Diffusion: familiar old behavior.
- Created, edited, reloaded, submitted inlines with suggestions in Differential: familiar new behavior.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21278
Summary: Ref T13513. When rendering an inline suggestion for display, use highlighting and diffing.
Test Plan: {F7495053}
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21277
Summary:
Ref T13513. This still has quite a few rough edges and some significant performance isssues, but appears to mostly work.
Allow reviewers to "Suggest Edit" on an inline comment and provide replacement text for the highlighted source.
Test Plan: Created, edited, reloaded, and submitted inline comments in various states with and without suggestion text.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21276
Summary:
Ref T13513. If your 10 most recently authored inlines have all been deleted, these queries can fail by overheating. This is silly and probably rarely happens outside of development.
For now, just let them overheat. This may create a false negative (incorrect "no draft" signal when the real condition is "drafts, but 10 most recent comments were deleted"). This could be sorted out later with a query mode like "executeAny()", perhaps.
Test Plan:
- Created and deleted 10 inlines.
- Submitted comments.
- Before: overheating fatal during draft flag generation.
- After: clean submission.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21274
Summary: See PHI1745. This callsite for "ChangesetParser" was not properly updated for recent changes.
Test Plan:
- Set `metamta.differential.inline-patches` to 100.
- Created a new revision with a small (<100 line) diff, with at least one reviewer.
- Ran `bin/phd debug` and observed outbound mail queue with `bin/mail list-outbound`.
- Before: fatal when trying to generate the inline changes for mail.
- After: clean mail generation.
Differential Revision: https://secure.phabricator.com/D21270
Summary: See PHI1743. If a build has no initiator PHID, the rendering pathway incorrectly tries to access a handle for it anyway.
Test Plan:
- Set a build to have no initiator PHID.
- Viewed the build plan for the build.
- Before: fatal when trying to access the `null` handle.
- After: clean build plan rendering.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D21269
Summary:
Fixes T13539. See that task for discussion and a reproduction case.
This algorithm currently counts "\ No newline at end of file" lines as though they were normal source lines. This can cause offset issues in the rare case that a diff contains two of these lines (for each side of the file) and has changes between them (because the last line of the file was modified between the diffs).
Instead, don't count "\" as a display line.
Test Plan:
- See T13539 and PHI1740.
- Before: got fatals on the "wild" diff and the synthetic simplified version.
- After: clean intradiff rendering in both cases.
Maniphest Tasks: T13539
Differential Revision: https://secure.phabricator.com/D21267
Summary:
Ref T13538. See PHI1739. Synthetic Git commits with no author and/or no commit message currently extract `null` and then fail to parse.
Ideally, we would carefully distinguish between `null` and empty string. In practice, that requires significant schema changes (these columns are non-nullable and have indexing requirements) and these cases are degenerate. These commits are challenging to build and can not normally be constructed with `git commit`.
At least for now, merge the `null` cases into the empty string cases so we can survive import.
Test Plan:
- Constructed a commit with no author and no commit message using the approach described in T13538; pushed and parsed it.
- Before: fatals during identity selection and storing the commit message (both roughly NULL inserts into non-null columns).
- After: clean import.
This produces a less-than-ideal UI in Diffusion, but it doesn't break anything:
{F7492094}
Maniphest Tasks: T13538
Differential Revision: https://secure.phabricator.com/D21266
Summary:
Fixes T13536. See that task for discussion.
Older versions of MySQL (roughly, prior to 8.0.19) emit "int(10)" types. Newer versions emit "int" types. Accept these as equivalent.
Test Plan: Ran `bin/storage upgrade --force` against MySQL 8.0.11 and 8.0.20. Got clean adjustment lists on both versions.
Maniphest Tasks: T13536
Differential Revision: https://secure.phabricator.com/D21265
Summary: On the "New User" web workflow, if you use an invalid email address, you get a failure with an empty message.
Test Plan:
- Before: Tried to create a new user with address "asdf". Got no specific guidance.
- After: Got specific guidance about email address formatting and length.
Differential Revision: https://secure.phabricator.com/D21264
Summary:
Ref T13529. Now that instances can be renamed, an instance may have multiple valid SSH usernames and the preferred SSH username may not be the intenal instance name.
`PhacilitySiteSource` should already always set `diffusion.ssh-username` correctly, to the current preferred SSH username (which may be "new-name" after a rename from "old-name"), so we should never be able to reach this code without an accurate `diffusion.ssh-username` value available.
The code to resolve names into instances also already works for both "ssh old-name@..." and "ssh new-name@...".
So I believe this code has no beneficial effects and only causes harm: it may force us to return "old-name" when falling through would correctly return "new-name".
Test Plan:
- Previously: renamed an instance, then SSH'd to it using both the old and new names. Both work.
- Previously: verified that `diffusion.ssh-username` is set correctly after a rename.
- Verified that Diffusion "Clone" UI now shows "new-name" after an instance rename.
- The real question here is: does this break something I'm not thinking of? And the change probably has to go to production to answer that.
Maniphest Tasks: T13529
Differential Revision: https://secure.phabricator.com/D21259
Summary:
Ref T13513. The way I'm highlighting lines won't work for Jupyter notebooks or other complex content blocks, and I don't see an obvious way to make it work that's reasonably robust.
However, we can just ignore the range behavior for complex content and treat the entire block as selected. This isn't quite as fancy as the source behavior, but pretty good.
Also, adjust unified diff behavior to work correctly with highlighting and range selection.
Test Plan:
- Used range selection in a Jupyter notebook, got reasonable behavior (range is treated as "entire block").
- Used range selection in a unified diff, got equivalent behavior to 2-up diffs.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21257
Summary:
Ref T13513. Give selected inlines a selection state and visual cues which are similar to the changeset selection state.
Also fix a couple of minor issues with select interactions and offset comments.
Test Plan: Selected inlines, saw obvious visual cues.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21256
Summary:
Ref T13513. When a user selects a text range and uses "New Inline Comment" to create a comment directly from a range, store the offset information alongside the comment.
When hovering the comment, highlight the original range.
Test Plan: {F7480926, size=full}
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21250
Summary: Ref T13513. Support direct text selection for inlines. This is currently just an alternate way to get to the same place as using line numbers, but can preserve offset/range information in the future.
Test Plan:
- Selected some text, hit "c", clicked "New Inline Comment", got sensible comments on both sides of a diff in Safari, Chrome, and (with limitations) Firefox.
- Caveats: no unified support, doesn't work across lines in Firefox.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21248
Summary:
Ref T13454. See <https://discourse.phabricator-community.org/t/newly-created-ssh-private-keys-with-passphrase-not-working-anymore/3883>.
After changes to distinguish between invalid and passphrase-protected keys, SSH private key management code incorrectly uses "-y ..." ("print public key") when it means "-p ..." ("modify input file, removing passphrase"). This results in the command having no effect, and Passphrase stores the raw input credential, not the stripped version.
We can't recover the keys because we don't store the passphrase, so no migration here is really possible. (We could add more code to detect this case, but it's presumably rare.)
Also, correct the behavior of the "Show Public Key" action: this is available for users who can see the credential and does not require edit permission.
Test Plan:
- Created a new credential with a passphrase, then showed the public key.
Maniphest Tasks: T13006, T13454
Differential Revision: https://secure.phabricator.com/D21245
Summary:
Ref T13513. Currently, viewing a Jupyter document, hidden context just gets a plain "* * *" facade with no way to expand it.
Support click-to-expand, like source changes.
Test Plan:
- Clicked to expand various Jupyter diffs.
- Clicked to expand normal source changes.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21243
Summary:
Ref T13513. If you leave an inline on line 20 of a Jupyter document, we currently render context around *raw* line 20, which is inevitably some unrelated piece of JSON.
Instead, drop this context. (Ideal behavior would be to render context around Jupyter block 20, but that's a whole lot of work.)
Test Plan:
- On Jupyter changes and normal source changes, made and submitted inline comments, then viewed text and HTML mail.
- Saw no context on Jupyter comments (instead of bad context), and unchanged behavior (useful context) on normal source changes.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21242
Summary:
Ref T13513. Currently, "View as Document Type..." lists every available engine.
This is hard to get completely right because we can't always rebuild the document ref accurately in the endpoint, but try harder to fake something reasonable.
Test Plan: Used "View as Document Type..." on Jupyter notebooks, was given "Jupyter" and "Source" as options.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21241
Summary:
Ref T13513. As part of inline metadata, save the document engine the change is being rendered with.
This will allow other parts of the UI to detect that an inline was created on a Jupyter notebook but is being rendered on raw source, or whatever else.
The immediate goal is to fix nonsensical inline snippet rendering in email on Jupyter notebooks.
Test Plan:
- Created inlines and replies on normal soure code, saw no document engine annotated in the database.
- Created inlines and replies on a Jupyter notebook rendered in Jupyter mode, saw "jupyter" annotations in the database.
- Swapped document engines between Jupyter and Source, etc.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21240
Summary:
Ref T13513. If an intradiff has at least one unchanged file ("hasSameEffectAs()") or more than 100 files ("Large Change"), we hit this block and don't upcast storage inlines to runtime inlines. I missed this in testing.
Add the conversion step.
Test Plan: Viewed an intradiff with at least one unchanged file and at least one inline comment, saw correct rendering instead of fatal.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21239
Summary:
Ref T13513. Currently:
- If you click the "Show Changeset" button, your state change doesn't actually get saved on the server.
- It's hard to select a changeset path name for copy/paste because the "highlight the header" code tends to eat the event.
Instead: persist the former event; make the actual path text not be part of the highlight hitbox.
Test Plan:
- Clicked "Show Changeset", reloaded, saw changeset visibility persisted.
- Selected changeset path text without issues.
- Clicked non-text header area to select/deselect changesets.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21236
Summary:
Ref T13513. Currently, if you:
- click a line to create an inline;
- type some text;
- wait a moment; and
- close the page.
...you don't get an "Unsubmitted Draft" marker in the revision list.
Lift all the draft behavior to "InlineController" and make saving a draft dirty the overall container draft state.
Test Plan:
- Took the steps described above, got a draft state marker.
- Created, edited, submitted, etc., inlines in Diffusion and Differential.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21235
Summary: Ref T13513. All queries now go through a reasonably minimal set of pathways and should have consistent behavior.
Test Plan:
- Loaded a revision with inlines.
- Created a new empty inline, reloaded page, saw it vanish.
- Created a new empty inline, typed draft text, did not save, reloaded page, saw draft present.
- Created a new empty inline, typed draft text. Submitted feedback, got prompt, answered "Y", saw draft text submit.
- Created a new empty inline, typed draft text, scrolled down to bottom of page, typed non-draft text, saw preview include draft text.
- Marked and submitted "Done".
- Used hide/show on inlines, verified state persisted.
- Did much of the same stuff in Diffusion, where it all works the same way (except: there's no prompt when submitting draft is-editing inlines).
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21234
Summary: Ref T13513. Replaces "DifferentialInlineCommentQuery" with the similar but more modern "DifferentialDiffInlineCommentQuery".
Test Plan: Viewed comments in timeline, changesets. Created, edited, and submitted comments. Hid and un-hid comments, reloading (saw state preserved).
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21233
Summary: Ref T13513. Continue removing usage sites for the obsolete "DifferentialInlineCommentQuery".
Test Plan: Viewed the inline list in Differential, saw sensible inlines.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21232
Summary: Ref T13513. Move querying to "DiffInlineCommentQuery" classes and lift them into the base Controller.
Test Plan: In Differential and Diffusion, created, edited, and submitted inline comments.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21231
Summary: Ref T13513. Another step closer to the light.
Test Plan: Created, edited, deleted, replied to, and submitted inline comments in Diffusion.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21230
Summary: Ref T13513. Continue marching toward coherent query pathways for all access to inline comments.
Test Plan:
- Viewed a commit and a path within that commit, as a user with unpublished inlines and a different user.
- Saw appropriate inlines in all cases (published inlines, plus undeleted unpublished inlines authored by the current viewer).
- Grepped for "loadDraftAndPublishedComments()".
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21228
Summary:
Ref T13513. Improve consistency and robustness of the "InlineComment" queries.
The only real change here is that these queries now implicitly add a clause for selecting inlines ("pathID IS NULL" or "changesetID IS NULL").
Test Plan: Browed, created, edited, and submitted inlines.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21227
Summary:
Ref T13513. Currently, inline storage objects ("TransactionComment") can't directly generate a runtime object ("InlineComment").
Allow this transformation to be performed in a genric way so clunky code which does it per-object-type can be removed, lifted, or simplified.
Simplify an especially gross callsite in preview code.
Test Plan: Previewed inline comments.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21226
Summary: Ref T13513. See that task for some discussion. This prepares to lift "loadUnsubmittedInlineComments(...)" into shared code.
Test Plan: Grepped for callers, found none in the upstream. This is a backward compatibilty break. See T13513.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21225
Summary: Ref T13513. This controller was obsoleted by EditEngine and appears unreachable without explicitly typing the URL.
Test Plan:
- Grepped for the route, didn't find any hits.
- Deleted the controller, successfully previewed comments in Diffusion.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21224
Summary:
Ref T13519. This is a little fuzzy, but I think the workflow here is:
- View an intradiff, generating an ephemeral comparison changeset with no changeset ID. This produces a state key of "*".
- Apply "hidden" state changes to the changeset.
- View some other intradiff and/or diff view.
- The code attempts to use "*" as a changset ID?
I'm not entirely sure this is accurate; this was observed in production and I couldn't get a clean reproduction case locally.
Optimistically, try making changeset IDs explicit rather than relying on state keys to be "usually changeset-ID-like".
Test Plan: Used "hidden" locally across multiple intradiffs, but I wasn't cleanly able to reproduce the initial issue.
Maniphest Tasks: T13519
Differential Revision: https://secure.phabricator.com/D21223
Summary: Ref T13523. If a file hasn't been touched in the newer changeset, we can currently hit an error in the interdiff.
Test Plan:
- Touched "moo.txt" in Diff 1.
- Reverted the changes to "moo.txt" in Diff 2.
- Diffed 2 vs 1.
- Before patch: fatal (call to getFilename() on null).
- After patch: clean interdiff.
Maniphest Tasks: T13523
Differential Revision: https://secure.phabricator.com/D21220
Summary: Ref T13513. When users choose to publish inlines, we want to publish the visible text, not the last "checkpointed" state.
Test Plan:
- Created an inline ("AAA").
- Edited it into "BBB", did not save.
- Submitted.
- Confirmed that I want to publish the unsaved inline.
- Saw "BBB" publish.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21218
Summary:
Ref T13513. As users type text into inline comments, save the comment state as a draft on the server.
This has some rough edges, particularly around previews, but mostly works. See T13513 for notes.
Test Plan: Started an inline, typed some text, waited a second, reloaded the page, saw an editing inline with the saved text.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21216
Summary: Ref T13513. When computing whether a revision has draft comments or not, ignore empty inlines.
Test Plan: Added empty inlines to a revision, no longer saw a yellow "draft" bubble in the list UI.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21215
Summary: Ref T13513. When you load a changeset, discard all empty inlines. This is likely a more desirable behavior than keeping empty editors around, even though the rest of the pipeline generally handles them fairly well now.
Test Plan:
- Started an inline, didn't type any text or save, reloaded page.
- Before: page restores empty editor in the same place.
- After: we just discard this likely-pointless empty inline.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21214
Summary:
Ref T13513. Currently, if you start an inline and then submit overall comments, we publish an empty inline. This is literally faithful to what you did, but almost certainly not the intent.
Instead, simply ignore empty inlines at publishing time (and ignore "done" state changes for those comments).
We could delete them outright, but if we do, they'll break if you have another window open with the empty inline (since the stored comment won't exist anymore). At least for now, leave them in place.
Test Plan: Created empty inlines, submitted comments, no longer saw them publish.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21211
Summary: Ref T13513. This slightly expands the existing-but-hacky "warning" workflow to cover both "mentions on draft" and "submitting inlines being edited".
Test Plan:
- Submitted changes to a revision with mentions on a draft, inlines being edited, both, and neither.
- Got sensible warnings in the cases where warnings were appropriate.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21191
Summary:
Ref T13513. If you submit top-level comments while an inline comment editor is open, kick the comment out of the editing state.
(An improvement to this behavior would be to warn the user that we're going to do this first, but this is currently less straightforward.)
Test Plan:
- Clicked a line number to create an inline.
- Type text, save, click edit.
- (Optional: reload page.)
- Save changes overall using the form at the bottom of the page.
- Outcome: published inline is no longer in an "editing" state.
Weirdness:
- If you click a line number (and, optionally, type text), then submit without using "Save", the server-side version of the inline has no content.
- This gives you a no-effect warning. Instead, these inlines should probably just be marked as deleted somewhere in the pipeline.
- This saves the last "Saved" copy of the inline. That's (probably?) desired, but somewhat destructive without a warning.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21188
Summary:
Ref T13513. This is mostly an infrastructure cleanup change.
In a perfect world, this would be a series of several changes, but they're tightly interconnected and don't have an obvious clean, nontrivial partition (or, at least, I don't see one). Followup changes will exercise this code repeatedly and all of these individual mutations are "obviously good", so I'm not too worried about the breadth of this change.
---
Inline comments are stored as transaction comments in the `PhabricatorAuditTransactionComment` and `DifferentialTransactionComment` classes.
On top of these two storage classes sit `PhabricatorAuditInlineComment` and `DifferentialInlineComment`. Historically, these were an indirection layer over significantly different storage classes, but nowadays both storage classes look pretty similar and most of the logic is actually the same. Prior to this change, these two classes were about 80% copy/pastes of one another.
Part of the reason they're so copy/pastey is that they implement a parent `Interface`. They are the only classes which implement this interface, and the interface does not provide any correctness guarantees (the storage objects are not actually constrained by it).
To simplify this:
- Make `PhabricatorInlineCommentInterface` an abstract base class instead.
- Lift as much code out of the `Audit` and `Differential` subclasses as possible.
- Delete methods which no longer have callers, or have only trivial callers.
---
Inline comments have two `View` rendering classes, `DetailView` and `EditView`. They share very little code.
Partly, this is because `EditView` does not take an `$inline` object. Historically, it needed to be able to operate on inlines that did not have an ID yet, and even further back in history this was probably just an outgrowth of a simple `<form />`.
These classes can be significantly simplified by passing an `$inline` to the `EditView`, instead of individually setting all the properties on the `View` itself. This allows the `DetailView` and `EditView` classes to share a lot of code.
The `EditView` can not fully render its content. Move the content rendering code into the view.
---
Prior to this change, some operations need to work on inlines that don't have an inline ID yet (we assign an ID the first time you "Save" a comment). Since "editing" comments will now be saved, we can instead create a row immediately.
This means that all the inline code can always rely on having a valid ID to work with, even if that ID corresponds to an empty, draft, "isEditing" comment. This simplifies more code in `EditView` and allows the "create" and "reply" code to be merged in `PhabricatorInlineCommentController`.
---
Client-side inline events are currently handled through a mixture of `ChangesetList` listeners (good) and ad-hoc row-level listeners (less good). In particular, the "save", "cancel", and "undo" events are row-level. All other events are list-level.
Move all events to list-level. This is supported by all inlines now having an ID at all stages of their lifecycle.
This allows some of the client behavior to be simplified. It currently depends on binding complex ad-hoc dictionaries into event handlers in `_drawRows()`, but it seems like almost all of this code can be removed. In fact, no more than one row ever seems to be drawn, so this code can probably be simplified further.
---
Finally, save an "isEditing" state. When we rebuild a revision on the client, click the "edit" button if it's in this state. This is a little hacky, but simpler to get into a stable state, since the row layout of an inline depends on a "view row" followed by an "edit row".
Test Plan:
- Created comments on either side of a diff.
- Edited a comment, reloaded, saw edit stick.
- Saved comments, reloaded, saw save stick.
- Edited a comment, typed text, cancelled, "unedited" to get state back.
- Created a comment, typed text, cancelled, "unedited" to get state back.
- Deleted a comment, "undeleted" to get state back.
Weirdness / known issues:
- Drafts don't autosave yet.
- Fixed in D21187:
- When you create an empty comment then reload, you get an empty editor. This is a bit silly.
- "Cancel" does not save state, but should, once drafts autosave.
- Mostly fixed in D21188:
- "Editing" comments aren't handled specially by the overall submission flow.
- "Editing" comments submitted in that state try to edit themselves again on load, which doesn't work.
Subscribers: jmeador
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21186
Summary: Ref T13513. This plans for "currently editing", character range comments, code suggestions, document engine tracking. And absolutely nothing else.
Test Plan:
- Ran `bin/storage upgrade -f`, got a clean upgrade.
- Created and submitted some inline comments; nothing exploded.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21184
Summary:
Ref T13523. In the caching layer, there's a tricky clause about filetypes that skips some body rendering behavior.
Provide file type information which at least has a better chance of representing all changes (e.g., an image file may be replaced with a text file, but this can not be represented by a single file type).
Formalize "hasSourceTextBody()", to mean the changeset parser should engage the change as source text.
Test Plan: Intradiffed text changes, saw the body render properly.
Maniphest Tasks: T13523
Differential Revision: https://secure.phabricator.com/D21210
Summary:
See PHI1722, which requests transaction details about reviewer changes.
This adds them; they're structured to be similar to "projects" and "subscribers" transactions and the "reviewers" attachment on revisions.
Test Plan: {F7410675}
Differential Revision: https://secure.phabricator.com/D21207
Summary: This supports the IntelliJ IDEA editor.
Test Plan:
- Looked at the editor settings panel, saw "idea://".
- Set my editor pattern to "idea://a?b".
- (Did not actually install IntelliJ IDEA.)
Differential Revision: https://secure.phabricator.com/D21206
Summary: Ref T13528. Now that we're hinting users into Files, put the content first and move the detail panel under it. Move the most-useful details (author, size, dimensions) into the curtain.
Test Plan: {F7409925}
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21201
Summary:
Ref T13528. Paste data is stored in files, but the files are always named "raw.txt".
Now that Paste provides a hint to use Files for "DocumentEngine" rendering, try to use the same name as the paste instead.
Test Plan:
- Created a paste named "staggering-insight.ipynb".
- Clicked "View as Jupyter Notebook" from Paste.
- Saw a file named "staggering-insight.ipynb", not "raw.txt".
- Created a paste with no name, saw a file named "raw-paste-data.txt" get created.
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21197
Summary: Ref T13528. When a file in Paste (like a Jupyter notebook) has a good/useful document engine, provide a link to Files.
Test Plan: {F7409881}
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21196
Summary:
See PHI1719. User agents making hard-coded requests to "/favicon.ico" currently 404. This is a mild source of log noise, and we can reasonably route this request.
Limitations:
- This only routes the "PlatformSite". Other sites (custom Phame blogs, third-party sites, Phurl redirectors) won't route here for now.
- This returns a "Location:" redirect to the correct resource rather than icon data directly. This produces the right icon with the right caching behavior, and returning icon data directly is difficult in the general case. However, it won't perform/cache as well as a direct response would.
Test Plan:
- Visted `/favicon.ico`.
- Before: 404.
- After: redirect to favicon.
Differential Revision: https://secure.phabricator.com/D21195
Summary:
Ref T13526. Currently, if a build plan is restricted, viewers may fatal when trying to view related builds.
The old behavior allowed them to see the build even if they can not see the build plan. This is sort of incoherent, but try to stabilize things before fixing this.
Test Plan:
This is a muddy change.
- Created a build with a build plan that Alice can't see.
- As Alice, viewed the build page (restricted before, restricted after); the buildable page (fatal before, works after).
- Also viewed a revision page (works before and after, but user-reported fatal).
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13526
Differential Revision: https://secure.phabricator.com/D21194
Summary:
See PHI1714. This code is incorrectly rendering the chart panel twice, sort of, and passing a non-View object to rendering.
After D21044, this fatals by raising an exception in rendering.
Test Plan: Loaded page, no more exception.
Differential Revision: https://secure.phabricator.com/D21185
Summary:
Ref T13520. Generally, make the table of contents look and more like the paths panel:
- Show a hierarchy, with compression for single-sibling children.
- Use the same icons, instead of "M D" and "(img)" stuff.
- Use EditDistanceMatrix to do a piece-by-piece diff of paths changes.
- Show path changes within the path list.
I'm not entirely sold on this, but it was complicated to write and I've never heard the term "sunk cost fallacy". I think this is mostly a net improvement, but may need some adjustments and followup.
Test Plan: Viewed various changes in Differential and Diffusion, saw a more usable table of contents.
Maniphest Tasks: T13520
Differential Revision: https://secure.phabricator.com/D21183
Summary:
See <https://discourse.phabricator-community.org/t/runtimeexception-during-import-of-commit/3801>. When importing commits with "Auditors:", a raw transaction new value (with an edge edit map using a "+" key) may be passed as an unmentionable PHID list.
Instead, pass an actual PHID list.
Test Plan:
- Pushed a commit with "Auditors: duck".
- Ran daemons.
- Before patch: umentionable PHID exception.
- After patch: clean commit import.
- Verified "duck" was added as an auditor.
Differential Revision: https://secure.phabricator.com/D21181
Summary:
Ref T13523. Currently, when building a "comparison" changeset, metadata is taken from the left changeset. This is somewhat arbitrary.
This means that intradiffs of images don't work properly because the rendered changeset has only the left (usually "old") information.
Later, some of the code attempts to ignore the file data stored on the changeset and reconstruct the correct file data, which is how the result ends up not-completely-wrong.
Be more careful about building sensible-ish metadata, and then just use it directly later on. This fixes the "spooky" code referencing D955 + D6851.
There are some related issues, where "change type" and "file type" are selected arbitrarily and then used to determine whether the change has an "old/new" state or not (i.e., is the left side of the diff empty, since the change creates the file)?
In many cases, neither of the original changesets have a "change type" which will answer this question correctly. Separate this concept from "has state" from "change type", and make more of the code ask narrower questions about the specific conditions or states it cares about, rather than "change type".
Test Plan:
- Created a revision with Diff 1, Diff 2, and Diff 3. Diff 1 takes an image from "null -> A". Diff 2 takes the same image from "null -> B". Diff 3 takes the same image from "A -> B'.
- Intradiffed 1v2 and 1v3.
- Before patch:
- Left side usually missing, which is incorrect (should always be "A").
- Change properties are a mess ("null -> image/png" for MIME type, e.g.)
- Uninteresting/incorrect "unix:filemode" stuff.
- After patch;
- Left side shows state "A".
- Change properties only show size changes (which is correct).
{F7402012}
Maniphest Tasks: T13523
Differential Revision: https://secure.phabricator.com/D21180
Summary:
Ref T13524. If a Harbormaster lint message has no line number (which is permitted), we try to access an invalid index here. This is an exception after D21044.
Treat comments with no line number as unchanged. These comments do not have "ghost" behavior and do not port across diffs.
Test Plan:
- Used "harbormaster.sendmessage" to submit lint with no line number on a changeset.
- Viewed changeset.
- Before patch: "Undefined index: <null>" error.
- After patch: Clean changeset with lint message.
{F7400072}
Maniphest Tasks: T13524
Differential Revision: https://secure.phabricator.com/D21178
Summary:
See PHI1710. Python encodes `True` as `True` (with an uppercase "T") when building URLs.
We currently do not accept this as a "truthy" value, but it's reasonable and unambiguous. Accept "True", "TRUE", "tRuE", etc.
Test Plan: Made a cURL conduit call with "True" and "tRuE". Before patch: failure to decoded booleans; after patch: successful interpretation of "true" variations.
Differential Revision: https://secure.phabricator.com/D21177
Summary: See PHI1710. Until D21044, some transactions could omit "value" and apply correctly. This now throws an exception when accessing `$xaction['value']`. All transactions are expected to have a "value" key, so require it explicitly rather than implicitly.
Test Plan: Submitted a transaction with a "type" but no "value". After D21044, got a language-level exception. After this change, got an explicit exception.
Differential Revision: https://secure.phabricator.com/D21176
Summary: Ref T13522. When changesets update an image, we currently compute no effect hash. A content hash of the image (or other binary file) is a reasonable effect hash, and enalbes effect-hash-based behavior, including hiding files in intradiffs.
Test Plan:
- Created a revision affecting `cat.png` and `quack.txt` (currently, there must be 2+ changesets to trigger the hide logic).
- Updated it with the exact same changes.
- Viewed revision:
- Saw the image renderered in the interdiff.
- Applied patch.
- Ran `bin/differential rebuild-changesets ...`.
- Viewed revision:
- Saw both changesets collapse as unchanged.
Maniphest Tasks: T13522
Differential Revision: https://secure.phabricator.com/D21174
Summary: Ref T13518. See <https://discourse.phabricator-community.org/t/more-exceptions-when-viewing-diffs/3789/>. Under PHP 7.4, accessing an array index of values like `false` and `null` is no longer valid. This is great, but we occasionally do it.
Test Plan:
- Upgraded to PHP 7.4.
- Loaded revisions with added/changed lines, inlines, and Asana support configured.
- Before patch: saw various fatals around accessing indexes of booleans and nulls.
- After patch: clean revision.
Maniphest Tasks: T13518
Differential Revision: https://secure.phabricator.com/D21172
Summary:
Ref T13493. At time of writing, the old API method no longer functions: `1/session` does not return an `accountId` but all calls now require one.
Use the modern `3/myself` API instead. The datastructure returned by `2/user` (older appraoch) and `3/myself` (newer approach) is more or less the same, as far as I can tell.
Test Plan: Linked an account against modern-at-time-of-writing Atlassian-hosted JIRA.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21170
Summary:
Ref T13455. Viewstates are fairly small and will probably grow less quickly than the changeset table, but the data is also not important to retain in the long term: if you revisit a change several months after hiding some files, it's fine if we've forgotten that you adjusted the view parameters.
Add a GC with a long default collection policy (180 days) so installs can manage the size of this table if it becomes necessary.
Test Plan: Ran via `bin/garbage` to adjust the GC policy and collect viewstates.
Maniphest Tasks: T13455
Differential Revision: https://secure.phabricator.com/D21164
