Summary:
Ref T13286. The current (very safe / conservative) rules for retrying git reads generalize to git writes, so we can use the same ruleset in both cases.
Normally, writes converge rapidly to only having good nodes at the head of the list, so this has less impact than the similar change to reads, but it generally improves consistency and allows us to assert that writes which can be served will be served.
Test Plan:
- In a cluster with an up node and a down node, pushed changes.
- Saw a push to the down node fail, retry, and succeed.
- Did some pulls, saw appropriate retries and success.
- Note that once one write goes through, the node which received the write always ends up at the head of the writable list, so nodes need to be explicitly thawed to reproduce the failure/retry behavior.
Maniphest Tasks: T13286
Differential Revision: https://secure.phabricator.com/D20778
Summary: Ref T13286. When retrying a read request, keep retrying as long as we have canididate services. Since we consume a service with each attempt, there's no real reason to abort early, and trying every service allows reads to always succeed even if (for example) 8 nodes of a 16-node cluster are dead because of a severed network link between datacenters.
Test Plan: Ran `git pull` in a clustered repository with an up node and a down node; saw retry count dynamically adjust to available node count.
Maniphest Tasks: T13286
Differential Revision: https://secure.phabricator.com/D20777
Summary:
Depends on D20775. Ref T13286. When a Git read request fails against a cluster and there are other nodes we could safely try, try more nodes.
We DO NOT retry the request if:
- the client read anything;
- the client wrote anything;
- or we've already retried several times.
Although //some// requests where bytes went over the wire in either direction may be safe to retry, they're rare in practice under Git, and we'd need to puzzle out what state we can safely emit.
Since most types of failure result in an outright connection failure and this catches all of them, it's likely to almost always be sufficient in practice.
Test Plan:
- Started a cluster with one up node and one down node, pulled it.
- Half the time, hit the up node and got a clean pull.
- Half the time, hit the down node and got a connection failure followed by a retry and a clean pull.
- Forced `$err = 1` so even successful attempts would retry.
- On hitting the up node, got a "failure" and a decline to retry (bytes already written).
- On hitting the down node, got a failure and a real retry.
- (Note that, in both cases, "git pull" exits "0" after the valid wire transaction takes place, even though the remote exited non-zero. If the server gave Git everything it asked for, it doesn't seem to care if the server then exited with an error code.)
Maniphest Tasks: T13286
Differential Revision: https://secure.phabricator.com/D20776
Summary:
Ref T13286. To support request retries, allow the service lookup method to return an ordered list of structured service references.
Existing callsites continue to immediately discard all but the first reference and pull a URI out of it.
Test Plan: Ran `git pull` in a clustered repository with an "up" node and a "down" node, saw 50% serivce failures and 50% clean pulls.
Maniphest Tasks: T13286
Differential Revision: https://secure.phabricator.com/D20775
Ref T13401. The checkout UI didn't get fully updated to the new View objects,
and account handles are still manually building a URI that goes to the wrong
place.
Summary:
Ref T13393. See some previous discussion in T13366.
Caching is hard and all approaches here have downsides, but the request cache likely has fewer practical downsides for this kind of policy check than other approaches. In particular, the grant approach (at least, as previously used in Phortune) has a major downside that "Query" classes can no longer fully enforce policies.
Since Phortune no longer depends on grants and they've now been removed from instances, drop the mechanism completely.
Test Plan: Grepped for callsites, found none.
Maniphest Tasks: T13393
Differential Revision: https://secure.phabricator.com/D20754
Summary:
Ref T13393. While doing a shard migration in the Phacility cluster, we'd like to stop writes to the migrating repository. It's safe to continue serving reads.
Add a simple maintenance mode for making repositories completely read-only during maintenance.
Test Plan: Put a repository into read-only mode, tried to write via HTTP + SSH. Viewed web UI. Took it back out of maintenance mode.
Maniphest Tasks: T13393
Differential Revision: https://secure.phabricator.com/D20748
Summary:
Fixes T13336.
- Prevent `--no-indexes` from being combined with `--for-replica`, since combining these options can only lead to heartbreak.
- In `--for-replica` mode, dump caches too. See discussion in T13336. It is probably "safe" to not dump these today, but fragile and not correct.
- Mark the "MarkupCache" table as having "Cache" persistence, not "Data" persistence (no need to back it up, since it can be fully regenerated from other datasources).
Test Plan: Ran `bin/storage dump` with various combinations of flags.
Maniphest Tasks: T13336
Differential Revision: https://secure.phabricator.com/D20743
Summary:
Fixes T13389. Currently, we try to "newSubtypeMap()" unconditionally, even if the underlying object does not support subtypes.
- Only try to build a subtype map if subtype transactions are actually being applied.
- When subtype transactions are applied to a non-subtypable object, fail more explicitly.
Test Plan: Clicked "Make Editable" in a fresh Calendar transaction form, got an editable form instead of a fatal from "newSubtypeMap()". (Calendar events are not currently subtypable.)
Maniphest Tasks: T13389
Differential Revision: https://secure.phabricator.com/D20741
Summary:
Fixes T13390. We have some old code which doesn't dynamically select between "utf8mb4" and "utf8". This can lead to dumping utf8mb4 data over a utf8 connection in `bin/storage dump`, which possibly corrupts some emoji/whales.
Instead, prefer "utf8mb4" if it's available.
Test Plan: Ran `bin/storage dump` and `bin/storage shell`, saw sub-commands select utf8mb4 as the client charset.
Maniphest Tasks: T13390
Differential Revision: https://secure.phabricator.com/D20742
Summary:
Depends on D20739. Ref T13366. Slightly modularize/update components of order views, and make orders viewable from either an account context (existing view) or an external context (new view).
The new view is generally simpler so this mostly just reorganizes existing code.
Test Plan: Viewed orders as an account owner and an external user.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20740
Summary: Depends on D20738. Ref T13366. Fixes T8389. Now that the infrastructure is in place, actually send email to external addresses.
Test Plan: Used `bin/phortune invoice` to generate invoices and saw associated external accounts receive mail in `bin/mail list-outbound`.
Maniphest Tasks: T13366, T8389
Differential Revision: https://secure.phabricator.com/D20739
Summary: Depends on D20737. Ref T13367. Allow external addresses to have their access key rotated. Account managers can disable them, and anyone with the link can permanently unsubscribe them.
Test Plan: Enabled/disabled addresses; permanently unsubscribed addresses.
Maniphest Tasks: T13367
Differential Revision: https://secure.phabricator.com/D20738
Summary: Ref T13366. This gives each account email address an "external portal" section so they can access invoices and receipts without an account.
Test Plan: Viewed portal as user with authority and in an incognito window.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20737
Summary:
Depends on D20734. Ref T13366. This makes the cart/order flow work under the new policy scheme with no "grantAuthority()" calls.
It prepares for a "Void Invoice" action, although the action doesn't actually do anything yet.
Test Plan: With and without merchant authority, viewed and paid invoices and went through the other invoice interaction workflows.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20735
Summary: Refresh the 404 text since it hasn't been updated in a while, and swap the "Save Query" button back to grey since I never got used to blue.
Test Plan: Hit 404 page, saved a query.
Differential Revision: https://secure.phabricator.com/D20734
Summary:
Depends on D20732. Ref T13366. This generally makes the "Merchant" UI look and work like the "Payment Account" UI.
This is mostly simpler since the permissions have largely been sorted out already and there's less going on here and less weirdness around view/edit policies.
Test Plan: Browsed all Merchant functions as a merchant member and non-member.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20733
Summary:
Ref T13366. Depends on D20721. Continue applying UI and policy updates to the last two Phortune objects.
Charges aren't mutable and Carts are already transactional, so this is less involved than prior changes.
Test Plan: Viewed various charge/order interfaces as merchants and account members.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20732
Summary:
Depends on D20720. Ref T13366.
- Use modern policies and policy interfaces.
- Use new merchant authority cache.
- Add (some) transactions.
- Move MFA from pre-upgrade-gate to post-one-shot-check.
- Simplify the autopay workflow.
- Use the "reloading arrows" icon for subscriptions more consistently.
Test Plan: As a merchant-authority and account-authority, viewed, edited, and changed autopay for subscriptions.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20721
Summary:
Depends on D20719. Currently, if a page throws an exception (like a policy exception) and rendering that exception into a response (like a policy dialog) throws another exception (for example, while constructing breadcrumbs), we only show the orginal exception.
This is usually the more useful exception, but sometimes we actually care about the other exception.
Instead of guessing which one is more likely to be useful, throw them both as an "AggregateException" and let the high-level handler flatten it for display.
Test Plan: {F6749312}
Differential Revision: https://secure.phabricator.com/D20720
Summary:
Depends on D20718. Ref T13366. Ref T13367.
- Phortune payment methods currently do not use transactions; update them.
- Give them a proper view page with a transaction log.
- Add an "Add Payment Method" button which always works.
- Show which subscriptions a payment method is associated with.
- Get rid of the "Active" status indicator since we now treat "disabled" as "removed", to align with user expectation/intent.
- Swap out of some of the super weird div-form-button UI into the new "big, clickable" UI for choice dialogs among a small number of options on a single dimension.
Test Plan:
- As a mechant-authority and account-authority, created payment methods from carts, subscriptions, and accounts. Edited and viewed payment methods.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13367, T13366
Differential Revision: https://secure.phabricator.com/D20719
Summary:
Depends on D20717. Ref T13366. Make PhortunePaymentMethod use an extended policy interface for consistency with modern approaches. Since Accounts have hard-coded policy behavior (and can't have object policies like "Subscribers") this should have no actual impact on program behavior.
This leaves one weird piece in the policy dialog UIs, see T13381.
Test Plan: Viewed and edited payment methods as a merchant and account member. Merchants can only view, not edit.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20718
Summary: Depends on D20716. Ref T13366. This implements the new policy behavior cleanly in all top-level Phortune payment account interfaces.
Test Plan: As a merchant with an account relationship (not an account member) and an account member, browsed all account interfaces and attempted to perform edits. As a merchant, saw a reduced-strength view.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20717
Summary:
Depends on D20715. Ref T13366. See that task for discussion.
Replace the unreliable "grantAuthority()"-based check with an actual "can the viewer edit any merchant this account has a relationship with?" check.
This makes these objects easier to use from a policy perspective and makes it so that the `Query` alone can fully enforce permissions properly with no setup, so general infrastructure (like handles and transactions) works properly with Phortune objects.
Test Plan: Viewed merchants and accounts as users with no authority, direct authority on the account, and indirect authority via a merchant relationship.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20716
Summary:
Depends on D20713. Ref T13366. When a payment account establishes a relationship with a merchant by creating a cart or subscription, create an edge to give the merchant access to view the payment account.
Also, migrate all existing subscriptions and carts to write these edges.
This aims at straightening out Phortune permissions, which are currently a bit wonky on a couple of dimensions. See T13366 for detailed discussion.
Test Plan:
- Created and edited carts/subscriptions, saw edges write.
- Ran migrations, saw edges write.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20715
Summary: Depends on D20697. Ref T8389. Add support for adding "billing@enterprise.com" and similar to Phortune accounts.
Test Plan: Added and edited email addresses for a payment account.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T8389
Differential Revision: https://secure.phabricator.com/D20713
Summary:
Ref T13366. Some of the information architecture is a little muddy here, notably an item called "Billing / History" which contains payment methods.
Split things up a bit to prepare for adding support for "Email Addresses".
Test Plan: {F6676988}
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20697
Summary: See PHI1396. Ideally this would be some kind of general-purpose tie-in to object relationships, but see D18456 for precedent.
Test Plan: Used `maniphest.edit` to edit associated commits for a task.
Differential Revision: https://secure.phabricator.com/D20731
Summary:
Fixes T13386. See PHI1391. These constraints largely exist already, but are not yet exposed to Conduit.
Also, tweak some keys to support the underlying query.
Test Plan: Ran `differential.revision.search` queries with the new constraints.
Maniphest Tasks: T13386
Differential Revision: https://secure.phabricator.com/D20730
Summary: Ref T13382. Currently, the "Make Administrator" action in the web UI does state-based MFA. Convert it to one-shot MFA.
Test Plan: Empowered and unempowered a user from the web UI, got one-shot MFA'd. Empowered a user from the CLI, no MFA issues.
Maniphest Tasks: T13382
Differential Revision: https://secure.phabricator.com/D20729
Summary:
Ref T13386. If you issue `differential.query` with a large offset (like 3000), it can overheat regardless of policy filtering and fail with a nonsensical error message.
This is because the overheating limit is based only on the query limit, not on the offset.
For example, querying for "limit = 100" will never examine more than 1,100 rows, so a query with "limit = 100, offset = 3000" will always fail (provided there are at least that many revisions).
Not all numbers work like you might expect them to becuase there's also a 1024-row fetch window, but basically small limits plus big offsets always fail.
Test Plan: Artificially reduced the internal window size from 1024 to 5, then ran `differential.query` with `offset=50` and `limit=3`. Before: overheated with weird error message. After: clean result.
Maniphest Tasks: T13386
Differential Revision: https://secure.phabricator.com/D20728
Summary:
Ref T13385. Currently, if you run `arc diff` in a CWD with more than 255 characters, the workflow fatals against the length of the `sourcePath` database column.
In the long term, removing this property is likely desirable.
For now, truncate long values and continue. This only meaningfully impacts relatively obscure interactive SVN workflows negatively, and even there, "some arc commands are glitchy in very long working directories in SVN" is still better than "arc diff fatals".
Test Plan:
- Modified `arc` to submit very long source paths.
- Ran `arc diff`.
- Before: Fatal when inserting >255 characters into `sourcePath`.
- After: Path truncated at 255 bytes.
Maniphest Tasks: T13385
Differential Revision: https://secure.phabricator.com/D20727
Summary:
Fixes T13384. Currently, the subtype "disabled" configuration is not respected when selecting fields for `ROLE_EDIT`.
The only meaningful caller for `ROLE_EDIT` is transaction validation, but transaction validation should respect fields being disabled by subtype configuration.
Test Plan:
- Added a "required" Maniphest custom field "F", then "disabled" it in a subtype "S".
- Created a task of subtype "S".
- Before: Form submission fails with error "F is required", even though the field is not actually visible on the form and can not be set.
- After: Form submits cleanly and creates the task.
Maniphest Tasks: T13384
Differential Revision: https://secure.phabricator.com/D20726
Summary:
Fixes T13382. Depends on D20724. These ancient scripts are no longer necessary since we've had a smooth web-based onboarding process for a long time.
I retained `bin/user empower` and `bin/user enable` for recovering from situations where you accidentally delete or disable all administrators. This is normally difficult, but some users are industrious.
Test Plan: Grepped for `accountadmin` and `add_user.php`, found no more hits.
Maniphest Tasks: T13382
Differential Revision: https://secure.phabricator.com/D20725
Summary:
Ref T13382.
- Remove "bin/people profileimage" which previously generated profile image caches but now feels obsolete.
- Replace it with "bin/user", with "enable" and "empower" flows. This command is now focused on regaining access to an install after you lock your keys inside.
- Document the various ways to unlock objects and accounts from the CLI.
Test Plan:
- Ran `bin/user enable` and `bin/user empower` with various flags.
- Grepped for `people profileimage` and found no references.
- Grepped for `bin/people` and found no references.
- Read documentation.
Maniphest Tasks: T13382
Differential Revision: https://secure.phabricator.com/D20724
Summary: Fixes T13383. Provide a basic "drydock.resource.search". Also allow "drydock.lease.search" to be queried by resource PHID.
Test Plan: Called "drydock.resource.search" and "drydock.lease.search" with various constraints.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13383
Differential Revision: https://secure.phabricator.com/D20723
Summary: See PHI1392. This flag is `--all`, not `--all-caches`.
Test Plan: Ran `bin/cache purge --all`.
Differential Revision: https://secure.phabricator.com/D20722
Summary:
Fixes T13378. If we join Ferret tables and page, we can end up with an ambiguous `id` column here.
Explicitly refer to "project.x" in all cases that we're interacting with the project table.
Test Plan:
- Changed page size to 3.
- Issued a Projects query for "~e", matching more than 3 results.
- Clicked "Next Page".
- Before: ambiguous id column fatal.
- After: next page.
Maniphest Tasks: T13378
Differential Revision: https://secure.phabricator.com/D20714
Summary:
Ref T13369. See that task for discussion.
When the discovery daemon finds more than 64 commits to import, demote the worker queue priority of the resulting tasks.
Test Plan:
- Pushed one commit, ran `bin/repository discover --verbose --trace ...`, saw commit import with "at normal priority" message and priority 2500 ("PRIORITY_COMMIT").
- Pushed 3 commits, set threshold to 3, ran `bin/repository discover ...`, saw commist import with "at lower priority" message and priority 4000 ("PRIORITY_IMPORT").
Maniphest Tasks: T13369
Differential Revision: https://secure.phabricator.com/D20712
Summary:
Ref T13373. When you "bin/config set x ..." a value, the success message ("Set x ...") is somewhat ambiguous and can be interpreted as "First, you need to set x..." rather than "Success, wrote x...".
Make the messaging more explicit. Also make this string more translatable.
Test Plan: Ran `bin/config set ...` with various combinations of flags, saw more clear messaging.
Maniphest Tasks: T13373
Differential Revision: https://secure.phabricator.com/D20711
Summary: Fixes T13374. The "Temporary Failures" row is missing a cell definiton from the addition of "Average Queue Time".
Test Plan: Viewed "/daemon/" with some temporary failures and and odd number of rows above the "Temporary Failures" row. Saw cell properly zebra-striped.
Maniphest Tasks: T13374
Differential Revision: https://secure.phabricator.com/D20710
Summary:
Ref T13349. This is almost the same change as D20678, but for project profiles instead of user profiles.
The general reproduction case is "view a project where you can't see more than 50 of the 500 most recent feed stories".
Test Plan:
- Forced all queries to overheat.
- Viewed a project profile page.
- Before: overheating fatal near top level.
- After: damage contained to feed panel.
Maniphest Tasks: T13349
Differential Revision: https://secure.phabricator.com/D20704
Summary: See downstream <https://phabricator.wikimedia.org/T229757>. The "autofocus" attribute mostly just works, so add it to this input.
Test Plan: As a user with TOTP enabled, established a new session. Saw browser automatically focus the "App Code" input on the TOTP prompt screen.
Differential Revision: https://secure.phabricator.com/D20703
Summary:
Fixes T13370. We currently show an "Award Badge" button conditionally, based on whether the viewer can award any badges or not.
The query to test this may overheat and this pattern isn't consistent with other UI anyway. Stop doing this test.
Test Plan:
- Created 12 badges.
- As a user who could not edit any of the badges, viewed the "Badges" section of a user profile.
Maniphest Tasks: T13370
Differential Revision: https://secure.phabricator.com/D20702
Summary: Fixes T13368. Some workflows (like "Move tasks to...") execute board layout without objects to update. In these cases, we can hit a warning because `objectPHIDs` is not initialized to `array()`.
Test Plan: Went through the "Move tasks to..." workflow on a workboard, no longer saw a warning when trying to iterate over an empty `objectPHIDs` list.
Maniphest Tasks: T13368
Differential Revision: https://secure.phabricator.com/D20701
Summary:
Ref T13368. Currently, both visible and hidden columns are shown in the "Move tasks to..." dropdown on workflows from workboards.
When the dropdown contains hidden columns, move them to a separate section to make it clear that they're not likely targets.
Test Plan:
- Used "Move tasks to project..." targeting a board with no hidden columns. Saw a single ungrouped dropdown.
- Used "Move tasks to project..." targeting a board with hidden columns. Saw a dropdown grouped into "Visible" and "Hidden" columns.
Maniphest Tasks: T13368
Differential Revision: https://secure.phabricator.com/D20700
Summary:
Ref T13368. Proxy columns should not be selectable from this workflow. If you want to move tasks to milestone/subproject X, do "Move tasks to project..." and pick X as the project.
(This could be made to work some day.)
Test Plan: Went through a "Move tasks to project..." workflow targeting a project with subprojects. No longer saw subproject columns presented as dropdown options.
Maniphest Tasks: T13368
Differential Revision: https://secure.phabricator.com/D20699
Summary: Ref T13368. The column options presented to the user are currently incorrect because the wrong set of columns are drawn from.
Test Plan: On a workboard, used "Move tasks to project..." to target another board, saw that board's columns.
Maniphest Tasks: T13368
Differential Revision: https://secure.phabricator.com/D20698
Summary:
Fixes T13341. Currently, cart emails (invoices/receipts) are sent to members of the associated merchant account. This was just a simple way to keep an eye on things when this was first written.
The system works fine, and recent changes (almost certainly D20525) stopped these emails from working (presumably because of the slightly weird merchant permissions model).
This could be sorted out in more detail, but it looks like the path forward is to introduce a side channel for email anyway (via T8389), and that's a better way to implement this behavior since it means the normal recipients won't see a bunch of random staff/merchant email addresses on their receipts.
Test Plan: Grepped for `merchant` in this editor.
Maniphest Tasks: T13341
Differential Revision: https://secure.phabricator.com/D20696
