Summary:
Ref T13627. Currently, global locks always return connections (even external connections) to the connection pool when unlocked.
This code is obviously buggy: `isExternalConnection` is set to false immediately before it is tested. This bug has existed since this code was introduced, in D15792.
- Instead of storing a flag, store the actual connection.
- Don't clear it when unlocking.
- Don't return external connections to the pool.
Test Plan:
- Added a failing test, made it pass.
Maniphest Tasks: T13627
Differential Revision: https://secure.phabricator.com/D21583
Summary:
Ref T13625. See that task for discussion.
Currently, the Viewer when performing revision updates in response to commits may be an arbitrary low-privilege user (an Application, a disabled User, a bot, a mailing list, etc).
Today, this leads to an exception when trying to make API calls.
Ideally, we probably would not perform the update in these cases. However, performing the update isn't a policy violation and is generally less surprising than not performing it, so continue performing it for now: just use the omnipotent user to interact with the API.
Test Plan:
- Authored a commit as a bot user without permission to view the repository or revision.
- Commented out a couple of caches, and used `bin/repository reparse --publish ...` to republish the commit.
- Before: exception when trying to interact with the API.
- After: clean publish.
Maniphest Tasks: T13625
Differential Revision: https://secure.phabricator.com/D21582
Summary:
Ref T13623. When paginating notifications, we may currently construct a query which:
- loads from non-unique rows; and
- returns multiple results.
In particular, `chronologicalKey` isn't unique across the whole table (only for a given viewer). We can get away with this because no user-facing view of notifications is truly "every notification for every viewer" today.
One fix would be to implicitly force the paging query to include `withUserPHIDs(viewerPHID)`, but puruse a slightly more general fix:
- Load only unique stories.
- Explictly limit the pagination subquery to one result.
Test Plan:
- Set page size to 1, inserted duplicate notifications of all stories for another user, clicked "Next", got the GROUP BY error.
- Applied the "only load unique stories" part of the change, got a "expected one row" error instead.
- Applied the "limit 1" part of the change, got a second page of notifications.
Maniphest Tasks: T13623
Differential Revision: https://secure.phabricator.com/D21577
Summary: Ref T13624. Depends on D21578. In "sshd" subprocess contexts, use "PhutilErrorLog" to direct errors to both stderr and, if configured, a logfile on disk.
Test Plan:
- Confiugured an error log.
- Forced `ssh-auth` to fatal.
- Saw errors on stderr and in log.
Maniphest Tasks: T13624
Differential Revision: https://secure.phabricator.com/D21579
Summary: Ref T13611. This property worked correctly when implemented in D19357. The behavior was broken by D20775, which tested node-level routing but did not specifically re-test the "writable" property. This was difficult to spot because ref query outcomes weren't observable in the UI, and the ref itself had the correct property value.
Test Plan:
See D21575. After this change, the UI shows the correct state, rather than showing a read-only service ref as writable:
{F8465865}
Maniphest Tasks: T13611
Differential Revision: https://secure.phabricator.com/D21576
Summary: Ref T13611. Currently, the "writable" property on service bindings has no effect because of a trivial bug. Provide more information in the UI to make this kind of problem observable.
Test Plan:
Viewed "Storage" section of management UI, saw a more-obvious problem with ref management (a non-writable ref is listed as writable).
{F8465851}
Maniphest Tasks: T13611
Differential Revision: https://secure.phabricator.com/D21575
Summary:
Ref T13620.
- Make generic edge stories render links with hovercards. Other story types (like subscriptions) already do this so I'm fairly certain this is just old code from before hovercards.
- Include a longer commit message snippet in hovercards.
Test Plan: {F8465645}
Maniphest Tasks: T13620
Differential Revision: https://secure.phabricator.com/D21574
Summary:
Ref T13617. When an inline comment is added inside a block of added lines, it currently ends up off-by-one when porting forward.
This is a disagreement between the mapping engine and the display engine about what "offset" means. Choose the simpler of the two interpretations.
Test Plan:
- Created a revision with the diff in T13617.
- Added an inline in the middle of the added block.
- Updated the revision with the same diff.
- Before: inline incorrectly moves up by one line.
- After: inline maps correctly.
Maniphest Tasks: T13617
Differential Revision: https://secure.phabricator.com/D21572
Summary: Ref T13586. In the footsteps of D21563, make Herald rule results more formal and structured to support meaningful exception reporting.
Test Plan:
Ran various Herald rules and viewed transcripts, including rules with recursive dependencies and condition exceptions.
{F8447894}
Maniphest Tasks: T13586
Differential Revision: https://secure.phabricator.com/D21565
Summary: Ref T13586. Lift the behavioral core of "HeraldConditionResult" into a new abstract base "HeraldTranscriptResult", with the intent to introduce a "HeraldRuleResult".
Test Plan:
- Ran Herald rules, reviewed transcripts.
- This change should have no behavioral effect.
Maniphest Tasks: T13586
Differential Revision: https://secure.phabricator.com/D21564
Summary:
Ref T13586. Currently, Herald condition logs encode "pass" or "fail" robustly, "forbidden" through a sort of awkward side channel, and can not properly encode "invalid" or "exception" outcomes.
Structure the condition log so results are represented unambiguously and all possible outcomes (pass, fail, forbidden, invalid, exception) are clearly encoded.
Test Plan:
{F8446102}
{F8446103}
Maniphest Tasks: T13586
Differential Revision: https://secure.phabricator.com/D21563
Summary: Ref T13586. The Herald transcript page has become more and more complicated over time, and recently added "Transactions" and "Profiler" sections. Split these across separate navigation tabs to limit the maximum complexity of any single view and make it easier to navigate to particular sections, like the profiler section.
Test Plan: Viewed various transcripts, saw nice digestible sections.
Maniphest Tasks: T13586
Differential Revision: https://secure.phabricator.com/D21493
Summary: Ref T13615. This property was removed from the Facebook API at some point, perhaps November 2020. Stop relying no it.
Test Plan: Created a local Facebook OAuth app, registered a new account locally.
Maniphest Tasks: T13615
Differential Revision: https://secure.phabricator.com/D21571
Summary: Ref T13609. Add the Object PHID (object being built), Container PHID (container of the object being built), Build PHID, and Buildable PHID to Harbormaster build variables.
Test Plan:
{F8448191}
{F8448192}
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13609
Differential Revision: https://secure.phabricator.com/D21569
Summary:
Ref T13608. When searching for bare URIs in remarkup text, don't look for URIs with a protocol string longer than 32 characters.
This avoids a case where the regexp engine may be tricked into executing at `O(N^2)` or some similar complexity.
Test Plan:
- Applied remarkup to "AAAA..." (512KB).
- Before: 64 seconds to process.
- After: <10ms to process.
- Ran unit tests.
Maniphest Tasks: T13608
Differential Revision: https://secure.phabricator.com/D21562
Summary:
Ref T13587. D21495 has significant changes to the ngram indexer, which might possibly contain bugs.
Make it easier to reindex a subset of documents (based on the date when the index was built, and/or the software version which generated the index).
This is in addition to the existing versioning, which is focused on object versions.
Test Plan: Ran `bin/search index` with various old and new arguments. Spot-checked the `IndexVersion` table.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13587
Differential Revision: https://secure.phabricator.com/D21560
Summary:
Ref T13587. Currently, when a document is reindexed by Ferret, the old document is completely discarded and a new version is inserted to replace it.
This approach is simple to implement, but can lead to exhaustion of the ngram AUTO_INCREMENT id column in reasonable circumstances.
Conceptually, this approach "should" be fine and this exhaustion is an awkard implementation detail. However, since it's easy to be less wasteful when performing document updates and all the other approaches are awkward or leaky in other ways that are probably worse, use a more complex implementation to avoid executing unnecessary INSERT statements.
Test Plan:
- Created and indexed a new document, searched for it.
- Updated a document, indexed it with `bin/search index ... --force --trace`, saw only modifications updated in the index.
- Searched for newly added terms (got hits) and removed terms (no longer got hits) to verify add/delete index behavior.
Maniphest Tasks: T13587
Differential Revision: https://secure.phabricator.com/D21495
Summary: Ref T13607. Add some time-oriented constraints to this API method to support compiling build statistics.
Test Plan:
- Called "harbormaster.target.search" with all new constraints.
- Viewed documentation in API console.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13607
Differential Revision: https://secure.phabricator.com/D21559
Summary: Ref T13605. Support selecting a diff's changesets (to get a list of affected file paths) via the API.
Test Plan: Called API with no arguments, diffPHIDs, PHIDs, IDs. Got sensible output.
Maniphest Tasks: T13605
Differential Revision: https://secure.phabricator.com/D21558
Summary:
Ref T13605. Changesets currently have no PHID, which limits their ability to use standard API infrastructure.
Give them a PHID, since there's no reason they don't have one other than their age.
Test Plan:
- Ran migrations, saw PHIDs populated.
- Created new changesets, saw PHIDs.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13605
Differential Revision: https://secure.phabricator.com/D21557
Summary: Ref T13602. Warn when a reivison has at least one human reviewer, no non-human reviewers, and no human reviewers can view it.
Test Plan: {F8430683}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21556
Summary: Ref T13602. Similar to subscriber and mention treatments, make it clear when a user doesn't have view permission.
Test Plan: {F8430595}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21555
Summary:
Ref T13602. When rendering a user hovercard, pass the object on which the reference appears. If the user can't see the object, make it clear on the hovecard.
Restyle the "nopermission" markup in mentions to make it more obvious what the style means: instead of grey text, use red with an explicit icon.
Test Plan: {F8430398}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21554
Summary:
Ref T13602. Currently, Hovercards are functions only of the object they represent (and the viewer, etc).
Recent changes to how users who can't see an object are rendered motivate making them a function of both the object they represent //and// the context in which they are being viewed. In particular, this enables a hovecard for a user to explain "This user can't see the thing you're lookign at right now.", so visual "exiled" markers can have a path forward toward discovery.
Test Plan:
- This change isn't expected to affect any behavior.
- Viewed hovercards, moused over/out, resized windows, viewed standalone cards, viewed debug cards, saw no behavioral changes.
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21553
Summary:
Ref T13602. Currently, timeline comment rendering does not (by default) propagate the context object to the rendering layer.
This means that `@mentions` of users who can't see the object aren't rendered properly (currently: they show up as blue, but should show up as grey).
Pass the context down the stack and into the remarkup engine.
Test Plan: {F8382905}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21548
Summary:
Ref T13602. When a subscriber can't see an object, it's currently hard to figure it out.
Show this status clearly in the curtain UI.
Test Plan: {F8382865}
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21547
Summary:
Ref T13602. Currently, the policy framework can not execute "test if many users can see one object" particluarly efficiently. This test must be executed more broadly to implement the changes in T13602.
To avoid making this any worse than it already is, lift this block into a wrapper class that has a bulk queue + fetch API and could eventually be optimized.
Test Plan: Viewed a task with an `@mention` of a user without permission to see it in the summary, saw it rendered in a disabled style.
Maniphest Tasks: T13602
Differential Revision: https://secure.phabricator.com/D21546
Summary:
Ref T13395. Libphutil has merged into Arcanist and no longer needs to be installed or upgraded. Additionally:
- The minimum PHP version is now PHP 5.5.
- Although older versions of PHP should still install APC, modern versions come with Opcache and do not need APC. Setup issues guide administrators thorugh the correct install procedure now.
Test Plan: Read documentation.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D21550
Summary:
Ref T13600. When migrating observed repositories between cluster services, impact can be better controlled by fetching a copy of the repository on the target host before clusterizing it.
In particular, in the Phacility cluster, migrations are generally from one shared shard to one dedicated shard. It's helpful to perform these migrations synchronously without waiting for the cluster to sync in the background (helpful in the sense that there are fewer steps and fewer commands to run).
This supports an "--observe" mode to the internal "bin/services load-repository" workflow, which transfers repository data by refetching it from the remote rather than by getting it from the older host. This fetch occurs before cluster configuration is adjusted.
Test Plan: Ran locally as a sanity check, will apply in production.
Maniphest Tasks: T13600
Differential Revision: https://secure.phabricator.com/D21544
Summary:
See <https://discourse.phabricator-community.org/t/i-cant-create-almanac-space/4424/>.
Almanac namespaces have never really had property support, but they implemented the interface in the original implementation.
At the time, this had no effect. Later changes integrated properties into the edit flows and broke this no-op integration.
Remove the interface for now. They could be given property support later, but need a bit of support code.
This feature is very rarely used and primarily useful for Phacility instances.
Test Plan: Created new namespaces and edited namespaces, browsed namespace UI.
Differential Revision: https://secure.phabricator.com/D21543
Summary:
Ref T13588. This has never been meaningful, but a "final private" method is specifically forbidden in PHP8.
Remove meaningless "final" from these methods, per new lint checks.
Test Plan: Ran `arc lint --everything` to identify affected methods, then `... | xargs -n1 arc lint --apply-patches`.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21540
Summary: Ref T13591. Provide some guidance on the most common cases for wanting to interact with the worker queue.
Test Plan: Read documentation.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21536
Summary: Ref T13591. Support delaying selected tasks until a later time and bulk-adjustment of task priority.
Test Plan: Ran `bin/worker delay` and `bin/worker priority` to delay and reprioritize tasks. Confirmed outcomes with daemon console.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21535
Summary:
Ref T13591. Add more selector flags to let "bin/worker" commands operate on tasks by container PHID, object PHID, priority, etc.
This anticipates adding "bin/worker reprioritize" and "bin/worker delay" workflows, to provide more tools for handling repository imports.
Test Plan:
- Ran `bin/worker execute`, `cancel`, `retry`, and `free` with various sets of selector flags.
- Used `--min-priority`, `--max-priority`, `--object`, `--container`, `--archived`, `--max-failure-count` to select tasks.
- Specified invalid, duplicate, aliased objects with "--object".
- Specified invalid range priority selectors.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21534
Summary: Ref T13591. This is a minor consistency change to use PHIDs instead of IDs in the commit import processing pipeline. PHIDs are generally more powerful in more contexts and it would be unusual for a modern worker to use an ID here.
Test Plan:
- Made the "accept either ID or PHID" part of the change only.
- Pushed a commit, parsed and reparsed it step by step (this tests that "commitID" tasks can still process normally).
- Made the "write PHIDs" part of the change.
- Pushed a commit, parsed and reparsed it step by step.
- Looked at the task row in the database, saw PHID data.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21533
Summary:
Ref T13591. Improve how parameters are passed between commit worker tasks:
- Always pass "via", to track where tasks came from.
- Always provide "objectPHID" (with the commit PHID).
- Always provide "containerPHID" (with the repository PHID).
Test Plan:
- Pushed a new commit.
- Ran `bin/repository pull` + `bin/repository discover`, saw commit with all parameters.
- Ran `bin/worker execute ...`, saw a Change worker and then a Publish worker with appropriate parameters.
- Ran `bin/repository reparse ... --background`, saw workers queue with appropriate parameters.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21532
Summary:
Ref T13591. Worker queue tasks which affect commits currently (mostly) store the commit as an "objectPHID", but do not directly reference the repository the commit belongs to.
This can make certain operations (like "change the priority of all tasks affecting repository Y") more difficult than it needs to be.
Support a "containerPHID", similar to the field of the same name on builds, that can store a parent object like a repository and better support operations against subsets of tasks.
See also D11044 for the genesis of "objectPHID".
This depends on the introduction of storage patch phases (in D21529) so that earlier migrations which queue worker tasks don't try to insert this column before it actually exists.
Test Plan:
- Ran `bin/storage upgrade`.
- No callers yet, see further changes for usage.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21531
Summary:
Ref T13591. Some storage patches queue worker tasks, currently always to rebuild search indexes.
These patches can not execute in creation order if a later patch modifies the worker task table, since they'll try to perform a modern INSERT against an out-of-date table schema. Such a modification is desirable in the context of T13591, but making it causes these patches to fail.
Patches have an existing "after" mechanism which allows them to have explicit dependencies. This mechanism could be used to resolve this issue, but all patches with a dependency like this would need to be updated every time the queue table changes.
Instead, introduce "phases" to provide broader ordering rules. There are now two phases: "default" and "worker". Patches in the "worker" phase execute after patches in the "default" phase.
Phases may eventually be further separated, but
Test Plan:
- Ran `bin/storage status`, saw patches annotated with phases.
- Will apply `containerPHID` changes on top of this.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21529
Summary:
Ref T13596. See that task for discussion. Executing "INSERT ... SELECT" at default isolation levels requires more locking than executing "SELECT" + "INSERT" separately.
Decompose this "INSERT ... SELECT" into "SELECT + INSERT", and reformat it to execute a minimal set of changes instead of wiping everything out and then writing all of it back. In most cases, this means we write 1 row instead of `O(number of project members)` rows.
Test Plan:
- Created a project. Added and removed members, looked at database and saw a consistent membership/materialization list.
- Created a subproject. Added and removed members, looked at database and saw a consistent membership/materialization list.
I wasn't successful in reproducing the LOCK WAIT issue locally by trying various concurrent SELECT / INSERT / INSERT ... SELECT strategies. It may depend on the "DELETE + INSERT ... SELECT" structure used here, or versions/config/etc, so we'll have to see how that fares in production.
Maniphest Tasks: T13596
Differential Revision: https://secure.phabricator.com/D21527
Summary:
See PHI1983. Ref T13599. Ref T13589. Currently, if you browse to a path browse URI in Diffusion without a trailing slash (`/browse/master/src`), you get a nonsensical view (the directory as a single item).
Be more precise in how "git ls-tree" arguments are constructed.
Test Plan: Visited files and directories in the browse view, with and without trailing slashes. Saw improved behavior for directories with no trailing slash and reasonable behavior in all other cases.
Maniphest Tasks: T13599, T13589
Differential Revision: https://secure.phabricator.com/D21528
Summary:
Ref T13590. By default, PHP kills execution after web scripts run for 30 seconds. If this occurs in the locked section of a repository write while we're holding the durable write lock, the lock will get stuck.
Use "set_time_limit(0)" to prevent this mechanism from interrupting execution while the durable lock is held.
Test Plan:
- Added "set_time_limit(1)" before the lock and "while (1);" in the critical section of the lock.
- Pushed, got the lock stuck.
- Cleared the lock, applied this patch, pushed.
- Got an infinite hang instead. (Normally, we expect the script to take more than 30 seconds to execute because there is a large push that executes in finite time, not because there's an infinte loop.)
Maniphest Tasks: T13590
Differential Revision: https://secure.phabricator.com/D21526
Summary:
Ref T13590. Currently, errors arising from cluster locking (like the "stuck write lock" exception) are not caught and converted into VCS responses on the HTTP VCS workflow.
Catch a broader range of exceptions and convert them into appropriate responses.
Test Plan:
- Forced a "stuck write lock" exception, pushed to a Git repository over HTTP.
- Before: generic fatal.
- After: VCS-specific fatal with a useful message in the "X-Phabricator-Message" response header.
Maniphest Tasks: T13590
Differential Revision: https://secure.phabricator.com/D21525
Summary:
Ref T13590. User objects have some inline caches that don't do readthrough generation by default because it may be indicative of high-impact performance problems in code.
During a VCS request, these caches are normally unnecessary, but they may be hit on some unusual pathways (like error handling).
Flag VCS users as okay for inline generation. This does not indicate a performance problem and access to these caches is very rare, at least today.
Test Plan:
- Executed a Git HTTP request which hit an unhandled exception (stuck write lock).
- Before: got a second-level exception while handling the first exception, when trying to access user preferences to render a standard uncaught exception page.
- After: no second-level exception.
Maniphest Tasks: T13590
Differential Revision: https://secure.phabricator.com/D21524
Summary:
Ref T13590. Currently, when you encounter a HTTP error in Git, there is no apparent way to make the client show any additional useful information. In particular, the response body is ignored.
We can partially get around this by putting the information in an "X-Phabricator-Message: ..." HTTP header, which is visible with "GIT_CURL_VERBOSE=1 git ...". Users won't normally know to look here, but it's still better than nothing.
Test Plan:
- Ran "GIT_CURL_VERBOSE=1 git fetch" against a Phabricator HTTP URI that returned a HTTP/500 error.
- Before: no clue what happened on the client.
- After: client shows useful message in the "X-Phabricator-Message" header in debug output.
Maniphest Tasks: T13590
Differential Revision: https://secure.phabricator.com/D21523
Summary:
Ref T13593. The commit cache in this Engine has a maximum fixed size (currently 65,535 entries).
If we execute discovery in a repository with more refs than this (e.g., 180K), we get fast lookups for the first 65,535 refs and slow lookups for the remaining refs.
Instead, divide the refs into chunks no larger than the cache size, and perform an explicit cache fill before each chunk is processed.
Test Plan:
- Created a repository with 1K refs. Set cache size to 256. Ran discovery.
- Before patch: saw one large cache fill and then ~750 single-gets.
- After patch: saw four large cache fills.
- Compared `bin/repository discover ... --verbose` output before and after patch for overall effect; saw no differences.
Maniphest Tasks: T13593
Differential Revision: https://secure.phabricator.com/D21521
Summary:
Ref T13595. See that task for discussion.
D21511 renamed the iteration variable here (previously "$path") but did not rename this use of it.
Test Plan:
- In Diffusion, browsed a directory with a submodule.
- Before: "setExternalURI()" fatal in conduit call.
- After: directory listing including submodule.
Maniphest Tasks: T13595
Differential Revision: https://secure.phabricator.com/D21520
Summary:
Ref T13589. The output for "git ls-tree commit:path" (the old invocation) and "git ls-tree commit -- path" (the new invocation) differs: the latter emits absolute paths.
Update the code to account for this difference in behavior.
Test Plan:
- Browsed a non-root directory in a Git repository in Diffusion.
- Before: saw absolute paths.
- After: saw relative paths.
Maniphest Tasks: T13589
Differential Revision: https://secure.phabricator.com/D21519
Summary: Ref T13591. Fixes a few issues with the recent updates here discovered in more thorough testing.
Test Plan:
- Stopped the daemons.
- Created a new copy of Phabricator in Diffusion.
- Pulled it with `bin/repository pull ...`.
- Got 17,278 commits on disk with `git log --all --format=%H`.
- Set permanent refs to "master".
- Discovered it with `bin/repository discover ...`.
- This took 31.5s and inserted 17,278 tasks.
- Verified that all tasks have priority 4,000 (PRIORITY_IMPORT).
- Observed that 16,799 commits have IMPORTED_PERMANENT and 479 commits do not.
- This matches `git log master --format=%H` exactly.
- Ran `bin/repository refs ...`. Expected no changes and saw no changes.
- Ran `bin/worker execute --active` for a minute or two. It processed all the impermanent changes first (since `bin/worker` is LIFO and these are supposed to process last).
- Ran `bin/repository refs`. Expected no changes and saw no changes.
- Marked all refs as permanent.
- Starting state: 16,009 message tasks, all at priority 4000.
- Ran `bin/repository refs`, expecting 479 new tasks at priority 4000.
- Saw count rise to 16,488 as expected.
- Saw all the new tasks have priority 4000 and all commits now have the IMPORTED_PERMANENT flag.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21518
Summary: Ref T13591. This is mostly a workaround for Big Sur not having pcntl/posix installed by default and the mess with M1 / Homebrew / SIP / Code Signing (see T13232) so I can't easily run actual daemons and need to fake them with `bin/worker execute --active`, but it's a reasonable flag on its own.
Test Plan:
- Ran `bin/worker execute --active` and `bin/worker cancel --active`.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21517
Summary:
Ref T13591. There are currently two pathways to queue an import task for a commit: via repository discovery, or via a ref becoming permanent.
These pathways duplicate some logic and have behavioral differences: one does not set `objectPHID` properly, one does not set the priority correctly.
Unify these pathways, make them both set `objectPHID`, and make them both use the same priority logic.
Test Plan:
- Discovered refs.
- See later changes in this series for more complete test cases.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21516
Summary:
Ref T13591. Currently, the "IMPORTED_PERMANENT" flag (previously "IMPORTED_CLOSEABLE", until D21514) flag is set by using the result of "shouldPublishRef()".
This method returns the wrong value for the flag when there is a repository-level reason not to publish the ref (most commonly, because the repository is currently importing).
Although it's correct that commits should not be published in an importing repository, that's already handled in the "PublishWorker" by testing "shouldPublishCommit()". The "IMPORTED_PERMANENT" flag should only reflect whether a commit is reachable from a permanent ref or not.
- Move the relevant logic to a new method in Publisher.
- Fill "IMPORTED_PERMANENT" narrowly from "isPermanentRef()", rather than broadly from "shouldPublishRef()".
- Deduplicate some logic in "PhabricatorRepositoryRefEngine" which has the same intent as the logic in the Publisher.
Test Plan:
- Ran discovery on a new repository, saw permanent commits marked as permanent from the beginning.
- See later changes in this patch series for additional testing.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21515
Summary:
Ref T13591. This is an old flag with an old name, and there's an import bug because the outdated concept of "closable" is confusing two different behaviors.
This flag should mean only "is this commit reachable from a permanent ref?". Rename it to "IMPORTED_PERMANENT" to make that more clear.
Rename the "Unpublished" query to "Permanent" to make that more clear, as well.
Test Plan:
- Grepped for all affected symbols.
- Queried for all commmits, permament commits, and impermanent commits.
- Ran repository discovery.
- See also further changes in this change series for more extensive tests.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21514
Summary:
Ref T13591. Since D8781, this flag does not function correctly in Git and Mercurial repositories, since ref discovery pre-fills the cache.
Move the "don't look at the database" behavior the flag enables into the cache lookup. D8781 should have been slightly more aggressive and done this, it was just overlooked.
Test Plan:
- Ran `bin/repository discover --help` and read the updated help text.
- Ran `bin/repository discover --repair` in a fully-discovered Git repository.
- Before: no effect.
- After: full rediscovery.
Maniphest Tasks: T13591
Differential Revision: https://secure.phabricator.com/D21513
Summary:
Ref T13589. In D21510, not every ref selector got touched, and this isn't a valid construction in Git:
```
$ git ls-tree ... -- ''
```
Thus:
- Disambiguate more (all?) ref selectors.
- Correct the construction of "git ls-tree" when there is no path.
- Clean some stuff up: make the construction of some flags and arguments more explicit, get rid of a needless "%C", prefer "%Ls" over acrobatics, etc.
Test Plan: Browsed/updated a local Git repository. (This change is somewhat difficult to test exhaustively, as evidenced by the "ls-tree" issue in D21510.)
Maniphest Tasks: T13589
Differential Revision: https://secure.phabricator.com/D21511
Summary: Ref T13589. See that task for discussion.
Test Plan: Executed most commands via "bin/conduit" or in isolation.
Maniphest Tasks: T13589
Differential Revision: https://secure.phabricator.com/D21510
Summary: These characters are missing support in `{key ...}` but are reasonable to include.
Test Plan: {F8302969}
Differential Revision: https://secure.phabricator.com/D21508
Summary: Ref T13575. Particularly with the new Apple silicon, I think there are enough domain collisions for `M1`, `M2`, `P1`, etc., to justify adding them to the default ignore list.
Test Plan: Created a mock, then wrote a comment referencing an object on the list (`M1`) and an object not on the list (`T1`). Got text and a link respectively.
Maniphest Tasks: T13575
Differential Revision: https://secure.phabricator.com/D21507
Summary:
Ref T13575. Since PHP builtin webserver support was added, the pathway for parsing request parameters became more complex. We now rebuild "$_REQUEST" later, and this rebuild will destroy any mutations made to it here, so the assignment to "__path__" is lost.
Instead of "validating" the request path, make this method "read" the request path and store it explicitly, so it will survive any later request mutations.
Test Plan:
- Submitted any POST form while running Phabricator under the builtin PHP webserver. Old behavior was an error when accessing "__path__"; new behavior is a working application.
- Loaded normal pages, etc.
Maniphest Tasks: T13575
Differential Revision: https://secure.phabricator.com/D21506
Summary: Without this change, the landing page for the Packages app is https://secure.phabricator.com/packages, which is a 404. There's probably a better way to fix this, but this was the fewest characters.
Test Plan: doitlive
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21494
Summary: Ref T13585. Provide a minimal but technically functional "harbormaster.step.edit" API method.
Test Plan: Used the web console to modify the URI for a "Make HTTP Request" build step.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13585
Differential Revision: https://secure.phabricator.com/D21489
Summary: Ref T13585. This isn't particularly useful (notably, it does not include custom field values and isn't searchable by build plan PHID) but get the basics into place.
Test Plan: Used the web UI to make API calls, reviewed results.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13585
Differential Revision: https://secure.phabricator.com/D21488
Summary:
Ref T2312. Numeric strings are read out of arrays as integers, and modern PHP raises appropriate warnings when they're then treated as strings.
For now, cast the keys to strings explicitly (we know we inserted only strings). In the future, introduction of a `StringMap` type or similar might be appropriate.
Test Plan:
- Added "abc.12345.xyz" to the blocklist, changed my VCS password.
- Before: fatal when trying to "strpos()" an integer.
- After: password change worked correctly.
Maniphest Tasks: T2312
Differential Revision: https://secure.phabricator.com/D21487
Summary:
Changes the heuristic method by which non-zero exit statuses from git-http-backend are found to be due to packfile negotiation during shallow fetches, etc.
Instead of checking git-http-backend stderr for a generic "hung up" error message, see if the pack-result response contains a terminating flush packet ("0000"). This should give a greater assurance that the request was handled correctly and the response is complete.
Test Plan: Run `GIT_CURL_VERBOSE=1 git fetch --depth 1 https://host.example/source/repo.git HEAD` to ensure it completes and includes two successful POST requests during packfile negotiation (the last one actually receives the packfile).
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, dzduvall
Tags: #diffusion
Differential Revision: https://secure.phabricator.com/D21484
Summary:
See <https://discourse.phabricator-community.org/t/mail-details-view-broken/4315>. The change in D21400 detects a missing "GROUP BY" in some variations of this query.
Specifically, we may join multiple recipient rows (since mail may have multiple recipients) and then fail to group the results.
Fix this by adding the "GROUP BY". Additionally, remove the special-cased behavior when no authors or recipients are specified -- it's complicated and not entirely correct (e.g., may produce a "no object" instead of a policy error when querying by ID), and likely predates overheating.
Test Plan:
- Disabled `metamta.one-mail-per-recipient` in Config.
- Generated a message to 2+ recipients.
- Viewed the message detail; queried for the message by specifying 2+ recipients.
- Viewed the unfiltered list of messages, saw the query overheat.
Differential Revision: https://secure.phabricator.com/D21486
Summary:
See PHI1876. Normally, deleted inlines are undeleted with an "undelete" operation, which clears the "isDeleted" flag.
However, when an inline is deleted implicitly by using "Cancel" without first saving it, the flag currently isn't cleared properly. This can lead to cases where inlines seem to vanish (they are shown to the user in the UI, but treated as deleted on submission).
Test Plan:
There are two affected sequences here:
- Create a new inline, type text, cancel, undo.
- Create a new inline, type text, cancel, undo, save.
The former sequence triggers an "edit" operation. The subsequent "Save" in the second sequence triggers a "save" operation.
It's normally impossible in the UI to execute a "save" without executing an "edit" first, but "save" clearly should undelete the comment if you get there somehow, so this change clears the deleted flag in both cases for completeness.
- Executed both sequences, saw comment persist in preview, on reload, and after submission.
Differential Revision: https://secure.phabricator.com/D21483
Summary: See PHI1912. Ref T13491. "arc" now requires "--" when stdin is not a TTY; provide this argument for users.
Test Plan: Viewed example in console, saw "--". Executed example.
Maniphest Tasks: T13491
Differential Revision: https://secure.phabricator.com/D21482
Summary:
See PHI1900. Recent changes to how commit graphs are drawn made the height automatic in most cases, but it fails in Differential because the element isn't initially visible so the computed height is 0.
Just give them an explicit height so they show up again.
Test Plan: Viewed graphs in Maniphest, Differential, and Diffusion; saw them all render properly.
Differential Revision: https://secure.phabricator.com/D21481
Summary:
See PHI1901. An install would like improved support for identifying files related to an object (like a task or revision) for retention/archival/backup/migration/snapshotting purposes.
The "attachment" edge is not really user-level: it just means "if you can see the object, that allows you to see the file". This set includes files that users may not think of as "attached", like thumbnails and internal objects which are attached for technical reasons.
However, this is generally an appropriate relationship to expose for retention purposes.
Test Plan: Used "edge.search" to find files attached to a revision and objects attached to a file.
Differential Revision: https://secure.phabricator.com/D21480
Summary: Ref T13583. To improve support for making it harder to improperly mix data retention policies, allow Herald to act on comment content.
Test Plan:
- Wrote comment content Herald rules in Maniphest and Differential.
- Submitted non-matching comments (no action) and matching comments (Herald action).
- In Differential, triggered rules by submitting non-matching main content and a matching inline comment.
Maniphest Tasks: T13583
Differential Revision: https://secure.phabricator.com/D21479
Summary:
See PHI1896. If you do this:
- Create an inline comment over a wide range of lines.
- Suggest an edit.
- Make a change near the beginning of the block.
- Make a change near the end of the block.
- Save the inline.
...you get a rendering which includes a "Show More Context" fold in the middle.
Currently, this element renders in a visually broken way and consumes too many columns.
However, this element isn't ever desirable inside inline comment suggestions. Stop it from rendering entirely.
Test Plan:
- Made an inline comment suggestion across lines 1-50 with edits at the beginning and end, saw a contiguous diff.
- Made smaller inline comment suggestions (one line, a few lines).
Differential Revision: https://secure.phabricator.com/D21476
Summary:
Ref T13552. The behavior of "RepositoryQuery" with ambiguous identifiers under "withRepositoryPHIDs()" is tricky. This leads to failure to load commits in Subversion in some cases.
Use "withRepository()", which gives us the correct identifier resolution behavior.
Test Plan: Viewed a subversion repository history in Diffusion, saw commit details after change.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21469
Summary:
Ref T13552. The Herald field "Accepted Differential revision" (and similar fields) depend on the task/revision update steps running before Herald executes.
Herald currently executes first, so it never sees associated revisions. Swap this order.
Test Plan: Published a commit, got a clean parse/import. Will test with production rules ("Cowboy Commits").
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21468
Summary: See PHI1885. Repository operations are queryable by state and author, but neither column has a usable key. Add usable keys.
Test Plan: Ran EXPLAIN on a state query. Ran `bin/storage upgrade`. Ran EXPLAIN again, saw query go from a table scan to a `const` key lookup.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D21465
Summary:
Ref T13581. Currently, unexpected exceptions inside Conduit calls are passed to the client, but not logged on the server.
These exceptions should generally be unexpected, and producing a server-side trace is potentially useful.
Test Plan: Simulated a during-execution exception, saw it get logged on the server.
Maniphest Tasks: T13581
Differential Revision: https://secure.phabricator.com/D21464
Summary:
Ref T13581. If you query for revisions by hash and provide multiple hashes (A, B) which match a single revision (e.g., older and newer diffs for that revision), the query omits a GROUP BY clause but should contain one.
Add a GROUP BY clause in this case.
Test Plan:
With a working copy that has multiple hashes corresponding to a single revision, ran `arc branches` before and after the change. Before, got this error:
```
[2020-09-15 17:02:07] EXCEPTION: (ConduitClientException) ERR-CONDUIT-CORE: Rows passed to "loadAllFromArray(...)" include two or more rows with the same ID ("130"). Rows must have unique IDs. An underlying query may be missing a GROUP BY. at [<arcanist>/src/conduit/ConduitFuture.php:65]
```
After, clean execution.
Maniphest Tasks: T13581
Differential Revision: https://secure.phabricator.com/D21462
Summary:
Ref T13552. When a previously discovered commit becomes reachable from a permanent ref, we re-queue workers to update it. However, the commit may already be marked as "published", so the publish worker may do nothing.
It would perhaps be simpler to not mark the commit as published when it isn't reachable from a permanent ref, but this is tricky because the flag is also part of the "imported / all steps" state (see T13580).
Until that can be cleaned up, just clear the flag.
Test Plan:
- Pushed a commit with "fixes X" to a non-permanent branch.
- Pushed it to a permanent branch.
- Before change: task failed to close.
- After change: task closes properly.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21460
Summary:
Ref T13552. Fixes T13569. Currently, if a process uses in-process tasks (usually, a debugging/diagnostic workflow) and those tasks (or tasks those tasks queue) fail permanently, the exception escapes to top level and the process exits.
This isn't desirable; catch the exception and fail them locally instead.
Test Plan:
With a failing Asana integration and misconfigured Webhook, ran `bin/repository reparse --publish ...`.
- Before: fatals on each substep.
- After: warnings emitted for failed substep, but process completes.
Maniphest Tasks: T13569, T13552
Differential Revision: https://secure.phabricator.com/D21459
Summary:
Ref T13552. Now that these steps can build their own "CommitRef" object from storage on the "CommitData" object, move them from the "Message" step to the "Publishing" step.
This should resolve the root issue in T13552, where a commit moved from a non-permanent branch to a permanent branch does not publish closures properly.
Test Plan: Used "bin/repository reparse --publish ..." to republish changes.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21450
Summary:
Ref T13552. Turn "CommitData" into an application-level layer on top of the repository-level "CommitRef" object.
For older commits which will not have a "CommitRef" record on disk, build a synthetic one at runtime. This could eventually be migrated.
Test Plan: Ran "bin/repository reparse --message", browsed Diffusion.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21449
Summary: Ref T13552. Currently, various callers read raw properties off "CommitData" directly. Wrap these in accessors to support storage changes which persist "CommitRef" information instead.
Test Plan:
- Ran "diffusion.querycommits", saw the same data before and after.
- Looked at a commit, saw authorship information and date.
- Viewed tags in a repository, saw author information.
- Ran "rebuild-identities", saw no net effect.
- Grepped for callers to "getCommitDetail(...)".
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21448
Summary: Ref T13552. The internal caller for this now uses "internal.commit.search", which is always authority-reading. No legitimate external caller should rely on the behavior of "bypassCache"; no-op it to simplify behavior.
Test Plan: Called "diffusion.querycommits", saw the same data as before.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21447
Summary: Ref T13552. Swap the call we're using to build "CommitRef" objects here to the recently-introduced "internal.commit.search" method.
Test Plan: Used "bin/repository reparse --message ..." to reparse commits, added "var_dump()" to inspect results. Saw sensible CommitRef and CommitData objects get built.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21446
Summary:
Ref T13552. This is one of two callsites to "diffusion.querycommits". It's an old debugging workflow which I haven't used in years and which is likely obsoleted by identities and other changes.
I believe the root problem here was also ultimately user error (a user has misconfigured their local Git author email as another user).
Test Plan: Grepped for "lookup-users", got no hits.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21444
Summary:
Ref T13552. Commit parsers currently invoke a special mode of "diffusion.querycommits", which is an older frozen method.
The replacement, "diffusion.commit.search", is not really appropriate for low-level access. This mode of having a single method which operates in "cache" or "non-cache" modes also ends up in a lot of unnecessary field shuffling.
Provide "internal.commit.search" as a modern equivalent that returns a "DiffusionCommitRef"-compatible structure.
Test Plan: Executed "internal.commit.search", got sensible low-level commit results.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21443
Summary:
Ref T13552. Some Diffusion conduit calls may only be served by a node which hosts a working copy on disk, so they're proxied if received by a different node.
This capability is currently bound tightly to "DiffusionRequest", which is a bundle of context parameters used by some Diffusion calls. However, call proxying is not fundamentally a Diffusion behavior.
I want to perform proxying on a "*.search" call which does not use the "DiffusionRequest" parameter bundle. Lift proxying to the root level of Conduit.
Test Plan: Browsed diffusion in a clusterized repsository.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21442
Summary: Ref T13552. Neither "$hashes" or "$user" are used, and constructing them has no side effects.
Test Plan: Searched for these symbols.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21441
Summary:
Ref T13570. Fixes T13235. In most cases, we use modern (v4) signatures for almost all AWS API calls, and have for several years.
However, sending email via SES currently uses an older piece of external code which uses the older (v3) signature method.
AWS is retiring v3 signatures on October 1 2020, so this pathway will stop working.
Update the pathway to use `PhutilAWSFuture`, which provides v4 signatures.
T13235 discusses poor error messages from SES. Switching to Futures fixes this for free, as they have more useful error handling.
Test Plan:
- Configured an SES mailer, including the new `region` parameter.
- Used `bin/mail send-test` to send mail via SES.
- Sent invalid mail (from an unverified address); got a more useful error message.
- Grepped for removed external, no hits.
Maniphest Tasks: T13570, T13235
Differential Revision: https://secure.phabricator.com/D21461
Summary: Ref T13577. After the fix in D21453, lint identifies additional static errors in Phabricator; fix them.
Test Plan: Ran `arc lint`; these messages are essentially all very obscure.
Subscribers: hach-que, yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13577
Differential Revision: https://secure.phabricator.com/D21457
Summary: Ref T13579. This property was removed in D21425, but I missed this usage site. Remove the assignment; this class no longer tracks the subprocess PID directly.
Test Plan: Searched for "->pid", no further hits.
Maniphest Tasks: T13579
Differential Revision: https://secure.phabricator.com/D21452
Summary:
Ref T13573. Using the browser "Print" feature on pages produces "Thu, Aug 4, 12:22" timestamps which require context to interpret precisely (they don't have a year and don't have a timezone).
Instead, retain these timestamps in "screen" contexts but use "YYYY-MM-DD HH:MM:SS (UTC+X)" timestamps when printing.
Test Plan: Printed Maniphest tasks and other pages in Safari and Chrome using "?__print__=1" and "Print to PDF", saw absolute timestamps after this chagne in the printed documents.
Maniphest Tasks: T13573
Differential Revision: https://secure.phabricator.com/D21451
Summary:
See PHI1809, which identified a bug in Project search where queries with a large number of slugs could paginate improperly.
This change detects problems in this category: cases where multiple rows with the same ID are passed to "loadAllFromArray()". It's likely that all cases it detects are cases where a GROUP BY is missing.
Since this might have some false positives or detect some things which aren't fundamentally problematic, I'm planning to hold it until the next release.
Test Plan:
- Reverted D21399, then created a project with multiple slugs and queried for one of them via "project.search". Hit this new exeception.
- Browsed around a bit, didn't immediately catch any collateral damage.
Differential Revision: https://secure.phabricator.com/D21400
Summary: Ref T13552. There are currently some content overflow issues on the graph view where the menu height can exceed the content height and the frame is drawn on a sub-element. Make the frame draw around all the content.
Test Plan: Viewed commit graph history view, saw more sensible UI.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21440
Summary: Ref T13552. Provide a richer handle/status list item for commit lists.
Test Plan: Viewed commits in various interfaces, saw richer information.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21431
Summary:
Ref T13552. Build the "commit list" elements so that the menu action items collapse under the element on mobile.
Also change the mobile breakpoint to 512px because my Safari window can't go any narrower than 508px. Future changes to responsive design will be more content-aware anyway.
Test Plan: Looked at commits in various interfaces, at desktop and mobile widths.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21430
Summary:
Ref T13552. The current layout doesn't work particularly well on desktops or devices.
We have some device/desktop table layout code, but it isn't generic. We also have property list layout code, but it isn't generic either.
Provide generic layout elements ("Fuel", from "Phabricator UI Layout" to "PHUIL"?) and narrowly specialize their display behavior. Then swap the ListItemView stuff to use it.
Test Plan:
Saw slightly better responsive behavior:
{F7637457}
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21418
Summary:
Ref T13552. In unifying the various Graph/List/Table commit views, some information was dropped -- particularly, audit status.
Restore most of it. The result isn't very pretty, but has most of the required information.
Test Plan: {F7637411}
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21417
Summary: Ref T13552. Some of the CSS can be removed or simplified now that essentially all lists of commits are on a single rendering pathway.
Test Plan: Grepped for affected CSS, viewed commit graph.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21416
Summary: Ref T13552. Remove yet another way to render a list of commits, and unify it with "CommitGraphView".
Test Plan:
- Viewed commit search results.
- Viewed owners package detail page.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21415
Summary:
Ref T13552.
Currently, the "Browse" page shows a snippet of unmerged changes if you're looking at a non-default branch. Remove this for consistency with the simplified main "Browse" page. This is reachable via "Compare".
Update the "Compare" page to use the new "CommitGraphView".
Test Plan:
- Looked at the "Browse" page of "stable".
- Looked at the "Compare" page for "stable vs master".
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21414
Summary: Ref T13552. When viewing a merge commit, merged changes are currently shown inline. Update this view to use the new "GraphView" rendering pipeline.
Test Plan:
- Viewed a merge commit, saw merges.
- Viewed history, profile page, etc.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21413
Summary:
Ref T13552. This older view mostly duplicates other code and has only two callsites:
- The "Commits" section of user profile pages.
- The "Ambiguous Hash" page when you visit a commit hash page which is an ambiguous prefix of two or more commit hashes.
Replace both with "DiffusionCommitGraphView".
Test Plan:
- Visited profile page, clicked "Commits".
- Visited an ambiguous hash page (`rPbd3c23`).
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21412
Summary: Ref T13552. In the new combined "table/list" graph view, tidy up the graph rendering.
Test Plan: {F7633504}
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21411
Summary:
Ref T13552. Currently, commit lists are sometimes rendered as an object list and sometimes rendered as a table. There are two separate views for table rendering.
Add a fourth view ("list, with a graph") with the eventual intent of unifying all the other views. For now, this only replaces "HistoryListView" -- and needs some more work to really be a convincing replacement.
Test Plan:
- Looked at "History" in Diffusion, saw an ugly view with all the information we want.
- Grepped for "HistoryListView", no hits.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21410
Summary:
Ref T13552. Currently, Diffusion has two effectively identical history views, the "Graph" view and the "History" view.
These arose out of product uncertainty about the importance of the graph, but I think we can just put the graph on the "object item list" view and merge these views.
Test Plan: Looked at repositories in Diffusion, no longer saw a "Graph" tab. Grepped for "graph"-related symbols.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21409
Summary:
Ref T13552. Currently, the repository landing page has a panel with recent commits. This is accessible by clicking "History" and usually below the fold, so it's not clearly useful.
Since I'm consolidating this code anyway to fix an issue with the import pipeline, just get rid of this history view.
Test Plan: Viewed a repository landing page, no longer saw a history panel.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21408
Summary: Ref T13552. This older class has no callers; tag and branch listings were replaced with an "ObjectList" view.
Test Plan: Grepped for "DiffusionTagTableView", got no hits.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21407
Summary:
Ref T13552. I'm trying to reduce the number of direct callers to commit authorship metadata. This header seems low-value enough to simply remove; this information is shown more clearly and prominently in the "Provenance" UI.
In particular, commits have multiple dates (authored, committed, pushed) but this header shows only one. It currently shows the author identity and the commit date, which isn't entirely correct. And it potentially uses an "Identity" as a timeline actor, which is conceptually fine but not entirely firm ground.
Test Plan: Viewed a commit, saw no more subheader.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21406
Summary:
Ref T13552. Give "Commit" objects a more modern, identity-aware way to render author and committer information.
This uses handles in a more modern way and gives us a single read callsite for raw author and committer names.
Test Plan:
- Grepped for callers to the old methods, found none. (There are a lot of "renderAuthor()" callers in transactions, but this call takes no parameters.)
- Viewed some commits, saw sensible lists of authors and committers.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21405
Summary:
Ref T13552. When viewing a directory in Diffusion, we make an Ajax call to get the last commit for each path.
This call currently pulls author information, since an older version of this UI showed author information.
The current UI does not show author information, so this parameter is unused. Delete the code which builds it.
Test Plan: Grepped for `'author'` and references to the "pull-lastmodified" behavior. This behavior is invoked in only one place, which never generates an author placeholder.
Maniphest Tasks: T13552
Differential Revision: https://secure.phabricator.com/D21404
Summary:
See PHI1834. It's not obvious why this "+1" is present in the code, but it causes inlines to be adjusted incorrectly when a file is not modified across changes. See D21435.
Remove it, which appears to produce accurate adjustment behavior.
Test Plan:
- See D21435 for instructions to build a change, where a file with lines "A-Z" is unmodified across Diff 1 and Diff 2.
- Left inlines on lines 14, 17-19, and 16-26 (end of the file) on Diff 1.
- Before: saw inlines incorrectly adjusted to lines 15, 18, and 17 on Diff 2. Before D21435, the last inline was culled by the rendering engine.
- After: saw inlines correctly adjusted to lines 14, 17, and 16 (the same lines as the original), render properly, and highlight the correct lines when hovered.
Differential Revision: https://secure.phabricator.com/D21436
Summary:
See PHI1834. Currently, the inline adjustment engine can sometime "adjust" an inline off the end of a diff. If it does, we lay it out on an invalid display line here and never render it.
Instead, make sure that layout never puts a comment on an invalid line, so the UI is robust against questionable decisions by the adjustment engine: no adjustment should be able to accidentally discard an inline.
Test Plan:
- Created a two diff revision, where Diffs 1 and 2 have "alphabet.txt" with A-Z on one line each. The file is unchanged across diffs; some other file is changed.
- Added a comment to lines P-Z of Diff 1.
- Before: comment is adjusted out of range on Diff 2 and not shown in the UI.
- After: comment is still adjusted out of range internally, but now corrected into the display range and shown.
Differential Revision: https://secure.phabricator.com/D21435
Summary:
Ref PHI1835. Generally, Jupyter notebooks in the wild may store source and markdown content as either a single string or a list of strings.
Make the renderer read these formats more consistently. In particular, this fixes rendering of code blocks stored as a single string.
This also fixes an issue where cell labels were double-rendered in diff views.
Test Plan:
Created a notebook with a code block represented on disk as a single string, rendered a diff from it.
{F7696071}
Differential Revision: https://secure.phabricator.com/D21434
Summary:
See PHI1839. Currently, the "No newline at end of file" text is dropped in the 1-up diff view for changes that affect a file with no trailing newline.
Track it through the construction of diff primitivies more carefully.
Test Plan: {F7695760}
Differential Revision: https://secure.phabricator.com/D21433
Summary:
Although I'm not entirely thrilled about doing flow control like this (as an actual action in a build plan), I believe this build step works correctly and there's no fancy replacement mechanism on the immediate horizon, and this didn't send us down a slippery slope of Turing-complete builds encoded without real structure or context. Just kick it out of prototype.
(Other approaches which might be better in the long run are things like "this is a top-level behavior on the build plan itself" and/or "build plans are written in a DSL, not a Javascript UI".)
Test Plan: Added a new build step, saw this as an option in the "Flow Control" section.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D21432
Summary:
Reverts D21419. See PHI1814. Previously, I used "user-select: all" to group sequences of spaces for selection.
However, this has a side effect: the sequence is now selected with a single click. I didn't read the docuementation on the CSS property thoroughly and missed this in testing, since I was focused on drag-selection behavior.
This behavior is enough of a net negative that I think we're in a worse state overall; revert it.
Test Plan: Straight revert.
Differential Revision: https://secure.phabricator.com/D21429
Summary:
Ref T13556. These options are very old and effectively obsoleted by "bin/phd debug [--trace]". I haven't used either option diagnostically in many years, and they aren't mentioned in the documentation.
Remove them to simplify configuration, and because "phd.trace" doesn't work anyway and likely hasn't for a long time -- it has specific issues with TTY detection (see T13556).
Test Plan: Grepped for "phd.trace" and "phd.verbose". Ran "bin/phd debug [--trace]" and saw verbose/trace output.
Maniphest Tasks: T13556
Differential Revision: https://secure.phabricator.com/D21426
Summary:
Ref T13555. Currently, the daemon future may resolve into a failure state immediately inside "start()", and not have a valid PID when we read it.
Instead, read PIDs from the current active future in all cases, using "hasPID()" to test for the presence of a valid PID.
Since we don't query the PID immediately, we no longer need to explicitly start the future.
Also fix an issue where the same future could be added to the overseer pool more than once if it threw on "resolve()". In general:
- Before we "resolve()" a future, detach it from the DaemonHandle: we're always done with it.
- Catch exceptions on resolution and treat them the same way as subprocess resolution errors. These aren't common, but are possible in the general case.
- Have DaemonHandle add futures to the future pool directly when they're created.
Test Plan:
- Ran daemons with intentional subprocess creation failures, saw clean recovery.
- Ran daemons with intentional resolution exceptions, saw clean recovery.
Maniphest Tasks: T13555
Differential Revision: https://secure.phabricator.com/D21425
Summary:
Ref T13555. Although these callsites may not actually impact anything, it's possible for an active handle to have no PID (e.g., if the subprocess failed to start).
Handle these cases more carefully.
Test Plan: Started daemons, saw them run fine. See also next change.
Maniphest Tasks: T13555
Differential Revision: https://secure.phabricator.com/D21424
Summary:
Fixes T13554. For certain prose diff inputs and PCRE backtracking limits, this regular expression may back track too often and fail.
A characteristic input is "x x x x ...", i.e. many sequences where `(.*?)\s*\z` looks like it may be able to match but actually can not.
I think writing an expression which has all the behavior we'd like without this backtracking issue isn't trivial (at least, I don't think I know how to do it offhand); just use a strategy based on "trim()" insetad, which avoids any PCRE complexities here.
Test Plan: Locally, this passes the "x x x ..." test which the previous code failed. I'm not including that test because it won't reproduce across values of "pcre.backtrac_limit", PCRE versions, etc.
Maniphest Tasks: T13554
Differential Revision: https://secure.phabricator.com/D21422
Summary: See PHI1819. This structure may have `null` elements.
Test Plan: Will confirm user reproduction case.
Differential Revision: https://secure.phabricator.com/D21420
Summary:
Ref T2495. See PHI1814. Currently, Phabricator replaces tabs with spaces when rendering diffs.
This may or may not be the best behavior in the long term, but it gives us more control over expansion of tabs than using tab literals.
However, one downside is that you can use your mouse cursor to select "half a tab", and can't use your mouse cursor to distinguish between tabs and spaces. Although you probably shouldn't be doing this, this behavior is less accurate/correct than selecting the entire block as a single unit.
A specific correctness issue with this behavior is that the entire block is copied to the clipboard as a tab literal if you select any of it, so two different visual selection ranges can produce the same clipboard content.
This particular behavior can be improved with "user-select: all", to instruct browsers to select the entire element as a single logical element. Now, selecting part of the tab selects the whole thing, as though it were really a tab literal.
(Some future change might abandon this approach and opt to use real tab literals with "tab-size" CSS, but we lose some ability to control alignment behavior if we do that and it doesn't have any obvious advantages over this approach other than cursor selection behavior.)
Test Plan:
- In Safari and Firefox, dragged text to select a whitespace-expanded tab literal. Saw browsers select the whole sequence as though it were a single tab.
- In Chorme, this also mostly works, but there's some glitchiness and flickering. I think this is still a net improvement, it's just not as smooth as Safari and Firefox.
Maniphest Tasks: T2495
Differential Revision: https://secure.phabricator.com/D21419
Summary:
See PHI1810. In situations where:
- An author submits an urgent change for review.
- The author pings reviewers to ask them to look at it.
...the reviewers may not be able to move the review forward if the review is currently a "Draft". They can only "Commandeer" or ask the author to "Request Review" as ways forward.
Although I'm hesitant to support review actions (particularly, "Accept") on draft revisions, I think there's no harm in allowing reviewers to skip tests and promote the revision out of draft as an explicit action.
Additionally, lightly specialize some of the transaction strings to distinguish between "request review from draft" and other state transitions.
Test Plan:
- As an author, used "Request Review" to promote a draft and to return a change to reviewers for consideration. These behaviors are unchanged, except "promote a draft" has different timeline text.
- As a non-author, used "Begin Review" to promote a draft.
Differential Revision: https://secure.phabricator.com/D21403
Summary:
Currently, adding subscribers to a draft revision raises a warning that they won't get an email/notification.
This warning has some false positives:
- it triggers on any subscriber change, including removing subscribers; and
- it triggers if you're only adding yourself as a subscriber.
Narrow the scope of the warning so it is raised only if you're adding a subscriber other than yourself.
Test Plan:
- Added a non-self subscriber, got the warning as before.
- Added self as a subscriber, no warning (previously: warning).
- Removed a subscriber, no warning (previously: warning).
Differential Revision: https://secure.phabricator.com/D21402
Summary:
See PHI1810. Build toward support for "Request Review" by non-authors on drafts, to forcefully pull a revision out of draft.
Currently, some action strings can't vary based on revision state or the current viewer, so this "pull out of draft" action would have to either: say "Request Review"; or be a totally separate action.
Neither seem great, so allow the labels and messages to vary based on the viewer and revision state.
Test Plan: Grepped for affected symbols, see followup changes.
Differential Revision: https://secure.phabricator.com/D21401
Summary:
Modern Mercurial may emit some more patterns under "--debug".
This whole list is gross and can likely now be eliminated by increasing the minimum required Mercurial version (as `arc` has), but just paper over it for now.
Test Plan:
Locally, saw some views return to functional behavior that weren't previously working on a modern version of Mercurial.
The reproduction case is likely something in the vein of "repository is not writable by webserver, look at history view".
Differential Revision: https://secure.phabricator.com/D21398
Summary:
See PHI1809. This query may join the "slug" table, but each project may have multiple slugs, and the query does not "GROUP BY" when this join occurs.
This may lead to partial result sets and unusual paging behavior.
This could likely be caught categorically in `loadAllFromArray()`; I'll adjust this in a followup.
Test Plan:
A minimal reproduction case is something like:
- Give project P slugs: a, b, c.
- Give project Q slugs: d.
- Query for slugs: a, b, c, d; with limit 2.
- Order the query so P returns first.
- Expect: P and Q.
- Actual: P generates 3 raw rows and the final result is just P with no pagination cursor.
Differential Revision: https://secure.phabricator.com/D21399
Summary:
A handful of Phacility production shards have run into memory pressure issues recently. Although there's no smoking gun, and at least two other plausible contributors, one possible concern is that the Fact daemon was written before hibernation and can not currently hibernate. Even if there's no memory leak, this creates unnecessary memory pressure by holding the processes in memory.
Allow the Fact daemon to hibernate, like other daemons do.
Test Plan: Ran "bin/phd debug fact", saw the Fact daemon hibernate.
Subscribers: yelirekim
Differential Revision: https://secure.phabricator.com/D21389
Summary: Ref T13546. Companion change to D21372. Move URI normalization code to Arcanist to we can more-often resolve remote URIs correctly.
Test Plan: Grepped for affected symbols.
Maniphest Tasks: T13546
Differential Revision: https://secure.phabricator.com/D21373
Summary:
See PHI1794, which describes a connection exhaustion issue with a large number of webhook tasks in queue.
The "GlobalLock" mechanism manages a separate connection pool from the main pool, and webhook workers immediately try to grab a webhook lock with a 0-second wait when they start. So far, this is fine.
Prior to this change, good connections which fail to acqiure a lock are discarded. This can lead to connection exhaustion as the worker rapidly cycles through lock attempts: the connections will remain open for at least 60 seconds (since D16389) in an effort to avoid outbound port exhaustion, but they're effectively orphaned because they aren't part of the main pool and aren't part of the lock pool. We're basically leaking a connection every time we fail to lock.
Failing to lock doesn't mean we need to discard the connection: it's a completely suitable connection for reuse. Instead of dropping it on the floor, put it into the lock pool.
Test Plan:
- Used "bin/webhook call ... --count 10000 --background" to queue a large number of webhook calls against a slow ("sleep(15);") webhook.
- Used "bin/phd launch 32 taskmaster" to start taskmasters.
- Observed MySQL connection behavior:
- Before change: 2048 configured connections immediately exhausted.
- After change: connections stable at ~160ish.
- Ran queue for a while, saw expected single-threaded calls to webhook.
Differential Revision: https://secure.phabricator.com/D21369
Summary:
See PHI1794, which reports an issue where a large number of queued webhook calls led to connection exhaustion. To make this easier to reproduce and test, add "--count" and "--background" flags to "bin/webhook call".
This primarily supports "bin/webook call ... --background --count 10000" to quickly fill the queue with a bunch of calls.
Test Plan: Ran `bin/webhook call` in foreground and background modes, with and without counts. Saw appropriate console and queue behavior.
Differential Revision: https://secure.phabricator.com/D21368
Summary:
The "Export Data" workflow incorrectly uses the "Policy Favorites" setting to choose a default export format. This is just a copy/paste error; the correct setting exists and is unused.
If the setting value is an array (as the "Policy Favorites" value often is), we try to use it as an array index. This generates a runtime exception after D21044.
```
[2020-06-16 06:32:12] EXCEPTION: (RuntimeException) Illegal offset type in isset or empty at [<arcanist>/src/error/PhutilErrorHandler.php:263]
#0 <#2> PhutilErrorHandler::handleError(integer, string, string, integer, array) called at [<phabricator>/src/applications/search/controller/PhabricatorApplicationSearchController.php:460]
```
- Use the correct setting.
- Make sure the value we read is a string.
Test Plan:
- Used "Export Data" with a nonempty, array-valued "Policy Favorites" setting.
- Before: runtime exception.
- After: clean export.
- Used "Export Data" again, saw my selection from the first time persisted.
Differential Revision: https://secure.phabricator.com/D21361
Summary: Ref T13546. This makes some "arc" tasks a little easier, and will make them more correct if "arc" ever switches to using SSH.
Test Plan: Ran "harbormaster.buildable.search" from the web UI, saw URIs in the result set.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13546
Differential Revision: https://secure.phabricator.com/D21346
Summary:
See <https://discourse.phabricator-community.org/t/bad-regex-in-prose-diff-logic/3969>.
The prose splitting rules normally guarantee that newlines appear only at the beginning or end of blocks. However, if a prose sentence ends with text like "...x\n.", we can end up with a newline inside a "sentence".
If we do, the regular expression that breaks it into pieces will fail.
Arguably, this is an error in how sentences are split apart (we might prefer to split this into two sentences, "x\n" and ".", rather than a single "x\n." sentence) but in the general case it's not unreasonable for blocks to contain newlines, so a simple fix is to make the pattern more robust.
Test Plan: Added a failing test which includes this behavior, made it pass.
Differential Revision: https://secure.phabricator.com/D21295
Summary:
Currently, Phortune attempts to prevent users from removing themselves as account managers. It does this by checking that the new list includes them.
Usually this is sufficient, because you can't normally edit an account unless you're already a manager. However, we get the wrong result (incorrect rejection of the edit) if the actor is omnipotent and the acting user was not already a member.
It's okay to edit an account into a state which doesn't include you if you have permission to edit the account and aren't already a manager.
Specifically, this supports more formal tooling around staff modifications to billing accounts, where the actor has staff-omnipotence and the acting user is a staff member and only used for purposes of leaving a useful audit trail.
Test Plan: Elsewhere, ran staff tooling to modify accounts and was able to act as "alice" to add "bailey", even though "alice" was not herself a manager.
Differential Revision: https://secure.phabricator.com/D21288
Summary:
Ref T13513. An inline is not considered empty if it has a suggestion, but some of the shared transaction code doesn't test for this properly.
Update the shared transaction code to be aware that application comments may have more complex emptiness rules.
Test Plan:
- Posted an inline with only an edit suggestion, comment went through.
- Tried to post a normal empty comment, got an appropriate warning.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21287
Summary:
Ref T13541. The passthru future does not have time limit behavior, so if we reach this code we currently fail.
Phabricator never reaches this code normally, but this code is reachable during debugging if you try to foreground a slow fetch to inspect it.
Passthru commands generally only make sense to run interactively, and the caller or control script can enforce their own timeouts (usually by pressing "^C" with their fingers).
Test Plan: Used a debugging script to run ref-by-ref fetches in the foreground.
Maniphest Tasks: T13541
Differential Revision: https://secure.phabricator.com/D21284
Summary:
See PHI1752.
- Early exit of document layout can cause us to fail to populate available rows.
- Some Jupyter documents have "markdown" cells with plain strings, apparently.
Test Plan: Successfully rendered example diff from PHI1752.
Differential Revision: https://secure.phabricator.com/D21285
Summary:
Ref PHI1749. Instead of opening files to the last unchanged line on either side of the change, open files to the "simple" line number of the selected block.
For inlines, this is the inline line number.
For blocks, this is the first new-file line number, or the first old-file line number if no new-file line number exists in the block.
This may not always be what the user is hoping for (we can't know what the state of their working copy is) but should produce more obvious behavior.
Test Plan:
- In Diffusion, used "Open in Editor" with and without line selections. Saw same behavior as before.
- Used "n" and "r" to leave an inline with the keyboard, saw same behavior as before.
- Used "\" and "Open in Editor" menu item to open a file with:
- Nothing selected or changeset selected (line: 1).
- An inline selected (line: inline line).
- A block selected (line: first line in block, per above).
Differential Revision: https://secure.phabricator.com/D21282
Summary: Ref T13276. Ref T13513. All readers and writers were removed more than a year ago; clean up the last remnants of this table.
Test Plan: Grepped for table references, found none.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13513, T13276
Differential Revision: https://secure.phabricator.com/D21281
Summary:
Ref T13513. Syntax highlighting is potentially expensive, and the changeset rendering pipeline can cache it. However, the cache is currently keyed ONLY by Differential changeset ID.
Destroy the existing cache and rebuild it with a more standard cache key so it can be used in a more ad-hoc way by inline suggestion snippets.
Test Plan: Used Darkconsole, saw cache hits and no more inline syntax highlighting for changesets with many inlines.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21280
Summary: Ref T13513. Inline comment context information is somewhat expensive to construct and can be cached. Add a readthrough cache on top of it.
Test Plan: Loaded a source code changeset with many inline comments, used Darkconsole to inspect query activity. Saw caches get populated. Updated cache key, saw caches regenerate. Browsed Diffusion, nothing looked broken.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21279
Summary: Ref T13513. For now, I'm not supporting inline edit suggestions in Diffusion, although it's likely not difficult to do so in the future. Clean up some of the code so that plain ol' inlines work the same way they did before.
Test Plan:
- Created, edited, reloaded, submitted inlines in Diffusion: familiar old behavior.
- Created, edited, reloaded, submitted inlines with suggestions in Differential: familiar new behavior.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21278
Summary: Ref T13513. When rendering an inline suggestion for display, use highlighting and diffing.
Test Plan: {F7495053}
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21277
Summary:
Ref T13513. This still has quite a few rough edges and some significant performance isssues, but appears to mostly work.
Allow reviewers to "Suggest Edit" on an inline comment and provide replacement text for the highlighted source.
Test Plan: Created, edited, reloaded, and submitted inline comments in various states with and without suggestion text.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21276
Summary: Ref T13513. Introduce a formal server-side content state object so the whole state can be saved and restored to the drafts table, read from the request, etc.
Test Plan: Created and edited inlines. Reloaded drafts with edits. Submitted normal and editing comments. Grepped for affected symbols.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21275
Summary:
Ref T13513. If your 10 most recently authored inlines have all been deleted, these queries can fail by overheating. This is silly and probably rarely happens outside of development.
For now, just let them overheat. This may create a false negative (incorrect "no draft" signal when the real condition is "drafts, but 10 most recent comments were deleted"). This could be sorted out later with a query mode like "executeAny()", perhaps.
Test Plan:
- Created and deleted 10 inlines.
- Submitted comments.
- Before: overheating fatal during draft flag generation.
- After: clean submission.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21274
Summary:
Ref T13513. Currently, all the inline code passes around strings to describe content. I plan to add background music, animation effects, etc., soon. To prepare for this change, make content a state object.
This does not change any user-visible behavior, it just prepares for content to become more complicated than a single string.
Test Plan: Created, edited, submitted, cancelled, etc., comments.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21273
Summary: See PHI1745. Under PHP7, errors raised as Throwable miss this "generic exception" logic and don't increment their failure count. Instead, treat any "Throwable" we don't recognize like any "Exception" we don't recognize.
Test Plan:
- Under PHP7, caused a worker task to raise a Throwable (e.g., call to undefined method, see D21270).
- Ran `bin/worker execute --id ...`.
- Before: worker failed, but did not increment failure count.
- After: worker fails and increments failure count as it would for other types of unknown error.
Differential Revision: https://secure.phabricator.com/D21271
Summary: See PHI1745. This callsite for "ChangesetParser" was not properly updated for recent changes.
Test Plan:
- Set `metamta.differential.inline-patches` to 100.
- Created a new revision with a small (<100 line) diff, with at least one reviewer.
- Ran `bin/phd debug` and observed outbound mail queue with `bin/mail list-outbound`.
- Before: fatal when trying to generate the inline changes for mail.
- After: clean mail generation.
Differential Revision: https://secure.phabricator.com/D21270
Summary: See PHI1743. If a build has no initiator PHID, the rendering pathway incorrectly tries to access a handle for it anyway.
Test Plan:
- Set a build to have no initiator PHID.
- Viewed the build plan for the build.
- Before: fatal when trying to access the `null` handle.
- After: clean build plan rendering.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D21269
Summary:
Fixes T13539. See that task for discussion and a reproduction case.
This algorithm currently counts "\ No newline at end of file" lines as though they were normal source lines. This can cause offset issues in the rare case that a diff contains two of these lines (for each side of the file) and has changes between them (because the last line of the file was modified between the diffs).
Instead, don't count "\" as a display line.
Test Plan:
- See T13539 and PHI1740.
- Before: got fatals on the "wild" diff and the synthetic simplified version.
- After: clean intradiff rendering in both cases.
Maniphest Tasks: T13539
Differential Revision: https://secure.phabricator.com/D21267
Summary:
Ref T13538. See PHI1739. Synthetic Git commits with no author and/or no commit message currently extract `null` and then fail to parse.
Ideally, we would carefully distinguish between `null` and empty string. In practice, that requires significant schema changes (these columns are non-nullable and have indexing requirements) and these cases are degenerate. These commits are challenging to build and can not normally be constructed with `git commit`.
At least for now, merge the `null` cases into the empty string cases so we can survive import.
Test Plan:
- Constructed a commit with no author and no commit message using the approach described in T13538; pushed and parsed it.
- Before: fatals during identity selection and storing the commit message (both roughly NULL inserts into non-null columns).
- After: clean import.
This produces a less-than-ideal UI in Diffusion, but it doesn't break anything:
{F7492094}
Maniphest Tasks: T13538
Differential Revision: https://secure.phabricator.com/D21266
Summary:
Fixes T13536. See that task for discussion.
Older versions of MySQL (roughly, prior to 8.0.19) emit "int(10)" types. Newer versions emit "int" types. Accept these as equivalent.
Test Plan: Ran `bin/storage upgrade --force` against MySQL 8.0.11 and 8.0.20. Got clean adjustment lists on both versions.
Maniphest Tasks: T13536
Differential Revision: https://secure.phabricator.com/D21265
Summary: On the "New User" web workflow, if you use an invalid email address, you get a failure with an empty message.
Test Plan:
- Before: Tried to create a new user with address "asdf". Got no specific guidance.
- After: Got specific guidance about email address formatting and length.
Differential Revision: https://secure.phabricator.com/D21264
Summary:
Ref T13529. Now that instances can be renamed, an instance may have multiple valid SSH usernames and the preferred SSH username may not be the intenal instance name.
`PhacilitySiteSource` should already always set `diffusion.ssh-username` correctly, to the current preferred SSH username (which may be "new-name" after a rename from "old-name"), so we should never be able to reach this code without an accurate `diffusion.ssh-username` value available.
The code to resolve names into instances also already works for both "ssh old-name@..." and "ssh new-name@...".
So I believe this code has no beneficial effects and only causes harm: it may force us to return "old-name" when falling through would correctly return "new-name".
Test Plan:
- Previously: renamed an instance, then SSH'd to it using both the old and new names. Both work.
- Previously: verified that `diffusion.ssh-username` is set correctly after a rename.
- Verified that Diffusion "Clone" UI now shows "new-name" after an instance rename.
- The real question here is: does this break something I'm not thinking of? And the change probably has to go to production to answer that.
Maniphest Tasks: T13529
Differential Revision: https://secure.phabricator.com/D21259
Summary:
Ref T13513. The way I'm highlighting lines won't work for Jupyter notebooks or other complex content blocks, and I don't see an obvious way to make it work that's reasonably robust.
However, we can just ignore the range behavior for complex content and treat the entire block as selected. This isn't quite as fancy as the source behavior, but pretty good.
Also, adjust unified diff behavior to work correctly with highlighting and range selection.
Test Plan:
- Used range selection in a Jupyter notebook, got reasonable behavior (range is treated as "entire block").
- Used range selection in a unified diff, got equivalent behavior to 2-up diffs.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21257
Summary:
Ref T13513. Give selected inlines a selection state and visual cues which are similar to the changeset selection state.
Also fix a couple of minor issues with select interactions and offset comments.
Test Plan: Selected inlines, saw obvious visual cues.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21256
Summary:
Ref T13513. When a user selects a text range and uses "New Inline Comment" to create a comment directly from a range, store the offset information alongside the comment.
When hovering the comment, highlight the original range.
Test Plan: {F7480926, size=full}
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21250
Summary: Ref T13513. Support direct text selection for inlines. This is currently just an alternate way to get to the same place as using line numbers, but can preserve offset/range information in the future.
Test Plan:
- Selected some text, hit "c", clicked "New Inline Comment", got sensible comments on both sides of a diff in Safari, Chrome, and (with limitations) Firefox.
- Caveats: no unified support, doesn't work across lines in Firefox.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21248
Summary: Ref T11401. Ref T13513. This paves the way for more comment actions, particularly an edit-after-submit action.
Test Plan: Took all actions from menus, via mouse and via keyboard (where applicable).
Maniphest Tasks: T13513, T11401
Differential Revision: https://secure.phabricator.com/D21244
Summary:
Ref T13454. See <https://discourse.phabricator-community.org/t/newly-created-ssh-private-keys-with-passphrase-not-working-anymore/3883>.
After changes to distinguish between invalid and passphrase-protected keys, SSH private key management code incorrectly uses "-y ..." ("print public key") when it means "-p ..." ("modify input file, removing passphrase"). This results in the command having no effect, and Passphrase stores the raw input credential, not the stripped version.
We can't recover the keys because we don't store the passphrase, so no migration here is really possible. (We could add more code to detect this case, but it's presumably rare.)
Also, correct the behavior of the "Show Public Key" action: this is available for users who can see the credential and does not require edit permission.
Test Plan:
- Created a new credential with a passphrase, then showed the public key.
Maniphest Tasks: T13006, T13454
Differential Revision: https://secure.phabricator.com/D21245
Summary:
Ref T13513. Currently, viewing a Jupyter document, hidden context just gets a plain "* * *" facade with no way to expand it.
Support click-to-expand, like source changes.
Test Plan:
- Clicked to expand various Jupyter diffs.
- Clicked to expand normal source changes.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21243
Summary:
Ref T13513. If you leave an inline on line 20 of a Jupyter document, we currently render context around *raw* line 20, which is inevitably some unrelated piece of JSON.
Instead, drop this context. (Ideal behavior would be to render context around Jupyter block 20, but that's a whole lot of work.)
Test Plan:
- On Jupyter changes and normal source changes, made and submitted inline comments, then viewed text and HTML mail.
- Saw no context on Jupyter comments (instead of bad context), and unchanged behavior (useful context) on normal source changes.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21242
Summary:
Ref T13513. Currently, "View as Document Type..." lists every available engine.
This is hard to get completely right because we can't always rebuild the document ref accurately in the endpoint, but try harder to fake something reasonable.
Test Plan: Used "View as Document Type..." on Jupyter notebooks, was given "Jupyter" and "Source" as options.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21241
Summary:
Ref T13513. As part of inline metadata, save the document engine the change is being rendered with.
This will allow other parts of the UI to detect that an inline was created on a Jupyter notebook but is being rendered on raw source, or whatever else.
The immediate goal is to fix nonsensical inline snippet rendering in email on Jupyter notebooks.
Test Plan:
- Created inlines and replies on normal soure code, saw no document engine annotated in the database.
- Created inlines and replies on a Jupyter notebook rendered in Jupyter mode, saw "jupyter" annotations in the database.
- Swapped document engines between Jupyter and Source, etc.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21240
Summary:
Ref T13513. If an intradiff has at least one unchanged file ("hasSameEffectAs()") or more than 100 files ("Large Change"), we hit this block and don't upcast storage inlines to runtime inlines. I missed this in testing.
Add the conversion step.
Test Plan: Viewed an intradiff with at least one unchanged file and at least one inline comment, saw correct rendering instead of fatal.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21239
Summary:
Ref T13513. Currently, clicking "View" from the inline comment preview (below the "add comment" area at the bottom of the page) only works if the inline isn't being edited.
Update this behavior so it works on inlines in either "Viewing" or "Editing" states.
Test Plan:
- Clicked "View" on a normal inline, got jumped/selected.
- Clicked "View" on an editing inline, got jumped/selected.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21237
Summary:
Ref T13513. Currently:
- If you click the "Show Changeset" button, your state change doesn't actually get saved on the server.
- It's hard to select a changeset path name for copy/paste because the "highlight the header" code tends to eat the event.
Instead: persist the former event; make the actual path text not be part of the highlight hitbox.
Test Plan:
- Clicked "Show Changeset", reloaded, saw changeset visibility persisted.
- Selected changeset path text without issues.
- Clicked non-text header area to select/deselect changesets.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21236
Summary:
Ref T13513. Currently, if you:
- click a line to create an inline;
- type some text;
- wait a moment; and
- close the page.
...you don't get an "Unsubmitted Draft" marker in the revision list.
Lift all the draft behavior to "InlineController" and make saving a draft dirty the overall container draft state.
Test Plan:
- Took the steps described above, got a draft state marker.
- Created, edited, submitted, etc., inlines in Diffusion and Differential.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21235
Summary: Ref T13513. All queries now go through a reasonably minimal set of pathways and should have consistent behavior.
Test Plan:
- Loaded a revision with inlines.
- Created a new empty inline, reloaded page, saw it vanish.
- Created a new empty inline, typed draft text, did not save, reloaded page, saw draft present.
- Created a new empty inline, typed draft text. Submitted feedback, got prompt, answered "Y", saw draft text submit.
- Created a new empty inline, typed draft text, scrolled down to bottom of page, typed non-draft text, saw preview include draft text.
- Marked and submitted "Done".
- Used hide/show on inlines, verified state persisted.
- Did much of the same stuff in Diffusion, where it all works the same way (except: there's no prompt when submitting draft is-editing inlines).
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21234
Summary: Ref T13513. Replaces "DifferentialInlineCommentQuery" with the similar but more modern "DifferentialDiffInlineCommentQuery".
Test Plan: Viewed comments in timeline, changesets. Created, edited, and submitted comments. Hid and un-hid comments, reloading (saw state preserved).
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21233
Summary: Ref T13513. Continue removing usage sites for the obsolete "DifferentialInlineCommentQuery".
Test Plan: Viewed the inline list in Differential, saw sensible inlines.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21232
Summary: Ref T13513. Move querying to "DiffInlineCommentQuery" classes and lift them into the base Controller.
Test Plan: In Differential and Diffusion, created, edited, and submitted inline comments.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21231
Summary: Ref T13513. Another step closer to the light.
Test Plan: Created, edited, deleted, replied to, and submitted inline comments in Diffusion.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21230
Summary: Ref T13513. Continue marching toward coherent query pathways for all access to inline comments.
Test Plan:
- Viewed a commit and a path within that commit, as a user with unpublished inlines and a different user.
- Saw appropriate inlines in all cases (published inlines, plus undeleted unpublished inlines authored by the current viewer).
- Grepped for "loadDraftAndPublishedComments()".
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21228
Summary:
Ref T13513. Improve consistency and robustness of the "InlineComment" queries.
The only real change here is that these queries now implicitly add a clause for selecting inlines ("pathID IS NULL" or "changesetID IS NULL").
Test Plan: Browed, created, edited, and submitted inlines.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21227
Summary:
Ref T13513. Currently, inline storage objects ("TransactionComment") can't directly generate a runtime object ("InlineComment").
Allow this transformation to be performed in a genric way so clunky code which does it per-object-type can be removed, lifted, or simplified.
Simplify an especially gross callsite in preview code.
Test Plan: Previewed inline comments.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21226
Summary: Ref T13513. See that task for some discussion. This prepares to lift "loadUnsubmittedInlineComments(...)" into shared code.
Test Plan: Grepped for callers, found none in the upstream. This is a backward compatibilty break. See T13513.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21225
Summary: Ref T13513. This controller was obsoleted by EditEngine and appears unreachable without explicitly typing the URL.
Test Plan:
- Grepped for the route, didn't find any hits.
- Deleted the controller, successfully previewed comments in Diffusion.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21224
Summary:
Ref T13519. This is a little fuzzy, but I think the workflow here is:
- View an intradiff, generating an ephemeral comparison changeset with no changeset ID. This produces a state key of "*".
- Apply "hidden" state changes to the changeset.
- View some other intradiff and/or diff view.
- The code attempts to use "*" as a changset ID?
I'm not entirely sure this is accurate; this was observed in production and I couldn't get a clean reproduction case locally.
Optimistically, try making changeset IDs explicit rather than relying on state keys to be "usually changeset-ID-like".
Test Plan: Used "hidden" locally across multiple intradiffs, but I wasn't cleanly able to reproduce the initial issue.
Maniphest Tasks: T13519
Differential Revision: https://secure.phabricator.com/D21223
Summary: Ref T13523. If a file hasn't been touched in the newer changeset, we can currently hit an error in the interdiff.
Test Plan:
- Touched "moo.txt" in Diff 1.
- Reverted the changes to "moo.txt" in Diff 2.
- Diffed 2 vs 1.
- Before patch: fatal (call to getFilename() on null).
- After patch: clean interdiff.
Maniphest Tasks: T13523
Differential Revision: https://secure.phabricator.com/D21220
Summary:
Ref T13513. Overloading "original text" to get "edit-on-load" comments into the right state has some undesirable side effects.
Instead, provide the text when the editor opens. This fixes a cancel interaction.
Test Plan:
- Create an inline, type text, don't save.
- Reload page.
- Cancel.
- Before: cancelled into empty state.
- After: cancelled into deleted+undo state.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21219
Summary: Ref T13513. When users choose to publish inlines, we want to publish the visible text, not the last "checkpointed" state.
Test Plan:
- Created an inline ("AAA").
- Edited it into "BBB", did not save.
- Submitted.
- Confirmed that I want to publish the unsaved inline.
- Saw "BBB" publish.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21218
Summary:
Ref T13513. As users type text into inline comments, save the comment state as a draft on the server.
This has some rough edges, particularly around previews, but mostly works. See T13513 for notes.
Test Plan: Started an inline, typed some text, waited a second, reloaded the page, saw an editing inline with the saved text.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21216
Summary: Ref T13513. When computing whether a revision has draft comments or not, ignore empty inlines.
Test Plan: Added empty inlines to a revision, no longer saw a yellow "draft" bubble in the list UI.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21215
Summary: Ref T13513. When you load a changeset, discard all empty inlines. This is likely a more desirable behavior than keeping empty editors around, even though the rest of the pipeline generally handles them fairly well now.
Test Plan:
- Started an inline, didn't type any text or save, reloaded page.
- Before: page restores empty editor in the same place.
- After: we just discard this likely-pointless empty inline.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21214
Summary: Ref T13513. Refine some inline behaviors, see test plan.
Test Plan:
- Edit a comment ("A"), type text ("AB"), cancel, edit.
- Old behavior: edit and undo states (wrong, and undo does not function).
- New behavior: edit state only.
- Edit a comment ("A"), type text ("AB"), cancel. Undo ("AB"), cancel. Edit.
- Old behavior: "AB" (wrong: you never submitted this text).
- New behavior: "A".
- Create a comment, type text, cancel.
- Old behavior: counter appears in filetree (wrong, comment is undo-able but should not be counted).
- New behavior: no counter.
- Cancel editing an empty comment with no text.
- Old behavior: Something buggy -- undo, I think?
- New behavior: it just vanishes (correct behavior).
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21212
Summary:
Ref T13513. Currently, if you start an inline and then submit overall comments, we publish an empty inline. This is literally faithful to what you did, but almost certainly not the intent.
Instead, simply ignore empty inlines at publishing time (and ignore "done" state changes for those comments).
We could delete them outright, but if we do, they'll break if you have another window open with the empty inline (since the stored comment won't exist anymore). At least for now, leave them in place.
Test Plan: Created empty inlines, submitted comments, no longer saw them publish.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21211
Summary: Ref T13513. This slightly expands the existing-but-hacky "warning" workflow to cover both "mentions on draft" and "submitting inlines being edited".
Test Plan:
- Submitted changes to a revision with mentions on a draft, inlines being edited, both, and neither.
- Got sensible warnings in the cases where warnings were appropriate.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21191
Summary:
Ref T13513. If you submit top-level comments while an inline comment editor is open, kick the comment out of the editing state.
(An improvement to this behavior would be to warn the user that we're going to do this first, but this is currently less straightforward.)
Test Plan:
- Clicked a line number to create an inline.
- Type text, save, click edit.
- (Optional: reload page.)
- Save changes overall using the form at the bottom of the page.
- Outcome: published inline is no longer in an "editing" state.
Weirdness:
- If you click a line number (and, optionally, type text), then submit without using "Save", the server-side version of the inline has no content.
- This gives you a no-effect warning. Instead, these inlines should probably just be marked as deleted somewhere in the pipeline.
- This saves the last "Saved" copy of the inline. That's (probably?) desired, but somewhat destructive without a warning.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21188
Summary:
Ref T13513. Now that the "currently being edited" state of inlines is saved on the server side, clear the flag when the user clicks "Cancel" to leave the "editing" state on the client.
This also serves to delete empty comments.
Test Plan:
- Clicked a line number to create a new comment. Then:
- Clicked "Cancel". Reloaded page, saw no more comment.
- Typed text, saved. Reloaded page, saw non-editing draft. Clicked "Edit", reloaded page, saw editing draft. Clicked "Cancel", reloaded page, saw non-editing draft.
- Typed text, saved. Clicked "Edit", deleted text, saved. Reloaded page, saw no more comment.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21187
Summary:
Ref T13513. This is mostly an infrastructure cleanup change.
In a perfect world, this would be a series of several changes, but they're tightly interconnected and don't have an obvious clean, nontrivial partition (or, at least, I don't see one). Followup changes will exercise this code repeatedly and all of these individual mutations are "obviously good", so I'm not too worried about the breadth of this change.
---
Inline comments are stored as transaction comments in the `PhabricatorAuditTransactionComment` and `DifferentialTransactionComment` classes.
On top of these two storage classes sit `PhabricatorAuditInlineComment` and `DifferentialInlineComment`. Historically, these were an indirection layer over significantly different storage classes, but nowadays both storage classes look pretty similar and most of the logic is actually the same. Prior to this change, these two classes were about 80% copy/pastes of one another.
Part of the reason they're so copy/pastey is that they implement a parent `Interface`. They are the only classes which implement this interface, and the interface does not provide any correctness guarantees (the storage objects are not actually constrained by it).
To simplify this:
- Make `PhabricatorInlineCommentInterface` an abstract base class instead.
- Lift as much code out of the `Audit` and `Differential` subclasses as possible.
- Delete methods which no longer have callers, or have only trivial callers.
---
Inline comments have two `View` rendering classes, `DetailView` and `EditView`. They share very little code.
Partly, this is because `EditView` does not take an `$inline` object. Historically, it needed to be able to operate on inlines that did not have an ID yet, and even further back in history this was probably just an outgrowth of a simple `<form />`.
These classes can be significantly simplified by passing an `$inline` to the `EditView`, instead of individually setting all the properties on the `View` itself. This allows the `DetailView` and `EditView` classes to share a lot of code.
The `EditView` can not fully render its content. Move the content rendering code into the view.
---
Prior to this change, some operations need to work on inlines that don't have an inline ID yet (we assign an ID the first time you "Save" a comment). Since "editing" comments will now be saved, we can instead create a row immediately.
This means that all the inline code can always rely on having a valid ID to work with, even if that ID corresponds to an empty, draft, "isEditing" comment. This simplifies more code in `EditView` and allows the "create" and "reply" code to be merged in `PhabricatorInlineCommentController`.
---
Client-side inline events are currently handled through a mixture of `ChangesetList` listeners (good) and ad-hoc row-level listeners (less good). In particular, the "save", "cancel", and "undo" events are row-level. All other events are list-level.
Move all events to list-level. This is supported by all inlines now having an ID at all stages of their lifecycle.
This allows some of the client behavior to be simplified. It currently depends on binding complex ad-hoc dictionaries into event handlers in `_drawRows()`, but it seems like almost all of this code can be removed. In fact, no more than one row ever seems to be drawn, so this code can probably be simplified further.
---
Finally, save an "isEditing" state. When we rebuild a revision on the client, click the "edit" button if it's in this state. This is a little hacky, but simpler to get into a stable state, since the row layout of an inline depends on a "view row" followed by an "edit row".
Test Plan:
- Created comments on either side of a diff.
- Edited a comment, reloaded, saw edit stick.
- Saved comments, reloaded, saw save stick.
- Edited a comment, typed text, cancelled, "unedited" to get state back.
- Created a comment, typed text, cancelled, "unedited" to get state back.
- Deleted a comment, "undeleted" to get state back.
Weirdness / known issues:
- Drafts don't autosave yet.
- Fixed in D21187:
- When you create an empty comment then reload, you get an empty editor. This is a bit silly.
- "Cancel" does not save state, but should, once drafts autosave.
- Mostly fixed in D21188:
- "Editing" comments aren't handled specially by the overall submission flow.
- "Editing" comments submitted in that state try to edit themselves again on load, which doesn't work.
Subscribers: jmeador
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21186
Summary: Ref T13513. This plans for "currently editing", character range comments, code suggestions, document engine tracking. And absolutely nothing else.
Test Plan:
- Ran `bin/storage upgrade -f`, got a clean upgrade.
- Created and submitted some inline comments; nothing exploded.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21184
Summary:
At some point, the highlighting behavior for the timeline broke. When you follow a link to a particular timeline story, the story should be highlighted.
Prior to this change, the `<a />` tag itself highlights, but there's no associated CSS and it's too deep in the tree to do anything useful.
(Since this change is fairly straightforward, I gave up digging for the root cause before finding it.)
Test Plan:
- Clicked a timeline story anchor, saw the story highlight.
Differential Revision: https://secure.phabricator.com/D21213
Summary:
Ref T13523. In the caching layer, there's a tricky clause about filetypes that skips some body rendering behavior.
Provide file type information which at least has a better chance of representing all changes (e.g., an image file may be replaced with a text file, but this can not be represented by a single file type).
Formalize "hasSourceTextBody()", to mean the changeset parser should engage the change as source text.
Test Plan: Intradiffed text changes, saw the body render properly.
Maniphest Tasks: T13523
Differential Revision: https://secure.phabricator.com/D21210
Summary: Fixes T13530. The block parser could match too many lines in an unterminated "%%%" literal block. Adjust the logic to stop doing this (and hopefully be a little easier to read).
Test Plan: Added a failing test, made it pass.
Maniphest Tasks: T13530
Differential Revision: https://secure.phabricator.com/D21208
Summary:
See PHI1722, which requests transaction details about reviewer changes.
This adds them; they're structured to be similar to "projects" and "subscribers" transactions and the "reviewers" attachment on revisions.
Test Plan: {F7410675}
Differential Revision: https://secure.phabricator.com/D21207
Summary: This supports the IntelliJ IDEA editor.
Test Plan:
- Looked at the editor settings panel, saw "idea://".
- Set my editor pattern to "idea://a?b".
- (Did not actually install IntelliJ IDEA.)
Differential Revision: https://secure.phabricator.com/D21206
Summary: Ref T13528. Now that we're hinting users into Files, put the content first and move the detail panel under it. Move the most-useful details (author, size, dimensions) into the curtain.
Test Plan: {F7409925}
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21201
Summary:
Ref T13528. The original rationale here was that it's easier to find items at the bottom of the curtain than somewhere in the middle, since they're in a more clearly predictable visual location.
This might be true in some sense, but user feedback about this has fairly consistently indicated that the layout is surprising. Try the other order. See also D20967 for some discussion.
In practice, this primarily moves "Author / Assigned" above other panel elements in Maniphest.
Test Plan: Looked at tasks, aw "Author / Assigned" above "Tags / Subscribers".
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21200
Summary:
Ref T13528. Paste data is stored in files, but the files are always named "raw.txt".
Now that Paste provides a hint to use Files for "DocumentEngine" rendering, try to use the same name as the paste instead.
Test Plan:
- Created a paste named "staggering-insight.ipynb".
- Clicked "View as Jupyter Notebook" from Paste.
- Saw a file named "staggering-insight.ipynb", not "raw.txt".
- Created a paste with no name, saw a file named "raw-paste-data.txt" get created.
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21197
Summary: Ref T13528. When a file in Paste (like a Jupyter notebook) has a good/useful document engine, provide a link to Files.
Test Plan: {F7409881}
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21196
Summary:
See PHI1719. User agents making hard-coded requests to "/favicon.ico" currently 404. This is a mild source of log noise, and we can reasonably route this request.
Limitations:
- This only routes the "PlatformSite". Other sites (custom Phame blogs, third-party sites, Phurl redirectors) won't route here for now.
- This returns a "Location:" redirect to the correct resource rather than icon data directly. This produces the right icon with the right caching behavior, and returning icon data directly is difficult in the general case. However, it won't perform/cache as well as a direct response would.
Test Plan:
- Visted `/favicon.ico`.
- Before: 404.
- After: redirect to favicon.
Differential Revision: https://secure.phabricator.com/D21195
Summary:
Ref T13526. Currently, if a build plan is restricted, viewers may fatal when trying to view related builds.
The old behavior allowed them to see the build even if they can not see the build plan. This is sort of incoherent, but try to stabilize things before fixing this.
Test Plan:
This is a muddy change.
- Created a build with a build plan that Alice can't see.
- As Alice, viewed the build page (restricted before, restricted after); the buildable page (fatal before, works after).
- Also viewed a revision page (works before and after, but user-reported fatal).
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13526
Differential Revision: https://secure.phabricator.com/D21194
Summary:
See PHI1714. This code is incorrectly rendering the chart panel twice, sort of, and passing a non-View object to rendering.
After D21044, this fatals by raising an exception in rendering.
Test Plan: Loaded page, no more exception.
Differential Revision: https://secure.phabricator.com/D21185
Summary:
Ref T13520. Generally, make the table of contents look and more like the paths panel:
- Show a hierarchy, with compression for single-sibling children.
- Use the same icons, instead of "M D" and "(img)" stuff.
- Use EditDistanceMatrix to do a piece-by-piece diff of paths changes.
- Show path changes within the path list.
I'm not entirely sold on this, but it was complicated to write and I've never heard the term "sunk cost fallacy". I think this is mostly a net improvement, but may need some adjustments and followup.
Test Plan: Viewed various changes in Differential and Diffusion, saw a more usable table of contents.
Maniphest Tasks: T13520
Differential Revision: https://secure.phabricator.com/D21183
Summary:
See <https://discourse.phabricator-community.org/t/runtimeexception-during-import-of-commit/3801>. When importing commits with "Auditors:", a raw transaction new value (with an edge edit map using a "+" key) may be passed as an unmentionable PHID list.
Instead, pass an actual PHID list.
Test Plan:
- Pushed a commit with "Auditors: duck".
- Ran daemons.
- Before patch: umentionable PHID exception.
- After patch: clean commit import.
- Verified "duck" was added as an auditor.
Differential Revision: https://secure.phabricator.com/D21181
Summary:
Ref T13523. Currently, when building a "comparison" changeset, metadata is taken from the left changeset. This is somewhat arbitrary.
This means that intradiffs of images don't work properly because the rendered changeset has only the left (usually "old") information.
Later, some of the code attempts to ignore the file data stored on the changeset and reconstruct the correct file data, which is how the result ends up not-completely-wrong.
Be more careful about building sensible-ish metadata, and then just use it directly later on. This fixes the "spooky" code referencing D955 + D6851.
There are some related issues, where "change type" and "file type" are selected arbitrarily and then used to determine whether the change has an "old/new" state or not (i.e., is the left side of the diff empty, since the change creates the file)?
In many cases, neither of the original changesets have a "change type" which will answer this question correctly. Separate this concept from "has state" from "change type", and make more of the code ask narrower questions about the specific conditions or states it cares about, rather than "change type".
Test Plan:
- Created a revision with Diff 1, Diff 2, and Diff 3. Diff 1 takes an image from "null -> A". Diff 2 takes the same image from "null -> B". Diff 3 takes the same image from "A -> B'.
- Intradiffed 1v2 and 1v3.
- Before patch:
- Left side usually missing, which is incorrect (should always be "A").
- Change properties are a mess ("null -> image/png" for MIME type, e.g.)
- Uninteresting/incorrect "unix:filemode" stuff.
- After patch;
- Left side shows state "A".
- Change properties only show size changes (which is correct).
{F7402012}
Maniphest Tasks: T13523
Differential Revision: https://secure.phabricator.com/D21180
Summary:
Fixes T13525. Since D21044, the intermittent GC list warnings are treated more severely and can become user-visible errors.
Silence them, since this seems to be the only realistic response in most versions of APC/APCu.
Test Plan: Will deploy.
Maniphest Tasks: T13525
Differential Revision: https://secure.phabricator.com/D21179
Summary:
Ref T13524. If a Harbormaster lint message has no line number (which is permitted), we try to access an invalid index here. This is an exception after D21044.
Treat comments with no line number as unchanged. These comments do not have "ghost" behavior and do not port across diffs.
Test Plan:
- Used "harbormaster.sendmessage" to submit lint with no line number on a changeset.
- Viewed changeset.
- Before patch: "Undefined index: <null>" error.
- After patch: Clean changeset with lint message.
{F7400072}
Maniphest Tasks: T13524
Differential Revision: https://secure.phabricator.com/D21178
Summary:
See PHI1710. Python encodes `True` as `True` (with an uppercase "T") when building URLs.
We currently do not accept this as a "truthy" value, but it's reasonable and unambiguous. Accept "True", "TRUE", "tRuE", etc.
Test Plan: Made a cURL conduit call with "True" and "tRuE". Before patch: failure to decoded booleans; after patch: successful interpretation of "true" variations.
Differential Revision: https://secure.phabricator.com/D21177
Summary: See PHI1710. Until D21044, some transactions could omit "value" and apply correctly. This now throws an exception when accessing `$xaction['value']`. All transactions are expected to have a "value" key, so require it explicitly rather than implicitly.
Test Plan: Submitted a transaction with a "type" but no "value". After D21044, got a language-level exception. After this change, got an explicit exception.
Differential Revision: https://secure.phabricator.com/D21176
Summary: Ref T13522. When changesets update an image, we currently compute no effect hash. A content hash of the image (or other binary file) is a reasonable effect hash, and enalbes effect-hash-based behavior, including hiding files in intradiffs.
Test Plan:
- Created a revision affecting `cat.png` and `quack.txt` (currently, there must be 2+ changesets to trigger the hide logic).
- Updated it with the exact same changes.
- Viewed revision:
- Saw the image renderered in the interdiff.
- Applied patch.
- Ran `bin/differential rebuild-changesets ...`.
- Viewed revision:
- Saw both changesets collapse as unchanged.
Maniphest Tasks: T13522
Differential Revision: https://secure.phabricator.com/D21174
Summary: Ref T13518. See <https://discourse.phabricator-community.org/t/more-exceptions-when-viewing-diffs/3789/>. Under PHP 7.4, accessing an array index of values like `false` and `null` is no longer valid. This is great, but we occasionally do it.
Test Plan:
- Upgraded to PHP 7.4.
- Loaded revisions with added/changed lines, inlines, and Asana support configured.
- Before patch: saw various fatals around accessing indexes of booleans and nulls.
- After patch: clean revision.
Maniphest Tasks: T13518
Differential Revision: https://secure.phabricator.com/D21172
Summary:
Ref T13493. At time of writing, the old API method no longer functions: `1/session` does not return an `accountId` but all calls now require one.
Use the modern `3/myself` API instead. The datastructure returned by `2/user` (older appraoch) and `3/myself` (newer approach) is more or less the same, as far as I can tell.
Test Plan: Linked an account against modern-at-time-of-writing Atlassian-hosted JIRA.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21170
Summary:
Ref T13517. See that task for details about the underlying issue here.
Currently, we may decode a compressed response, then retransmit it with leftover "Content-Encoding" and "Content-Length" headers. Instead, strip these headers.
Test Plan:
- In a clustered repository setup, cloned a Git repository over HTTP.
- Before: Error while processing content unencoding: invalid stored block lengths
- After: Clean clone.
Maniphest Tasks: T13517
Differential Revision: https://secure.phabricator.com/D21167
Summary:
Ref T4369. During T13507, I set my "max_post_size" to a very small value, like 7 (i.e., 7 bytes). This essentially disables "enable_post_data_reading" even if the setting is technically on.
This breaks forms which use "multipart/form-data", which are rare but not nonexistent. Notably, forms in Config use this setting (because of `ui.header` stuff?) although perhaps they should not or no longer need to.
This can be fixed by parsing the raw input.
Since the only reason we don't parse the raw input is concern that we may not be able to read it (per documentation, but never actually observed), and we do a `strlen()` test anyway, just read it unconditionally.
This should fix cases where POST data wasn't read because of "max_post_size" without impacting anything else.
Test Plan: With very small "max_post_size", updated "ui.footer-items" in Config. Before: form acted as a no-op. After: form submitted.
Maniphest Tasks: T4369
Differential Revision: https://secure.phabricator.com/D21165
Summary: Ref T13516. This isn't terribly clean, but get the page footer into the bottom of the content page on FormationView pages so it doesn't overlap into the side panel.
Test Plan: With and without a footer, viewed normal and FormationView pages. Saw footers in appropriate places at appropriate times.
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21166
Summary:
Ref T13455. Viewstates are fairly small and will probably grow less quickly than the changeset table, but the data is also not important to retain in the long term: if you revisit a change several months after hiding some files, it's fine if we've forgotten that you adjusted the view parameters.
Add a GC with a long default collection policy (180 days) so installs can manage the size of this table if it becomes necessary.
Test Plan: Ran via `bin/garbage` to adjust the GC policy and collect viewstates.
Maniphest Tasks: T13455
Differential Revision: https://secure.phabricator.com/D21164
Summary: Ref T13516. Hide this UI on devices without the screen width to reasonably support it.
Test Plan: Viewed a revision at various window widths, saw the elements vanish at device widths and reappear at desktop widths.
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21162
Summary:
Ref T13516.
- Add an "Add Comment" navigation anchor.
- Make selection state more clear.
- Make hidden state tidier and more clear.
- Hide "View Options" in the hidden state to dodge all the weird behaviors it implies.
- Click to select/deselect changesets.
- When you open the view dropdown menu, then press "h", close the dropdown menu.
Test Plan: Fiddled with all these behaviors.
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21161
Summary:
Ref T13516. Minor improvements here:
- Show key commands in the "View Options" dropdown.
- Organize it slightly better.
- Improve disabled item behaviors a little bit.
- Add a "Browse Directory" action.
- Rename "...in Diffusion" to "...in Repository".
- Make "d", "D", and "h" use the same targeting rules as "\".
- When you hide a file with the "h" menu item, select it.
Test Plan: Poked at the menu a lot, ran into less questionable behavior.
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21160
Summary: See PHI1707, which has a Jupyter notebook which fails to diff nicely when modified. The root cause seems to be that the document does not end in a newline.
Test Plan: Applied patch, diffed the file, got a Jupyter diff out of it.
Differential Revision: https://secure.phabricator.com/D21159
Summary:
Ref T13455. Make "hidden" a changeset property similar to other changeset properties.
We don't need to render this on the server, so we make a request (to update the setting) and just discard the response.
Test Plan: {F7375468}
Maniphest Tasks: T13455
Differential Revision: https://secure.phabricator.com/D21158
Summary: Ref T13516. Mark low-importance changes (generated code, deleted files) and owned-with-authority changes in the filetree.
Test Plan: {F7375327}
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21157
Summary:
Ref T13516. Deletes all old filetree / flex / active / collapse nav code in favor of the new code.
Restores the inline tips in the path tree.
Test Plan: {F7374175}
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21154
Summary: Ref T13516. Apply basic UI styling to the new UI and make some more interaction work.
Test Plan: {F7374096}
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21153
Summary: Ref T13516. Generate a tree structure based on the page changesets. Still missing styles and a whole lot of behavior.
Test Plan: {F7373967}
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21152
Summary: Ref T13516. This glues "FormationView" to "ChangesetList". The actual tree is not functional in any meaningful way yet.
Test Plan: {F7373838}
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21151
Summary:
Ref T13516. Currently, the "File Tree" element is a semi-dynamic side panel that's implemented as a special mode of a side nav panel.
This implementation is fairly clunky, and arose from organic growth out of the side nav. As such, it has some weird behaviors, doesn't have builtin support for show/hide, and can't generalize easily.
Introduce a "FormationView" which supports loading a page up with piles of side panels in various modes.
Test Plan: No callers and no user-visible impact.
Maniphest Tasks: T13516
Differential Revision: https://secure.phabricator.com/D21150
Summary:
Ref T13515. This restores the "Open in Editor" behavior to Diffusion, and makes "\" work there.
The URI pattern is now sent as a structured template to the client, so the code will work properly if a file path contains "%l".
Test Plan:
- Clicked "Open in Editor" and pressed "\" in Diffusion when viewing a file.
- Clicked a line, hit "\", got the file opened to that line.
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21149
Summary: Ref T13515. External editor URIs currently depend on repositories having callsigns, but callsigns are no longer required. Add some variables to support configuring this feature for repositories that do not have callsigns.
Test Plan: Changed settings to use new variables, saw links generate appropriately.
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21147
Summary:
Ref T13515.
- Previously valid editor URIs may become invalid without being changed (if an administrator removes a protocol from the list, for example), but this isn't explained very well. Show an error on the settings page if the current value isn't usable.
- Generate a list of functions from an authority in the parser.
- Generate a list of protocols from configuration.
Test Plan: {F7370872}
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21146
Summary:
Ref T13515. Settings currently has some highly specialized code for rendering "Changes saved." messages. The "saved" state is communicated across a redirect-after-POST by adding `/saved/` to the end of the URI.
This isn't great. It needs a lot of moving pieces, including special accommodations in routing rules. It's user-visible. It has the wrong behavior if you reload the page or navigate directly to the "saved" URI.
Try this scheme, which is also pretty sketchy but seems like an upgrade on the balance:
- Set a cookie on the redirect which identifies the form we just saved.
- On page startup: if this cookie exists, save the value and clear it.
- If the current page started with a cookie identifying the form on the page, treat the page as a "saved" page.
This supports passing a small amount of state across the redirect-after-POST flow, and when you reload the page it doesn't keep the message around. Applications don't need to coordinate it, either. Seems somewhat cleaner?
Test Plan: In Firefox, Safari, and Chrome: saved settings, saw a "Saved changes" banner without any URI junk. Reloaded page, saw banner vanish properly.
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21144
Summary:
Ref T13515. Currently, opening a file to a particular line in an external editor relies on replacing "%l" with "%l" (which is escaped as "%25l") on the server, and then replacing "%25l" with the line number on the client. This will fail if the file path (or any other variable) contains "%l" in its unencoded form.
The parser also can't identify invalid variables.
Pull the parser out, formalize it, and make it generate an intermediate representation which can be sent to the client and reconstituted.
(This temporarily breaks Diffusion and permanently removes the weird, ancient integration in Dark Console.)
Test Plan:
- Added a bunch of tests for the actual parser.
- Used "Open in Editor" in Differential.
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21143
Summary: Ref T13515. No callsites actually use this, most editors don't support it, it doesn't seem terribly useful for the ones that do, it makes template-based APIs for line-number substitution complicated, and we can probably just loop on `window.open()` anyway.
Test Plan: Grepped for affected symbols, found no more references. Loaded settings page, saw no more setting.
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21142
Summary:
Ref T13515. Adding "\" ("Open in External Editor") made this slighlty worse, but it was already pretty bad.
Long ago the keys had a special style on them, but this got changed and dropped somewhere around D16568 -- although at the time, I think they still had a grey background (see T11654).
Some later change removed this background.
Put the background back and separate the keystrokes into groups.
Test Plan: {F7370615}
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21141
Summary:
Ref T13515. It's not intuitive that these settings are "Display Preferences", even thought they're intenrally related to some of the other display preferences.
Give them a separate group.
Test Plan: {F7370500}
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21140
Summary:
Ref T13455. Update the other "view state" properties to work like "highlight" now works.
Some complexity here arises from these concerns:
- In "View Standalone", we render the changeset inline. This is useful for debugging/development, and desirable to retain.
- In all other cases, we render the changeset with AJAX.
So the client needs to be able to learn about the "state" properties of the changeset on two different flows. Prior to this change, each pathway had a fair amount of unique code.
Then, some bookkeeping issues:
- At inital rendering time, we may not know which renderer will be selected: it may be based on the client viewport dimensions.
- Prior to this change, the client didn't separate "value of the property for the changeset as rendered" and "desired value of the property".
Test Plan:
- Viewed changes in Differential, Diffusion, and in standalone mode.
- Toggled renderer, character sets, and document engine (this one isn't terribly useful). Reloaded, saw them stick.
- Started typing a comment, cancelled it, hit the undo UI.
Maniphest Tasks: T13455
Differential Revision: https://secure.phabricator.com/D21138
Summary:
Ref T13455. Add container-level storage for persistent view state, and persist "Highlight As..." inside it.
The storage generates a "PhabricatorChangesetViewState" configuration object as an output.
When preferences are expressed on a diff and that diff is later attached to a revision, we attempt to copy the preferences.
The internal storage tracks per-changeset settings, but currently always uses "last update wins" to apply the settings in the UI.
Test Plan:
- Viewed revisions, changed highlighting, reloaded. Saw highlighting stick in revision view and standalone view.
- Viewed commits, changed highlighting, reloaded. Saw highlighting stick.
- Created a diff, changed highlighting, turned it into a revision, saw highlighting persist.
Subscribers: jmeador, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13455
Differential Revision: https://secure.phabricator.com/D21137
Summary:
Ref T13515. We "shield" some changesets, including generated code and intradiffs with no intermediate changes.
These files don't get shielded if they have inline comments.
But, if the viewer has collapsed all the comments, we can shield the file again.
Test Plan:
- Created a change affecting files A and B, with three diffs:
- Touch A and B.
- Touch B only.
- Touch nothing.
- Added an inline to A and collapsed it.
- Viewed Diff 1 vs Diff 2:
- Saw A collapse with a note about inlines.
- Saw B changes, normally.
- Viewed Diff 2 vs Diff 3:
- Saw A collapse with a note about inlines.
- Saw B collapse normally.
- Uncollapsed the inline, viewed 1v2 and 2v3, saw A expand in both cases.
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21136
Summary: Ref T13515. See PHI1661. If a file is selected, add a keystroke to click the "Open in External Editor" link.
Test Plan: In Safari, Chrome, and Firefox: used "J" to select a file, then "\" to open it in an external editor. (In Safari and Chrome, this prompts.)
Maniphest Tasks: T13515
Differential Revision: https://secure.phabricator.com/D21135
Summary: Fixes T13512. Archived packages in Owners are missing hinting, but should have it.
Test Plan:
Before:
{F7369122}
After:
{F7369128}
Maniphest Tasks: T13512
Differential Revision: https://secure.phabricator.com/D21134
Summary:
Ref PHI1292. Enable fulltext searchs in paste. Maybe this should only index a snippet instead of the entire content?
Also updates table names in `PhabricatorPasteQuery`.
Test Plan: Created some pastes, indexed them, searched for them.
Reviewers: amckinley
Subscribers: codeblock, Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D20650
Summary: Ref T13511. Currently, Ferret fulltext field functions (like "title:") are hard-coded. Modularize them so extensions may define new ones.
Test Plan: Added a new custom field which emits data for the indexer, searched for "animal-noises:moo", "animal-noises:-", etc., in global search and application search.
Maniphest Tasks: T13511
Differential Revision: https://secure.phabricator.com/D21131
Summary:
Ref T13511. Ferret functions currently define "aliases", and some applications override the default aliases.
This probably isn't really the right model, since it means the available function aliases in global search depend on the types of documents you're searching for. This isn't fundamentally unworkable but is kind of weird.
Regardless, these don't actually work. Searching for "description:x" is a syntax error.
Since they don't work, it's a good bet no one is relying on them. Just get rid of them until there's a clearer argument for the feature.
Test Plan: Grepped for "getFunctionMap", got no other hits. Ran some queries with the alias functions, got syntax errors.
Maniphest Tasks: T13511
Differential Revision: https://secure.phabricator.com/D21130
Summary:
Ref T13501. Depends on D21127. With the "prefix" behavior removed in D21127, we now have two virtually identical copies of the same code.
The newer one in Ferret is better: it slices utf8 correctly and is slightly more efficient on large inputs. Pull it out and make all callers call into it.
Test Plan:
- Grepped for all affected symbols.
- Ran `bin/search index --force ...` to reindex various objects (tasks, files).
- Searched for things in the UI.
Maniphest Tasks: T13501
Differential Revision: https://secure.phabricator.com/D21128
Summary:
Ref T13501. The older ngram code has some "prefix" behavior that tries to handle cases where a user issues a very short (one or two character) query.
This code doesn't work, presumably never worked, and can not be made to work (or, at least, I don't see a way, and am fairly sure one does not exist).
If the user searches for "xy", we can find trigrams in the form "xy*" using the index, but not in the form "*xy". The code makes a misguided effort to look for " xy", but this will only find "xy" in words that begin with "xy", like "xylophone".
For example, searching Files for "om" does not currently find "random.txt".
Remove this behavior. Without engaging the trigram index, these queries fall back to an unidexed "LIKE" table scan, but that's about the best we can do.
Test Plan: Searched for "om", hit "random.txt".
Maniphest Tasks: T13501
Differential Revision: https://secure.phabricator.com/D21127
Summary: Ref T13511. This function does nothing interesting and has no callers.
Test Plan: Grepped for callers.
Maniphest Tasks: T13511
Differential Revision: https://secure.phabricator.com/D21126
Summary:
Ref T13507. We currently compress normal responses, but do not compress file data responses because most files we serve are images and already compressed.
However, there are some cases where large files may be highly compressible (e.g., huge XML files stored in LFS) and we can benefit from compressing responses.
Make a reasonable guess about whether compression is beneficial and enable compression if we guess it is.
Test Plan:
- Used `curl ...` to download an image with `Accept-Encoding: gzip`. Got raw image data in the response (as expected, because we don't expect images to be worthwhile to recompress).
- Used `curl ...` to download a text file with `Accept-Encoding: gzip`. Got a compressed response. Decompressed the response into the original file.
Maniphest Tasks: T13507
Differential Revision: https://secure.phabricator.com/D21125
Summary:
See <https://hackerone.com/reports/850114>.
An attacker with administrator privileges can configure "notification.servers" to connect to internal services, either directly or with chosen parameters by selecting an attacker-controlled service and having it issue a "Location" redirect.
Generally, we allow this attack to occur. The same administrator can use an authentication provider or a VCS repository to perform the same attack, and we can't reasonably harden these workflows without breaking things that users expect to be able to do.
There's no reason this particular variation of the attack needs to be allowable, though, and the current behavior isn't consistent with how other similar things work.
- Hide the "notification.servers" configuration, which also locks it. This is similar to other modern service/server configuration.
- Don't follow redirects on these requests. Aphlict should never issue a "Location" header, so if we encounter one something is misconfigured. Declining to follow this header likely makes the issue easier to debug.
Test Plan:
- Viewed configuration in web UI.
- Configured a server that "Location: ..." redirects, got a followed redirect before and a failure afterward.
{F7365973}
Differential Revision: https://secure.phabricator.com/D21123
Summary:
Ref T13507. Now that we handle processing of "Content-Encoding: gzip" headers by default, this setup check can get a decompressed body back. Since it specifically wants a raw body back, disable this behavior.
Also, "@" a couple things which can get in the way if they fail now that error handling is more aggressive about throwing on warnings.
Test Plan: Ran setup check after other changes in T13507, got clean result.
Maniphest Tasks: T13507
Differential Revision: https://secure.phabricator.com/D21122
Summary: Ref T13507. If we believe the server can accept "Content-Encoding: gzip" requests, make the claim in an "X-Conduit-Capabilities" header in responses. Clients can use request compression on subsequent requests.
Test Plan: See D21119 for the client piece.
Maniphest Tasks: T13507
Differential Revision: https://secure.phabricator.com/D21120
Summary: Ref T13507. See that task for discussion.
Test Plan: Faked different response behaviors and hit both variations of this error.
Maniphest Tasks: T13507
Differential Revision: https://secure.phabricator.com/D21116
Summary:
See PHI1692. Currently, the Aphlict log is ridiculously verbose. As an initial pass at improving this:
- When starting in "debug" mode, pass "--debug=1" to Node.
- In Node, separate logging into "log" (lower-volume, more-important messages) and "trace" (higher-volume, less-important messages).
- Only print "trace" messages in "debug" mode.
Test Plan: Ran Aphlict in debug and non-debug modes. Behavior unchanged in debug mode, but log has more sensible verbosity in non-debug mode.
Differential Revision: https://secure.phabricator.com/D21115
Summary:
See PHI1692. Currently, it's hard to get a local profile or "--trace" of some Diffusion API methods, since they always proxy via HTTP -- even if the local node can serve the request.
This always-proxy behavior is intentional (so we always go down the same code path, to limit surprises) but inconvenient when debugging. Allow an operator to connect to a node which can serve a request and issue a `--local` call to force in-process execution.
This makes it straightforward to "--trace" or "--xprofile" the call.
Test Plan: Ran `bin/conduit call ...` with and without `--local` using a Diffusion method on a clustered repository. Without `--local`, saw proxy via HTTP. With `--local`, saw in-process execution.
Differential Revision: https://secure.phabricator.com/D21114
Summary: Ref T13509. In `title:big "red" dog`, keep "title" sticky across all three terms, since this seems like it's probably the best match for intent.
Test Plan: Added unit tests; ran unit tests.
Maniphest Tasks: T13509
Differential Revision: https://secure.phabricator.com/D21111
Summary:
Ref T13509. Now that the compiler can parse these queries, actually implement them.
These are fairly easy to implement:
- For present, just "JOIN". If it works, the field is present.
- For absent, we "LEFT JOIN" and then "WHERE any_column IS NULL".
Test Plan: Searched for various documents with and without fields present, got sensible results in Maniphest. For example, "body:-" finds tasks with no body, "body:- duck" finds tasks with no body and "duck" elsewhere in the content, and so on.
Maniphest Tasks: T13509
Differential Revision: https://secure.phabricator.com/D21110
Summary:
Ref T13509. Since `title:- cat` is now ambiguous, forbid spaces after operators.
Also, forbid spaces inside operators, although this has no effect today.
Test Plan: Added unit tests, ran unit tests.
Maniphest Tasks: T13509
Differential Revision: https://secure.phabricator.com/D21109
Summary:
Ref T13509. Currently, functions are "sticky", but this stickness is in the query execution layer.
Instead:
- move stickiness to the query compiler; and
- make it so that functions are not sticky if their arguments are quoted.
For example:
- `title:x y` previously meant `title:x title:y` (and still does). The "title:" is sticky.
- `title:"x" y` previously meant `title:x title:y`. It now means `title:x all:y`. The "title:" is not sticky because the argument is quoted.
Test Plan: Added unit tests, ran unit tests.
Maniphest Tasks: T13509
Differential Revision: https://secure.phabricator.com/D21108
Summary: Ref T13509. Parse "xyz:-" as "xyz is absent" and "xyz:~" as "xyz is present". These are new operators which the compiler emits separately from "not" and "substring".
Test Plan: Added unit tests, ran unit tests.
Maniphest Tasks: T13509
Differential Revision: https://secure.phabricator.com/D21107
Summary:
Ref T13509. Certain query tokens like `title:=""` are currently accepted by the parser but discarded, and have no impact on the query. This isn't desirable.
Instead, require that tokens making an assertion about field content must be nonempty.
Test Plan: Added unit tests, made them pass.
Maniphest Tasks: T13509
Differential Revision: https://secure.phabricator.com/D21106
Summary: Ref T13490. This simplifies some client behavior in the general case.
Test Plan: Called API method, saw URIs.
Maniphest Tasks: T13490
Differential Revision: https://secure.phabricator.com/D21105
Summary:
See PHI1697. If a diff is not attached to a revision (for example, if it was created with "arc diff --only"), but is attached to a repository, it is supposed to be visible only to users who can see that repository.
It currently skips this extended policy check and may incorrectly be visible to too many users.
(Once a diff is attached to a revision, this rule is enforced properly via the revision policy.)
Test Plan:
- Set repository R to be visible only to Alice.
- As Alice, created a diff from a working copy of repository R with "arc diff --only".
- As Bailey, viewed the diff.
- Before: visible diff.
- After: policy exception (as expected).
Differential Revision: https://secure.phabricator.com/D21103
Summary: Ref T13490. This simplifies mostly-theoretical cases where you're accessing Phabricator via arc-over-ssh and the Conduit protocol + domain may differ from the production protocol + domain.
Test Plan: Called API via web UI, saw sensible URI values in results.
Maniphest Tasks: T13490
Differential Revision: https://secure.phabricator.com/D21102
Summary: Ref T13505. See that task for details. When a class has exactly one "@task" block, this API returns a string. Some day, this should be made more consistent.
Test Plan: Viewed a class with exactly one "@task", no more fatal. Viewed classes with zero and more than one "@task" attributes, got clean renderings.
Maniphest Tasks: T13505
Differential Revision: https://secure.phabricator.com/D21062
Summary: Ref T13505. See that task for discussion.
Test Plan: Ran `diviner generate` locally, found a page fataling on this `strlen()`, applied patch, got a sketchy but not-broken page.
Maniphest Tasks: T13505
Differential Revision: https://secure.phabricator.com/D21061
Summary: See PHI1684. Expose the published state of the "Done" checkbox to the API.
Test Plan: Made API calls on a comment in all four states, got correct published states via the API in all cases.
Differential Revision: https://secure.phabricator.com/D21059
Summary:
See <https://discourse.phabricator-community.org/t/upgrade-from-sep-30-2016/3702/>. A user performing an upgrade from 2016 to 2020 ran into an issue where this setup query is overheating.
This is likely caused by too many rows changing state during query execution, but the particulars aren't important since this setup check isn't too critical and will catch the issue eventually. It's fine to just move on if this query fails for any reason.
Test Plan: Forced the query to overheat, loaded setup issues, got overheating fatal. Applied patch, no more fatal.
Differential Revision: https://secure.phabricator.com/D21057
Summary:
See PHI1688. If many refs with a large amount of shared ancestry are deleted from a repository, we can spend much longer than necessary marking their mutual ancestors as unreachable over and over again.
For example, if refs A, B and C all point near the head of an obsolete "develop" branch and have about 1K shared commits reachable from no other refs, deleting all three refs will lead to us performing 3,000 mark-as-unreachable operations (once for each "<ref, commit>" pair).
Instead, we can stop exploring history once we reach an already-unreachable commit.
Test Plan:
- Destroyed 7 similar refs simultaneously.
- Ran `bin/repository refs`, saw 7 entries appear in the `oldref` table.
- Ran `bin/repository discover` with some debugging statements added, saw sensible-seeming behavior which didn't double-mark any newly-unreachable refs.
Differential Revision: https://secure.phabricator.com/D21056
Summary:
Depends on D21053. Ref T11968. Three things have changed:
- Overseers can no longer use FutureIterator to continue execution of an arbitrary list of futures from any state. Use FuturePool instead.
- Same with repository daemons.
- Probably (?) fix an API change in the Harbormaster exec future.
Test Plan:
- Ran "bin/phd debug task" and "bin/phd debug pull", no longer saw Future-management related errors.
- The Harbormaster future is easiest to test by just seeing if production works once this change is deployed there.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T11968
Differential Revision: https://secure.phabricator.com/D21054
Summary:
We write some synthetic HTTP responses inside unit tests. Some responses have an indirect side effect of adjusting "zlib.output_compression", but this adjustment fails if headers have already been output. From a CLI context, headers appear to count as already-output after we write anything to stdout:
```
<?php
echo headers_sent() ? "Y" : "N";
echo "\n";
echo headers_sent() ? "Y" : "N";
echo "\n";
```
This script prints "N", then "Y".
Recently, the default severity of warnings was increased in libphutil; this has been a long-standing warning but now causes test failures.
This behavior is sort of silly but the whole thing is kind of moot anyway. Just skip it if "headers_sent()" is true.
Test Plan: Ran "arc unit --everything", got clean results.
Differential Revision: https://secure.phabricator.com/D21055
Summary: Jira allows creating projects which contain number in names, phabricator will not allow such projects but it should
Test Plan: Pasted URL with Jira project which contain number in project name and it was parsed and resolved properly in phabricator
Reviewers: epriestley, Pawka, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D21040
Summary:
Ref T13493. Google returns a lower-quality account identifier ("email") and a higher-quality account identifier ("id"). We currently read only "email".
Change the logic to read both "email" and "id", so that if Google ever moves away from "email" the transition will be a bit easier.
Test Plan: Linked/unlinked a Google account, looked at the external account identifier table.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21028
Summary:
Fixes T5028. Older versions of Git (apparently, from before 2010) did not provide a way to extract the raw body of a commit message from "git log", so we approximate it with "subject" and "wrapped body".
In newer versions of Git, the raw body can be extracted exactly.
Adjust how we extract messages based on the version of Git, and try to be more faithful to edge cases: particularly, be more careful to extract the correct number of trailing newlines.
Test Plan:
- Added "var_dump()" + "die(1)" later in this method, then pushed various commit messages. Used "&& false" to force execution down the old path (either path should work in modern Git).
- Observed more faithful extraction of messages, including a more faithful extraction of the number of trailing newlines. Extraction is fully faithful if we can go down the "%B" path, which we should be able to in nearly all modern cases.
- Not all messages extract faithfully or consistently across the old and new versions, but the old extraction is destructive so this is likely about as close as we can realistically ever get.
Maniphest Tasks: T5028
Differential Revision: https://secure.phabricator.com/D21027
Summary:
Depends on D21022. Ref T13493. The JIRA API has changed from using "key" to identify users to using "accountId".
By reading both identifiers, this linkage "just works" if you run against an old version of JIRA, a new version of JIRA, or an intermediate version of JIRA.
It also "just works" if you run old JIRA, upgrade to intermediate JIRA, everyone refreshes their link at least once, then you upgrade to new JIRA.
This is a subset of cases and does not include "sudden upgrade to new JIRA", but it's strictly better than the old behavior for all cases it covers.
Test Plan: Linked, unlinked, and logged in with JIRA. Looked at the "ExternalAccountIdentifier" table and saw a sensible value.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21023
Summary: Depends on D21019. Ref T13493. There are no more barriers to removing readers and writers of "accountID"; the new "ExternalAccountIdentity" table can replace it completely.
Test Plan: Linked and unlinked OAuth accounts, logged in with OAuth accounts, tried to double-link OAuth accounts, grepped for affected symbols.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21022
Summary:
Depends on D21018. Ref T13493. Ref T6703. The "ExternalAccount" table has a unique key on `<accountType, accountDomain, accountID>` but this no longer matches our model of reality and changes in this sequence end writes to `accountID`.
Remove this key.
Then, remove all readers of `accountType` and `accountDomain` (and all nontrivial writers) because none of these callsites are well-aligned with plans in T6703.
This change has no user-facing impact today: all the rules about linking/unlinking/etc remain unchanged, because other rules currently prevent creation of more than one provider with a given "accountType".
Test Plan:
- Linked an OAuth1 account (JIRA).
- Linked an OAuth2 account (Asana).
- Used `bin/auth refresh` to cycle OAuth tokens.
- Grepped for affected symbols.
- Published an Asana update.
- Published a JIRA link.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13493, T6703
Differential Revision: https://secure.phabricator.com/D21019
Summary: Depends on D21017. Ref T13493. Update the Asana integration so it reads the "ExternalAccountIdentifier" table instead of the old "accountID" field.
Test Plan: Linked an Asana account, used `bin/feed republish` to publish activity to Asana.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21018
Summary:
Depends on D21015. When we sync an external account and get a list of account identifiers, write them to the database.
Nothing reads them yet and we still write "accountId", this just prepares us for reads.
Test Plan: Linked, refreshed, unlinked, and re-linked an external account. Peeked at the database and saw a sensible-looking row.
Differential Revision: https://secure.phabricator.com/D21016
Summary: Depends on D21014. Ref T13493. Make these objects all use destructible interfaces and destroy sub-objects appropriately.
Test Plan:
- Used `bin/remove destroy --trace ...` to destroy a provider, a user, and an external account.
- Observed destruction of sub-objects, including external account identifiers.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21015
Summary:
Depends on D21013. Ref T13493. When users log in with most providers, the provider returns an "ExternalAccount" identifier (like an Asana account GUID) and the workflow figures out where to go from there, usually a decision to try to send the user to registration (if the external account isn't linked to anything yet) or login (if it is).
In the case of password providers, the password is really a property of an existing account, so sending the user to registration never makes sense. We can bypass the "external identifier" indirection layer and just say "username -> internal account" instead of "external GUID -> internal mapping -> internal account".
Formalize this so that "AuthProvider" can generate either a "map this external account" value or a "use this internal account" value.
This stops populating "accountID" on "password" "ExternalAccount" objects, but this was only an artifact of convenience. (These records don't really need to exist at all, but there's little harm in going down the same workflow as everything else for consistency.)
Test Plan: Logged in with a username/password. Wiped the external account table and repeated the process.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21014
Summary:
Depends on D21012. Ref T13493. Currently, auth adapters return a single identifier for each external account.
Allow them to return more than one identifier, to better handle cases where an API changes from providing a lower-quality identifier to a higher-quality identifier.
On its own, this change doesn't change any user-facing behavior.
Test Plan: Linked and unlinked external accounts.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21013
Summary:
Ref T13493. This check was introduced in D4647, but the condition can never be reached in modern Phabricator because the table has a unique key on `<accountType, accountDomain, accountID>` -- so no row can ever exist with the same value for that tuple but a different ID.
(I'm not entirely sure if it was reachable in D4647 either.)
Test Plan: Used `SHOW CREATE TABLE` to look at keys on the table and reasoned that this block can never have any effect.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21012
Summary:
Depends on D21010. Ref T13493. External accounts may have multiple different unique identifiers, most often when v1 of the API makes a questionable choice (and provies a mutable, non-unique, or PII identifier) and v2 of the API uses an immutable, unique, random identifier.
Allow Phabricator to store multiple identifiers per external account.
Test Plan: Storage only, see followup changes.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21011
Summary:
Ref T13493. The "AuthAccountView" UI element currently exposes raw account ID values, but I'm trying to make these many-to-one.
This isn't terribly useful as-is, so get rid of it. This element could use a design refresh in general.
Test Plan: Viewed the UI element in "External Accounts".
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21010
Summary:
See PHI1647, which asks for "vscode://" to be a configurable protocol on hosted Phacility instances.
I made the configuration editable in D21008, but this can reasonably just come upstream too.
Test Plan: Viewed config in Config, set my editor URI to `vscode://blahblah`.
Differential Revision: https://secure.phabricator.com/D21009
Summary:
Ref T13493. I'm updating callers to `getAccountID()` to prepare to move it to a separate table.
This callsite once supported this flow:
- External users with no accounts send mail to `bugs@`.
- This creates tasks in Maniphest.
- They're CC'd when the tasks are updated.
However, after T12237 we never actually send this mail (since their addresses are necessarily unverified).
I left this code in in case this needed to be revisited, but it hasn't been an issue. Just remove it and treat these users as undeliverable.
Test Plan: As a cursory test for nothing being horribly broken, sent some object mail.
Maniphest Tasks: T13493
Differential Revision: https://secure.phabricator.com/D21007
Summary:
Ref T13395. No library with this name loads any more, so we can't version check it.
(Ideally, the version check stuff would be more graceful when it fails now, since it's required to load "Config" after I moved it off a separate page.)
Test Plan: Loaded "Config".
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D20995
Summary: Fixes T2543. This mode has been a stable prototype for a very long time now; promote it so "--draft" can promote out of "experimental" in Arcanist.
Test Plan: See T2543 for discussion.
Maniphest Tasks: T2543
Differential Revision: https://secure.phabricator.com/D20983
Summary: Ref T13395. Moves a small amount of remaining "libphutil/" code into "phabricator/" and stops us from loading "libphutil/".
Test Plan: Browsed around; there are likely remaining issues.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D20981
Summary: Ref T13395. Move cache classes, syntax highlighters, other markup classes, and sprite sheets to Phabricator.
Test Plan: Attempted to find any callers for any of this stuff in libphutil or Arcanist and couldn't.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D20977
Summary:
`diffusion.branchquery` can return dictionary instead of array if some branches are filtered out.
Eg.:
```
{
"result": [
{
"shortName": "master",
"commitIdentifier": "2817b0d8f79748ddfad0220c46d9b20bea34f460",
"refType": "branch",
"rawFields": {
"objectname": "2817b0d8f79748ddfad0220c46d9b20bea34f460",
"objecttype": "commit",
```
might become:
```
{
"result": {
"1": {
"shortName": "master",
"commitIdentifier": "2817b0d8f79748ddfad0220c46d9b20bea34f460",
"refType": "branch",
"rawFields": {
"objectname": "2817b0d8f79748ddfad0220c46d9b20bea34f460",
"objecttype": "commit",
```
Reproduction - find repository which has couple of branches, setup to track only some of them, execute `diffusion.branchquery` API call - result is dictionary instead of array
Test Plan: Apply patch, execution `diffusion.branchquery` call - result is no longer dictionary if it was one before
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D20973
Summary: Ref T13395. Moves some Aphront classes from libphutil to Phabricator.
Test Plan: Grepped for symbols in libphutil and Arcanist.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D20975
Summary:
Fixes T5427. See PHI1630. See also T13160 and D20568.
In the full HTML table syntax with "<table>", respect linebreaks as literals inside "<td>" cells.
Test Plan: Previewed some full-HTML tables with and without linebreaks, saw what seemed like sensible rendering behavior.
Maniphest Tasks: T5427
Differential Revision: https://secure.phabricator.com/D20971
Summary: Ref T10635. An install with large blocks of remarkup (4MB) in test details is reporting slow page rendering. This is expected, but I've mostly given up on fighting this unless I absolutely have to. Degrade the interface more aggressively.
Test Plan:
- Submitted a large block of test details in remarkup format.
- Before patch: they rendered inline.
- After patch: degraded display.
- Verified small blocks are not changed.
{F7180727}
{F7180728}
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T10635
Differential Revision: https://secure.phabricator.com/D20970
Summary: Ref T13486. Currently, "Zarbo" sorts above "alice", but this isn't expected for a list of (mostly) human usernames.
Test Plan: Loaded a task with subscribers with mixed-case usernames.
Maniphest Tasks: T13486
Differential Revision: https://secure.phabricator.com/D20969
Summary: Fixes T13487. In PHI1628, an install has a 4MB remarkup corpus which takes a long time to render. This is broadly expected, but a few reasonable improvements fell out of running it through the profiler.
Test Plan:
- Saw local cold-cache end-to-end rendering time drop from 12s to 4s for the highly secret input corpus.
- Verified output has the same hashes before/after.
- Ran all remarkup unit tests.
Maniphest Tasks: T13487
Differential Revision: https://secure.phabricator.com/D20968
Summary:
Depends on D20966. Ref T13486. Curtains currently render subscribers in a plain text list, but the new ref list element is a good fit for this.
Also, improve the sorting and ordering behavior.
This makes the subscriber list take up a bit more space, but it should make it a lot easier to read at a glance.
Test Plan: Viewed object subscriber lists at varying limits and subscriber counts, saw sensible subscriber lists.
Maniphest Tasks: T13486
Differential Revision: https://secure.phabricator.com/D20967
Summary:
Ref T13486. When a curtain element like "Author" in Maniphest has a very long username, the wrapping and overflow behavior is poor: the date is obscured.
Adjust curtain elements which contain lists of references to other objects to improve wrapping behavior (put the date on a separate line) and overflow behavior (so we get a "..." when a name overflows).
Test Plan: {F7179376}
Maniphest Tasks: T13486
Differential Revision: https://secure.phabricator.com/D20966
Summary: Fixes T13335. When processing quoted blocks, we remove leading empty lines. This logic incorrectly continued after encountering a nonempty line.
Test Plan: Added a test, made it pass. Previewed blocks in web UI.
Maniphest Tasks: T13335
Differential Revision: https://secure.phabricator.com/D20965
Summary: Fixes T13485. GitHub has deprecated the "access_token" URI parameter for API authentication. Update to "Authorization: token ...".
Test Plan: Linked and unlinked a GitHub account locally.
Maniphest Tasks: T13485
Differential Revision: https://secure.phabricator.com/D20964
Summary: Ref T13480. Creating a rule in Herald currently uses the older radio-button flow. Update it to the "clickable menu" flow to simplify it a little bit.
Test Plan: Created new personal, object, and global rules. Hit the object rule error conditions.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20956
Summary:
Fixes T13463. Currently, if you use the web UI to set "Related Tasks" for a revision, the resulting commit does not link to the tasks.
If you use "Ref ..." in the message instead, the resulting commit does link to the tasks.
Broadly, this should all be cleaner (see T3577) but we can step toward better behavior by just copying these edges when commits are published.
Test Plan:
- Created a revision.
- Used the web UI to edit "Related Tasks".
- Landed the revision.
- Saw the commit link to the tasks as though I'd used "Ref ..." in the message.
Maniphest Tasks: T13463
Differential Revision: https://secure.phabricator.com/D20961
Summary:
Depends on D20933. Ref T13362. This reorganizes Config a bit and attempts to simplify it.
Subsections are now in a landing page console and groupings have been removed. We "only" have 75 values you can edit from the web UI nowadays, which is still a lot, but less overwhelming than it was in the past. And the trend is generally downward, as config is removed/simplified or moved into application settings.
This also gets rid of the "gigantic blobs of JSON in the UI".
Test Plan: Browsed all Config sections.
Maniphest Tasks: T13362
Differential Revision: https://secure.phabricator.com/D20934
Summary: Depends on D20931. Ref T13362. Move all "Console"-style interfaces to use a consistent layout based on a new "LauncherView" which just centers the content.
Test Plan: Viewed all affected interfaces.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13362
Differential Revision: https://secure.phabricator.com/D20933
Summary: Depends on D20930. Ref T13362. Put all the "Services" parts of Config in their own section.
Test Plan: Clicked through each section. This is just an organization / UI change with no significant behavioral impact.
Maniphest Tasks: T13362
Differential Revision: https://secure.phabricator.com/D20931
Summary:
Ref T13362. Config is currently doing a ton of stuff and fairly overwhelming. Separate out "Modules/Extensions" so it can live in its own section.
(This stuff is mostly useful for development and normal users rarely need to end up here.)
Test Plan: Visited seciton, clicked around. This is just a visual change.
Maniphest Tasks: T13362
Differential Revision: https://secure.phabricator.com/D20930
Summary:
Ref T13484. If you load a subproject S which has a mangled/invalid `parentPath`, the query currently tries to execute an empty edge query and fatals.
Instead, we want to deny-by-default in the policy layer but not fail the query. The subproject should become restricted but not fatal anything related to it.
See T13484 for a future refinement where we could identify "broken / data integrity issue" objects explicilty.
Test Plan:
- Modified the `projectPath` of some subproject in the database to `QQQQ...`.
- Loaded that project page.
- Before patch: fatal after issuing bad edge query.
- After patch: "functionally correct" policy layer failure, although an explicit "data integrity issue" failure would be better.
Maniphest Tasks: T13484
Differential Revision: https://secure.phabricator.com/D20963
Summary: Ref T13480. Some Herald fields need audit information, which recent changes to Herald adapters discarded. For now, just load it unconditionally.
Test Plan: Triggered an Audit-related rule locally.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20962
Summary:
Fixes T13468. See that task for discussion. The older source-rendering code mixes "line number" / "1-based" lists with "block number" / "0-based" lists and then has other bugs which cancel this out.
For block-based diffs, build an explicit block-based mask with only block numbers. This sort of sidesteps the whole issue.
Test Plan: Viewed the diff with the original reproduction case, plus various other block-based diffs, including one-block image diffs, in unified and side-by-side mode. Didn't spot any oddities.
Maniphest Tasks: T13468
Differential Revision: https://secure.phabricator.com/D20959
Summary:
Fixes T13475. Sometimes, we issue a "no op" / "default permit" / "unchallenged" MFA token, when a user with no MFA configured does something which is configured to attempt (but not strictly require) MFA.
An example of this kind of action is changing a username: usernames may be changed even if MFA is not set up.
(Some other operations, notably "Sign With MFA", strictly require that MFA actually be set up.)
When a user with no MFA configured takes a "try MFA" action, we see that they have no factors configured and issue a token so they can continue. This is correct. However, this token causes the assocaited timeline story to get an MFA badge.
This badge is incorrect or at least wildly misleading, since the technical assertion it currently makes ("the user answered any configured MFA challenge to do this, if one exists") isn't explained properly and isn't useful anyway.
Instead, only badge the story if the user actually has MFA and actually responded to some kind of MFA challege. The badge now asserts "this user responded to an MFA challenge", which is expected/desired.
Test Plan:
- As a user with no MFA, renamed a user. Before patch: badged story. After patch: no badge.
- As a user with MFA, renamed a user. Got badged stories in both cases.
Maniphest Tasks: T13475
Differential Revision: https://secure.phabricator.com/D20958
Summary: Fixes T13480. Adds the remaining missing Owners package rules for Herald commit adapters.
Test Plan: Created hooks which care about these fields, pushed commits, saw sensible transcript values.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20957
Summary: Ref T13480. The Herald "Commit" rules still use raw commit data properties to identify authors and committers. Instead, use repository identities.
Test Plan: Wrote a Herald rule using all four fields, ran it against various commits with and without known authors. Checked transcript for sensible field values.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20955
Summary:
Ref T13480. Currently, Herald commit hook rules use a raw address resolution query to identify the author and committer for a commit. This will get the wrong answer when the raw identity string has been explicitly bound to some non-default user (most often, it will fail to identify an author when one exists).
Instead, use the "IdentityEngine" to properly resolve identities.
Test Plan: Authored a commit as `X <y@example.com>`, a raw identity with no "natural" matches to users (e.g., no user with that email or username). Bound the identity to a particular user in Diffusion. Wrote a Herald pre-commit content rule, pushed the commit. Saw Herald recognize the correct user when evaluating rules.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20953
Summary:
Ref T13480. Currently, some Herald field types are rendered in an unfriendly way on transcripts. Particularly, PHID lists are rendered as raw PHIDs.
Improve this by delegating rendering to Value objects and letting "PHID List" value objects render more sensible handle lists. Also improve "bool" fields a bit and make more fields render an explicit "None" / empty value rather than just rendering nothing.
Test Plan: Viewed various transcripts, including transcripts covering boolean values, the "Always" condition, large blocks of text, and PHID lists.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20951
Summary:
Ref T13480. Currently, when Herald renders a transcript, it puts display labels into array keys. This is a bad pattern for several reasons, notably that the values must be scalar (so you can't add icons or other markup later) and the values must be unique (which is easily violated because many values are translated).
Instead, keep values as list items.
Test Plan: Viewed Herald transcripts, saw no (meaningful) change in rendering output.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20949
Summary: Ref T13480. When Herald renders rules, it partly uses a very old handle pre-loading mechanism where PHIDs are extracted and loaded upfront. This was obsoleted a long time ago and was pretty shaky even when it worked. Get rid of it to simplify the code a little.
Test Plan: Viewed Herald rules rendered into static text with PHID list actions, saw handles. Grepped for all affected methods.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20948
Summary: Ref T13480. Add an "Author's packages" field to Herald to support writing rules like "if affected packages include X, and author's packages do not include X, raise the alarm".
Test Plan: Wrote and executed rules with the field, saw a sensible field value in the transcript.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20947
Summary: Ref T13480. These fields don't serve a specific strong use case, but are broadly reasonable capabilities after "state" vs "change" actions were relaxed by T13283.
Test Plan: Wrote rules using the new fields, added and removed projects (and did neither) to fire them / not fire them. Inspected the transcripts to see the project PHIDs making it to the field values.
Maniphest Tasks: T13480
Differential Revision: https://secure.phabricator.com/D20946
Summary:
Fixes T13479. The behavior of "git rev-parse --show-toplevel" has changed in Git 2.25.0, and it now fails in bare repositories.
Instead, use "git rev-parse --git-dir" to sanity-check the working copy. This appears to have more stable behavior across Git versions, although it's a little more complicated for our purposes.
Test Plan:
- Ran `bin/repository update ...` on an observed, bare repository.
- ...on an observed, non-bare ("legacy") repository.
- ...on a hosted, bare repository.
Maniphest Tasks: T13479
Differential Revision: https://secure.phabricator.com/D20945
Summary: Ref T13472. Ref T13395. These classes are only used by Phabricator and not likely to find much use in Arcanist.
Test Plan: Grepped libphutil and Arcanist for removed symbols.
Maniphest Tasks: T13472, T13395
Differential Revision: https://secure.phabricator.com/D20939
Summary:
See <https://hackerone.com/reports/758002>. The link rules don't test that their parameters are flat text before using them in unsafe contexts.
Since almost all rules are lower-priority than these link rules, this behavior isn't obvious. However, two rules have broadly higher priority (monospaced text, and one variation of link rules has higher priority than the other), and the latter can be used to perform an XSS attack with input in the general form `()[ [[ ... | ... ]] ]` so that the inner link rule is evaluated first, then the outer link rule uses non-flat text in an unsafe way.
Test Plan:
Tested examples in HackerOne report. A simple example of broken (but not unsafe) behavior is:
```
[[ `x` | `y` ]]
```
Differential Revision: https://secure.phabricator.com/D20937
Summary: Maniphest object has `getURI` method, let's use it
Test Plan: Create event in task - URI generated as expected in email notification
Reviewers: epriestley, Pawka, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D20935
Summary:
Ref T13362. Some applications moved to fixed-width a while ago but I was generally unsatisfied with where they ended up and have been pushing them back to full-width.
Push Config back to full-width. Some of the subpages end up a little weird, but this provides more space to work with to make some improvements, like makign `maniphest.statuses` more legible in the UI>
Test Plan: Grepped for `setFixed(`, updated each page in `/config/`. Browsed each controller, saw workable full-width UIs.
Maniphest Tasks: T13362
Differential Revision: https://secure.phabricator.com/D20925
Summary:
See PHI1558. Ref T11860. Ref T13444. I partly implemented PHIDs for "UserEmail" objects, but they can't load on their own so you can't directly `bin/remove destroy` them yet.
Allow them to actually load by implementing "PolicyInterface".
Addresses are viewable and editable by the associated user, unless they are a bot/list address, in which case they are viewable and editable by administrators (in preparation for T11860). This has no real impact on anything today.
Test Plan:
- Used `bin/remove destroy <phid>` to destroy an individual email address.
- Before: error while loading the object by PHID in the query policy layer.
- After: clean load and destroy.
Maniphest Tasks: T13444, T11860
Differential Revision: https://secure.phabricator.com/D20927
Summary: Fixes T13465. This "phlog()" made some degree of sense at one time, but is no longer useful or consistent. Get rid of it. See T13465 for discussion.
Test Plan: Made a conduit call that hit a policy error, no longer saw error in log.
Maniphest Tasks: T13465
Differential Revision: https://secure.phabricator.com/D20924
Summary:
Fixes T13464. Fully-realized paths have a "path" (normalized, effective path) and a "display" path (user-facing, un-normalized path).
During transaction validation we build ref keys for paths before we normalize the "display" values. A few different approaches could be taken to resolve this, but just default the "display" path to the raw "path" if it isn't present since that seems simplest.
Test Plan: Edited paths in an Owners package, no longer saw a warning in the logs.
Maniphest Tasks: T13464
Differential Revision: https://secure.phabricator.com/D20923
Summary: Ref T13444. Allow the effects of performing an identity rebuild to be previewed without committing to any changes.
Test Plan: Ran "bin/repository rebuild-identities --all-identities" with and without "--dry-run".
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20922
Summary:
Fixes T13457. Ref T13444. When we iterate over commits in a particular repository, the default iteration strategy can't effectively use the keys on the table.
Tweak the ordering so the "<repositoryID, epoch, [id]>" key can be used.
Test Plan:
- Ran `bin/audit delete --repository X` and `bin/repository rebuild-identities --repository X` before and after changes.
- With just the key changes, performance was slightly better. My local data isn't large enough to really emphasize the key changes.
- With the page size changes, performance was a bit better (~30%, but on 1-3 second run durations).
- Used `--trace` and ran `EXPLAIN ...` on the new queries, saw them select the "<repositoryID, epoch, [id]>" key and report a bare "Using index condition" in the "Extra" column.
Maniphest Tasks: T13457, T13444
Differential Revision: https://secure.phabricator.com/D20921
Summary:
Ref T13444. Currently, many mutations to users and email addresses (particularly: user creation; and user and address destruction) do not propagate properly to repository identities.
Add hooks to all mutation workflows so repository identities get rebuilt properly when users are created, email addresses are removed, users or email addresses are destroyed, or email addresses are reassigned.
Test Plan:
- Added random email address to account, removed it.
- Added unassociated email address to account, saw identity update (and associate).
- Removed it, saw identity update (and disassociate).
- Registered an account with an unassociated email address, saw identity update (and associate).
- Destroyed the account, saw identity update (and disassociate).
- Added address X to account A, unverified.
- Invited address X.
- Clicked invite link as account B.
- Confirmed desire to steal address.
- Saw identity update and reassociate.
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20914
Summary:
Ref T13444. To interact meaningfully with "DestructionEngine", objects need a PHID. The "UserEmail" object currently does not have one (or a real "Query").
Provide basic PHID support so "DestructionEngine" can interact with the object more powerfully.
Test Plan:
- Ran migrations, checked data in database, saw sensible PHIDs assigned.
- Added a new email address to my account, saw it get a PHID.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20913
Summary: Ref T13444. Prepare to hook identity updates when user email addreses are destroyed.
Test Plan:
- Destroyed a user with `bin/remove destroy ... --trace`, saw email deleted.
- Destroyed an email from the web UI, saw email deleted.
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20912
Summary:
Ref T13444. Repository identities have, at a minimum, some bugs where they do not update relationships properly after many types of email address changes.
It is currently very difficult to fix this once the damage is done since there's no good way to inspect or rebuild them.
Take some steps toward improving observability and providing repair tools: allow `bin/repository rebuild-identities` to effect more repairs and operate on identities more surgically.
Test Plan: Ran `bin/repository rebuild-identities` with all new flags, saw what looked like reasonable rebuilds occur.
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20911
Summary: Ref T13444. Send all repository identity/detection through a new "DiffusionRepositoryIdentityEngine" which handles resolution and detection updates in one place.
Test Plan:
- Ran `bin/repository reparse --message ...`, saw author/committer identity updates.
- Added "goose@example.com" to my email addresses, ran daemons, saw the identity relationship get picked up.
- Ran `bin/repository rebuild-identities ...`, saw sensible rebuilds.
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20910
Summary: Ref T13444. This is an ancient event and part of the old event system. It is not likely to be in use anymore, and repository identities should generally replace it nowadays anyway.
Test Plan: Grepped for constant and related methods, no longer found any hits.
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20909
Summary:
Ref T13444. You can currently explicitly unassign an identity (useful if the matching algorithm is misfiring). However, this populates the "currentEffectiveUserPHID" with the "unassigned()" token, which mostly makes things more difficult.
When an identity is explicitly unassigned, convert that into an explicit `null` in the effective user PHID.
Then, realign "assigned" / "effective" language a bit. Previously, `withAssigneePHIDs(...)` actualy queried effective users, which was misleading. Finally, bulk up the list view a little bit to make testing slightly easier.
Test Plan:
- Unassigned an identity, ran migration, saw `currentEffectiveUserPHID` become `NULL` for the identity.
- Unassigned a fresh identity, saw NULL.
- Queried for various identities under the modified constraints.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20908
Summary:
Ref T13444. Currently, identities for a particular email address are queried with "LIKE" against a binary column, which makes the query case-sensitive.
- Extract the email address into a separate "sort255" column.
- Add a key for it.
- Make the query a standard "IN (%Ls)" query.
- Deal with weird cases where an email address is 10000 bytes long or full of binary junk.
Test Plan:
- Ran migration, inspected database for general sanity.
- Ran query script in T13444, saw it return the same hits for "git@" and "GIT@".
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13444
Differential Revision: https://secure.phabricator.com/D20907
Summary:
Depends on D20919. Ref T13462. When editing milestones, we currently predict they will have no members for policy evaluation purposes. This isn't the right rule.
Instead, predict that their membership will be the same as the parent project's membership, and pass this hint to the policy layer.
See T13462 for additional context and discussion.
Test Plan:
- Set project A's edit policy to "Project Members".
- Joined project A.
- Tried to create a milestone of project A.
- Before: policy exception that the edit policy excludes me.
- After: clean milestone creation.
- As a non-member, tried to create a milestone. Received appropriate policy error.
Maniphest Tasks: T13462
Differential Revision: https://secure.phabricator.com/D20920
Summary:
Ref T13462. Currently, when testing milestone edit policies during creation, the project object does not behave like a milestone:
- it doesn't have a milestone number yet, so it doesn't try to access the parent project; and
- the parent project isn't attached yet.
Instead: attach the parent project sooner (which "should" be harmless, although it's possible this has weird side effects); and give the adjusted policy object a dummy milestone number if it doesn't have one yet. This forces it to act like a milestone when emitting policies.
Test Plan:
- Set "Projects" application default edit policy to "No One".
- Created a milestone I had permission to create.
- Before: failed with a policy error, because the project behaved like a non-milestone and returned "No One" as the effective edit policy.
- After: worked properly, correctly evaluting the parent project edit policy as the effective edit policy.
- Tried to create a milestone I did not have permission to create (no edit permission on parent project).
- Got an appropriate edit policy error.
Maniphest Tasks: T13462
Differential Revision: https://secure.phabricator.com/D20919
Summary:
Fixes T13461. Some applications provide hints about policy strength in the header, but these hints are inconsistent and somewhat confusing. They don't make much sense for modern objects with Custom Forms, which don't have a single "default" policy.
Remove this feature since it seems to be confusing things more than illuminating them.
Test Plan:
- Viewed various objects, no longer saw colored policy hints.
- Grepped for all removed symbols.
Maniphest Tasks: T13461
Differential Revision: https://secure.phabricator.com/D20918
Summary: Fixes T13456. These edits are remarkup edits and should attach files, trigger mentions, and so on.
Test Plan: Created a text panel, dropped a file in. After changes, saw the file attach properly.
Maniphest Tasks: T13456
Differential Revision: https://secure.phabricator.com/D20906
Summary:
Ref T13454. Fixes T13006. When a user provide us with an SSH private key and (possibly) a passphrase:
# Try to verify that they're correct by extracting the public key.
# If that fails, try to figure out why it didn't work.
Our success in step (2) will vary depending on what the problem is, and we may end up falling through to a very generic error, but the outcome should generally be better than the old approach.
Previously, we had a very unsophisticated test for the text "ENCRYPTED" in the key body and questionable handling of the results: for example, providing a passphrase when a key did not require one did not raise an error.
Test Plan:
Created and edited credentials with:
- Valid, passphrase-free keys.
- Valid, passphrased keys with the right passphrase.
- Valid, passphrase-free keys with a passphrase ("surplus passphrase" error).
- Valid, passphrased keys with no passphrase ("missing passphrase" error).
- Valid, passphrased keys with an invalid passphrase ("invalid passphrase" error).
- Invalid keys ("format" error).
The precision of these errors will vary depending on how helpful "ssh-keygen" is.
Maniphest Tasks: T13454, T13006
Differential Revision: https://secure.phabricator.com/D20905
Summary:
Fixes T13443. When a panel raises an exception during edit action generation, it currently escapes to top level. Instead, catch it more narrowly.
Additionally, mark "ChangesetSearchEngine" as not being a suitable search engine for use in query panels. There's no list view or search URI so it can't generate a sensible panel.
Test Plan:
- Added a changeset panel to a dashboard.
- Before: entire dashboard fataled.
- After: panel fataled narrowly, menu fatals narrowly, dashboard no longer permits creation of another Changeset query panel.
Maniphest Tasks: T13443
Differential Revision: https://secure.phabricator.com/D20902
Summary:
Ref T13442. Ref T13157. There's a secret URI to look at an object's hovercard in a standalone view, but it's hard to remember and impossible to discover.
In developer mode, add an action to "View Hovercard". Also add "View Handle", which primarily shows the object PHID.
Test Plan: Viewed some objects, saw "Advanced/Developer...". Used "View Hovercard" to view hovercards and "View Handle" to view handles.
Maniphest Tasks: T13442, T13157
Differential Revision: https://secure.phabricator.com/D20887
Summary:
Ref T13453. Some of the Asana integrations also need API updates.
Depends on D20899.
Test Plan:
- Viewed "asana.workspace-id" in Config, got a sensible GID list.
- Created a revision, saw the associated Asana task get assigned.
- Pasted an Asana link I could view into a revision description, saw it Doorkeeper in the metadata.
Maniphest Tasks: T13453
Differential Revision: https://secure.phabricator.com/D20900
Summary: Ref T13453. The Asana API has changed, replacing all "id" fields with "gid", including the "users/me" API call result.
Test Plan: Linked an Asana account. Before: error about missing 'id'. After: clean link.
Maniphest Tasks: T13453
Differential Revision: https://secure.phabricator.com/D20899
Summary:
Fixes T13452. We currently give users mixed signals about the interaction mode of this text: the cursor says "text" but the behavior is "grab".
Make the behavior "text" to align with the cursor. An alternate variation of this change is to remove the cursor, but this is preferable if it doesn't cause problems, since copying the task ID is at least somewhat useful.
Test Plan: In Safari, Firefox, and Chrome: selected and copied object names from workboard cards; and dragged workboard cards by other parts of their UI.
Maniphest Tasks: T13452
Differential Revision: https://secure.phabricator.com/D20898
Summary: Ref T13425. When a change adds or removes a block (vs adding or removing a document, or changing a block), we currently try to render `null` as cell content in the unified view. Make this check broader to catch both "no opposite document" and "no opposite cell".
Test Plan: {F7008772}
Subscribers: artms
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20897
Summary: Ref T13448. Minor UI issue: setting a "Fetch Refs" value does not highlight the associated menu item, but should.
Test Plan: Set only "Fetch Refs", now saw menu item highlighted.
Maniphest Tasks: T13448
Differential Revision: https://secure.phabricator.com/D20895
Summary:
Fixes T13448. We currently "git clone" to initialize repositories, but this will fetch too many refs if "Fetch Refs" is configured.
In modern Phabricator, there's no apparent reason to "git clone"; we can just "git init" instead. This workflow naturally falls through to an update, where we'll do a "git fetch" and pull in exactly the refs we want.
Test Plan:
- Configured an observed repository with "Fetch Refs".
- Destroyed the working copy.
- Ran "bin/repository pull X --trace --verbose".
- Before: saw "git clone" pull in the world.
- After: saw "git init" create a bare empty working copy, then "git fetch" fill it surgically.
Both flows end up in the same place, this one is just simpler and does less work.
Maniphest Tasks: T13448
Differential Revision: https://secure.phabricator.com/D20894
Summary:
Ref T13448. The default behavior of "git fetch '+refs/heads/master:refs/heads/master'" is to follow and fetch associated tags.
We don't want this when we pass a narrow refspec because of "Fetch Refs" configuration. Tell Git that we only want the refs we explicitly passed.
Note that this doesn't prevent us from fetching tags (if the refspec specifies them), it just stops us from fetching extra tags that aren't part of the refspec.
Test Plan:
- Ran "bin/repository pull X --trace --verbose" in a repository with a "Fetch Refs" configuration, saw only "master" get fetched (previously: "master" and reachable tags).
- Ran "git fetch --no-tags '+refs/*:refs/*'", saw tags fetched normally ("--no-tags" does not disable fetching tags).
Maniphest Tasks: T13448
Differential Revision: https://secure.phabricator.com/D20893
Summary:
Ref T13448. When "Fetch Refs" is configured:
- We switch to a narrow mode when running "ls-remote" against the local working copy. This excludes surplus refs, so we'll incorrectly detect that the local and remote working copies are identical in cases where the local working copy really has surplus refs.
- We rely on "--prune" to remove surplus local refs, but it only prunes refs matching the refspecs we pass "git fetch". Since these refspecs are very narrow under "Fetch Only", the pruning behavior is also very narrow.
Instead:
- When listing local refs, always list everything. If we have too much stuff locally, we want to get rid of it.
- When we identify surplus local refs, explicitly delete them instead of relying on "--prune". We can just do this in all cases so we don't have separate "--prune" and "manual" cases.
Test Plan:
- Created a new repository, observed from a GitHub repository, with many tags/refs/branches. Pulled it.
- Observed lots of refs in `git for-each-ref`.
- Changed "Fetch Refs" to "refs/heads/master".
- Ran `bin/repository pull X --trace --verbose`.
On deciding to do something:
- Before: since "master" did not change, the pull declined to act.
- After: the pull detected surplus refs and deleted them. Since the states then matched, it declined further action.
On pruning:
- Before: if the pull was forced to act, it ran "fetch --prune" with a narrow refspec, which did not prune the working copy.
- After: saw working copy pruned explicitly with "update-ref -d" commands.
Also, set "Fetch Refs" back to the default (empty) and pulled, saw everything pull.
Maniphest Tasks: T13448
Differential Revision: https://secure.phabricator.com/D20892
Summary:
Fixes T13446. Currently, the validation logic here rejects a rename like "alice" to "ALICE" (which changes only letter case) but this is a permissible rename.
Allow collisions that collide with the same user to permit this rename.
Also, fix an issue where an empty rename was treated improperly.
Test Plan:
- Renamed "alice" to "ALICE".
- Before: username collision error.
- After: clean rename.
- Renamed "alice" to "orange" (an existing user). Got an error.
- Renamed "alice" to "", "!@#$", etc (invalid usernames). Got sensible errors.
Maniphest Tasks: T13446
Differential Revision: https://secure.phabricator.com/D20890
Summary: Fixes T13445. Make the meaning of this condition more clear, since the current wording is ambiguous between "any of" and "all of".
Test Plan: Edited a Herald rule with a PHID list field ("Project tags"), saw text label say "include none of".
Maniphest Tasks: T13445
Differential Revision: https://secure.phabricator.com/D20889
Summary:
Fixes T13441. Internally, projects can be queried by depth, but this is not exposed in the UI.
Add a "Is root project?" contraint in the UI, and "minDepth" / "maxDepth" constraints to the API.
Test Plan:
- Used the UI to query root projects, got only root projects back.
- Used "project.search" in the API to query combinations of root projects and projects at particular depths, got matching results.
Maniphest Tasks: T13441
Differential Revision: https://secure.phabricator.com/D20886
Summary: Ref T13440. Give the table more obvious visual structure and get rid of the largely useless header columns.
Test Plan: Viewed table, saw a slightly cleaner result.
Maniphest Tasks: T13440
Differential Revision: https://secure.phabricator.com/D20885
Summary: Depends on D20883. Ref T13440. In most cases, all changes belong to the same repository, which makes the "Repository" column redundant and visually noisy. Show repository information in a section header.
Test Plan: {F6989932}
Maniphest Tasks: T13440
Differential Revision: https://secure.phabricator.com/D20884
Summary: Depends on D20882. Ref T13440. Instead of lists of "Differential Revisions" and "Commits", show all changes related to the task in a tabular view.
Test Plan: {F6989816}
Maniphest Tasks: T13440
Differential Revision: https://secure.phabricator.com/D20883
Summary: Ref T13440. This feature is used in only one interface which I'm about to rewrite, so throw it away.
Test Plan: Grepped for all affected symbols, didn't find any hits anywhere.
Maniphest Tasks: T13440
Differential Revision: https://secure.phabricator.com/D20882
Summary:
Ref T12164. Ref T13439. Commit hovercards don't currently show the repository. Although this is sometimes obvious from context, it isn't at other times and it's clearly useful/important.
Also, use identities to render author/committer information and show committer if the committer differs from the author.
Test Plan: {F6989595}
Maniphest Tasks: T13439, T12164
Differential Revision: https://secure.phabricator.com/D20881
Summary:
On fresh installation which doesn't have yet any task closed you will not be able to open charts because of error below:
Fixes:
```
[Mon Oct 21 15:42:41 2019] [2019-10-21 15:42:41] EXCEPTION: (TypeError) Argument 1 passed to head_key() must be of the type array, null given, called in ..phabricator/src/applications/fact/chart/PhabricatorFactChartFunction.php on line 86 at [<phutil>/src/utils/utils.php:832]
[Mon Oct 21 15:42:41 2019] #0 phlog(TypeError) called at [<phabricator>/src/aphront/handler/PhabricatorAjaxRequestExceptionHandler.php:27]
[Mon Oct 21 15:42:41 2019] #1 PhabricatorAjaxRequestExceptionHandler::handleRequestThrowable(AphrontRequest, TypeError) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:797]
[Mon Oct 21 15:42:41 2019] #2 AphrontApplicationConfiguration::handleThrowable(TypeError) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:345]
[Mon Oct 21 15:42:41 2019] #3 AphrontApplicationConfiguration::processRequest(AphrontRequest, PhutilDeferredLog, AphrontPHPHTTPSink, MultimeterControl) called at [<phabricator>/src/aphront/configuration/AphrontApplicationConfiguration.php:214]
[Mon Oct 21 15:42:41 2019] #4 AphrontApplicationConfiguration::runHTTPRequest(AphrontPHPHTTPSink) called at [<phabricator>/webroot/index.php:35
```
To fix issue - lets return empty data set instead
Test Plan:
1) Create fresh phabricator installation
2) Create fresh project
3) Try viewing charts
Reviewers: epriestley, Pawka, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, yelirekim
Differential Revision: https://secure.phabricator.com/D20861
Summary:
Ref T13279. See PHI1491. Currently, the top-level "Burnup Rate" chart in Maniphest shows total created tasks above the X-axis, without adjusting for closures.
This is unintended and not very useful. The filtered-by-project charts show the right value (cumulative open tasks, i.e. open minus close). Change the value to aggregate creation events and status change events.
Test Plan: Viewed top-level chart, saw the value no longer monotonically increasing.
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20879
Summary: Ref T13438. This is a sort of minimal plausible implementation.
Test Plan: Used "harbormaster.artifact.search" to query information about artifacts.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13438
Differential Revision: https://secure.phabricator.com/D20878
Summary:
Fixes T13435. If you move Phabricator or copy data from one environment to another, the repository URI index currently still references the old URI, since it writes the URI as a plain string. This may make "arc which" and similar workflows have difficulty identifying repositories.
Instead, store the "phabricator.base-uri" domain and the "diffusion.ssh-host" domain as tokens, so lookups continue to work correctly even after these values change.
Test Plan:
- Added unit tests to cover the normalization.
- Ran migration, ran daemons, inspected `repository_uriindex` table, saw a mixture of sensible tokens (for local domains) and static domains (like "github.com").
- Ran this thing:
```
$ echo '{"remoteURIs": ["ssh://git@local.phacility.com/diffusion/P"]}' | ./bin/conduit call --method repository.query --trace --input -
Reading input from stdin...
>>> [2] (+0) <conduit> repository.query()
>>> [3] (+3) <connect> local_repository
<<< [3] (+3) <connect> 555 us
>>> [4] (+5) <query> SELECT `r`.* FROM `repository` `r` LEFT JOIN `local_repository`.`repository_uriindex` uri ON r.phid = uri.repositoryPHID WHERE (uri.repositoryURI IN ('<base-uri>/diffusion/P')) GROUP BY `r`.phid ORDER BY `r`.`id` DESC LIMIT 101
<<< [4] (+5) <query> 596 us
<<< [2] (+6) <conduit> 6,108 us
{
"result": [
{
"id": "1",
"name": "Phabricator",
"phid": "PHID-REPO-2psrynlauicce7d3q7g2",
"callsign": "P",
"monogram": "rP",
"vcs": "git",
"uri": "http://local.phacility.com/source/phabricator/",
"remoteURI": "https://github.com/phacility/phabricator.git",
"description": "asdf",
"isActive": true,
"isHosted": false,
"isImporting": false,
"encoding": "UTF-8",
"staging": {
"supported": true,
"prefix": "phabricator",
"uri": null
}
}
]
}
```
Note the `WHERE` clause in the query normalizes the URI into "<base-uri>", and the lookup succeeds.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13435
Differential Revision: https://secure.phabricator.com/D20872
Summary:
Ref T13425. The changes in D20865 could incorrectly lead to selection of a DocumentEngine that can not generate document diffs if a file was added or removed (for example, when a source file is added).
Move the engine pruning code to be shared -- we should always discard engines which can't generate a diff, even if we don't have both documents.
Test Plan: Viewed an added source file, no more document ref error arising from document engine selection.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20866
Summary: Ref T13425. When a file (like a Jupyter notebook) is added or removed, we can still render a useful line-by-line diff.
Test Plan:
- Viewed add/modify/remove of Jupyter, source code, and images in 2up/1up mode, everything looked okay.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20865
Summary:
Fixes T13431.
Increase the "panel" version of project member lists to 10 users, hide disabled users, swap the buttons to "tail buttons".
Sort disabled users to the bottom of "full list" versions of member lists.
For UI consistency, render the remove "X" as disabled but visible if users don't have permission to remove members.
Test Plan:
- Viewed a project with disabled members.
- Saw only enabled members on the main project page.
- Saw disabled members sorted to the bottom on the members page.
- Clicked "View All" to jump from the panel to the members page.
- As a user who could not edit a project, viewed the members page and saw a disabled "X" with a policy error when clicked.
- Removed a member as before, as a normal user with permission to remove members.
Maniphest Tasks: T13431
Differential Revision: https://secure.phabricator.com/D20864
Summary:
Fixes T13433. Currently, "Login Screen Instructions" in "Auth" are shown only on the main login screen. If you enter a bad password or bad LDAP credential set and move to the flow-specific login failure screen (for example, "invalid password"), the instructions vanish.
Instead, persist them. There are reasonable cases where this is highly useful and the cases which spring to mind where this is possibly misleading are fairly easy to fix by making the instructions more specific.
Test Plan:
- Configured login instructions in "Auth".
- Viewed main login screen, saw instructions.
- Entered a bad username/password and a bad LDAP credential set, got kicked to workflow sub-pages and still saw instructions (previously: no instructions).
- Grepped for other callers to `buildProviderPageResponse()` to look for anything weird, came up empty.
Maniphest Tasks: T13433
Differential Revision: https://secure.phabricator.com/D20863
Summary: Fixes T13430. Provide more information about repositories in "diffusion.repository.search".
Test Plan: Used API console to call method (with new "metrics" attachment), reviewed output. Saw new fields returned.
Maniphest Tasks: T13430
Differential Revision: https://secure.phabricator.com/D20862
Summary:
Ref T13429. It's currently possible to write "TYPE_EDGE" relationships for the "object has project" edge to PHIDs which may not actually be projects. Today, this fatals.
As a first step, unfatal it. T13429 discusses general improvements and greater context.
Test Plan:
Used "maniphest.edit" to write a "project" edge to a user PHID, viewed the task in the UI. Previously it fataled; now it renders unusually (the object is "tagged" with a user) but faithfully reflects database state.
{F6957606}
Maniphest Tasks: T13429
Differential Revision: https://secure.phabricator.com/D20860
Summary: See PHI1499. This error message doesn't provide parameters, and can be a little bit more helpful.
Test Plan: {F6957550}
Differential Revision: https://secure.phabricator.com/D20859
Summary:
Depends on D20853. See PHI1474. If the list of "--not" refs is sufficiently long, we may exceed the maximum size of a command.
Use "--stdin" instead, and swap "--not" for the slightly less readable but functionally equivalent "^hash", which has the advantage of actually working with "--stdin".
Test Plan: Ran `bin/repository refs ...` with nothing to be done, and with something to be done.
Differential Revision: https://secure.phabricator.com/D20854
Summary: See PHI1474. This query can become large enough to exceed reasonable packet limits. Chunk the query so it is split up if we have too many identifiers.
Test Plan: Ran `bin/repository refs ...` on a repository with no new commits and a repository with some new commits.
Differential Revision: https://secure.phabricator.com/D20853
Summary: If portal is created with id > 9 - then those portals are not reachable and always return 404.
Test Plan: Create at least 10 portals, 10th and rest of them will no longer return 404.
Reviewers: epriestley, Pawka, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D20852
Summary: Ref T13425. When we render a diff between two source lines, highlight intraline changes.
Test Plan: Viewed a Jupyter notebook with a code diff in it, saw the changed subsequences in the line highlighted.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20851
Summary:
See PHI1468. Engine selection for diffs is currently too aggressive in trying to find a shared engine and will fall back a shared engine with a very low score, causing all ".json" files to render as Jupyter files.
Only pick an engine as a difference engine by default if it's the highest-scoring engine for the new file.
Test Plan: Viewed ".json" files and ".ipynb" files in a revision. Before, both rendered as Jupyter. Now, the former rendered as JSON and the latter rendered as Jupyter.
Differential Revision: https://secure.phabricator.com/D20850
Summary:
Depends on D20844. Ref T13425. When we line up two blocks and they can be interdiffed (generally: they both have the same type of content), let the Engine interdiff them.
Then, make the Jupyter engine interdiff markdown.
Test Plan: {F6898583}
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20845
Summary:
Depends on D20843. Ref T13425. Add very basic support for "Show Hidden Context", in the form of folding it behind an unclickable shield. This isn't ideal, but should be better than nothing.
Prepare for "intraline" diffs on content blocks.
Fix newline handling in Markdown sections in Jupyter notebooks.
Remove the word "visibile" from the codebase.
Test Plan: {F6898192}
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20844
Summary:
Ref T13425. Some diff checks currently sequence incorrectly:
- When we're rendering block lists, syntax highlighting isn't relevant.
- The "large change" guard can prevent rendering of otherwise-renderable changes.
- Actual errors in the document engine (like bad JSON in a ".ipynb" file) aren't surfaced properly.
Improve sequencing somewhat to resolve these issues.
Test Plan:
- Viewed a notebook, no longer saw a "highlighting disabled" warning.
- Forced a notebook to fail, got a useful inline error instead of a popup dialog error.
- Forced a notebook to have a large number of differences, got a rendering out of it.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20843
Summary: This text is significantly more clear and helpful for users.
Test Plan: Tried to do something I'm not suppposed to, hit the 403 page.
Differential Revision: https://secure.phabricator.com/D20847
Summary: See PHI1466. When an install defines task subtypes, show them on the task graph.
Test Plan:
- On desktop with subtypes defined, column is visible.
- On desktop with subtypes not defined, column is hidden.
- On mobile, column is hidden.
{F6896845}
Differential Revision: https://secure.phabricator.com/D20842
Summary:
See PHI1442. If you have a bulk-editable datasource field with a composite datasource, it can currently fatal on the bulk edit workflow because the viewer is not passed correctly.
The error looks something like this:
> Argument 1 passed to PhabricatorDatasourceEngine::setViewer() must be an instance of PhabricatorUser, null given, called in /Users/epriestley/dev/core/lib/phabricator/src/applications/typeahead/datasource/PhabricatorTypeaheadCompositeDatasource.php on line 231
Test Plan: Configured a Maniphest custom field with a composite datasource, then tried a bulk edit. Things worked cleanly instead of fataling.
Differential Revision: https://secure.phabricator.com/D20841
Summary:
Ref T13425. Currently, code inputs and all outputs are grouped into a single block. This is fine for display notebooks but not great for diffing notebooks.
Instead, split source code input into individual lines with one line per block, and each output into its own block.
This allows you to leave actual line-by-line inlines on source, and comment on outputs individually.
Test Plan: {F6888583}
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20840
Summary: Depends on D20838. Fixes T13414. Instead of doing coarse diffing with "PhutilEditDistanceMatrix", use hash-and-diff with "DocumentEngine".
Test Plan:
- On a large document (~3K top level blocks), saw a more sensible diff, instead of the whole thing falling back to "everything changed" mode.
- On a small document, still saw a sensible granular diff.
{F6888249}
Maniphest Tasks: T13414
Differential Revision: https://secure.phabricator.com/D20839
Summary: Depends on D20836. Ref T13414. Ref T13425. Ref T13395. Move these to "phabricator/" before trying to improve the high-level diff engine in prose diffs.
Test Plan: Ran "arc liberate", looked at a prose diff (no behavioral change).
Maniphest Tasks: T13425, T13414, T13395
Differential Revision: https://secure.phabricator.com/D20838
Summary:
Depends on D20835. Ref T13425. Ref T13414. When a document has a list of content blocks, we may not be able to diff it directly, but we can hash each block and then diff the hashes (internally "diff" also does approximately the same thing).
We could do this ourselves with slightly fewer layers of indirection, but: diff already exists; we already use it; we already have a bunch of abstractions on top of it; and it's likely much faster on large inputs than the best we can do in PHP.
Test Plan: {F6888169}
Maniphest Tasks: T13425, T13414
Differential Revision: https://secure.phabricator.com/D20836
Summary:
Depends on D20834. Ref T13425. After the change from "th" to "td" for accessibility, the algorithm picks which cells it should highlight slightly improperly (it picks too many cells since it can no longer find the line numbers).
Ideally, it would probably highlight //only// the source content, but there isn't an easy way to do this right now. Settle for an incremental improvement for the moment.
Test Plan: Hovered over line numbers, saw a more accurate highlight area.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20835
Summary: Depends on D20833. Ref T13425. This look like it "just works"?
Test Plan: Left inline comments on a Juptyer notebook. Nothing seemed broken? Confusing.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20834
Summary:
Depends on D20832. Ref T13425. Emit Jupyter notebooks as diffable blocks with block keys.
No diffing or proper inlines yet.
Test Plan: {F6888058}
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20833
Summary:
Depends on D20831. Ref T13425. As an escape hatch to get out of future DocumentEngine rendering behavior, provide a "View As.." option.
Now I can break DocumentEngine real bad and no one can complain.
Test Plan: Used "View As" to swap document engines for image files.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20832
Summary:
Depends on D20830. Ref T13425. Have the image engine elect into block rendering, then emit blocks.
This is rough (the blocks aren't actually diffed yet) but image diffs were already pretty rough so this is approximately a net improvement.
Test Plan: Viewed image diffs, saw nothing worse than before.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20831
Summary:
Ref T13425. Allow DocumentEngines to claim they can produce diffs. If both sides of a change can be diffed by the same document engine and it can produce diffs, have it diff them.
This has no impact on runtime behavior because no upstream engine elects into diff generation yet.
Test Plan: Loaded some revisions, nothing broke.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20830
Summary:
Fixes T13284. See that task for substantial discussion. There are currently two cases where we'll skip over commits which we should publish:
- if a branch is not permanent, then later made permanent; or
- in some cases, the first time we examine branches in a repository.
In both cases, this error is one-shot and things work correctly going forward. The root cause is conflation between the states "this ref currently permanent" and "this ref was permanent the last time we updated refs".
Separate these pieces of state and cover all these cases. Also introduce a "--rebuild" flag to fix the state of bad commits.
Test Plan:
See T13284 for the three major cases:
- initial import;
- push changes to a nonpermanent branch, update, then make it permanent;
- push chanegs to a nonpermanent branch, update, push more changes, then make it permanent.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13284
Differential Revision: https://secure.phabricator.com/D20829
Summary: Fixes T13420. Allow installs to provide username change instructions if there's someone you should contact to get this done.
Test Plan: {F6885027}
Maniphest Tasks: T13420
Differential Revision: https://secure.phabricator.com/D20828
Summary: Ref T13420. This workflow currently dead-ends for non-administrators. Instead, provide explanatory text.
Test Plan:
- Clicked "Change Username" as an administrator, same workflow as always.
- Clicked "Change Username" as a non-administrator, got explanatory text.
Maniphest Tasks: T13420
Differential Revision: https://secure.phabricator.com/D20827
Summary:
Ref T13420. These warnings are currently more severe than they need to be; weaken them.
Among other cases, the upstream supports and encourages changing usernames when users change human names.
The login/password instructions are also out of date since sessions were decoupled from usernames about a year ago.
Test Plan: Hit dialog as an administrator
Maniphest Tasks: T13420
Differential Revision: https://secure.phabricator.com/D20826
Summary: Ref T13410. Fixes T4280. Allows you to put a named anchor into a document explicitly.
Test Plan: Used `{anchor ...}` in Remarkup, used location bar to jump to anchors.
Maniphest Tasks: T13410, T4280
Differential Revision: https://secure.phabricator.com/D20825
Summary:
Depends on D20820. Ref T13410. We currently cut anchor names in the middle, don't support emoji in anchors, and generate relatively short anchors.
Generate slightly longer anchors, allow more unicode, and try not to cut things in the middle.
Test Plan: Created a document with a variety of different anchors and saw them generate more usable names.
Maniphest Tasks: T13410
Differential Revision: https://secure.phabricator.com/D20821
Summary: Fixes T13423. The "Query" class for conduit call logs is missing a "withIDs()" method.
Test Plan: Paged through Conduit call logs.
Maniphest Tasks: T13423
Differential Revision: https://secure.phabricator.com/D20823
Summary: Ref T13279. Facts is still fairly rough, but not broken/policy-violating, so it can be unprototyped to fix the issue where Maniphest reports (which are now driven by Facts) don't work if prototypes are disabled.
Test Plan: Viewed Maniphest reports and Project reports with prototypes on/off and Fact installed/uninstalled.
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20822
Summary:
Depends on D20818. Ref T13279. The behavior of the "burndown" chart has wandered fairly far afield; make it look more like a burndown.
Move the other thing into an "Activity" chart.
Test Plan: {F6865207}
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20819
Summary:
Ref T13279. Allow engines to choose how areas in a stacked area chart stack on top of one another.
This could also be accomplished by using multiple stacked area datasets, but datasets would still need to know if they're stacking "up" or "down" so it's probably about the same at the end of the day.
Test Plan: {F6865165}
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20818
Summary: Ref T13279. Fix some tabular stuff, draw areas better, make the "compose()" API more consistent, unfatal the demo chart, unfatal the project burndown, make the project chart do something roughly physical.
Test Plan: Looked at charts, saw fewer obvious horrors.
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20817
Summary: Depends on D20815. Ref T13279. Give datapoints "refs", which allow us to figure out where particular datapoints came from even after the point is transformed by functions. For now, show the raw points in a table below the chart.
Test Plan: Viewed chart data, saw reasonable-looking numbers.
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20816
Summary:
Depends on D20814. Currently, "min()" and "max()" are still "min(f, n)". This is no longer consistent with the construction of functions a function-generators that are composed at top level.
Turn them into "min(n)" and "max(n)" (i.e., not higher-order functions).
Then, mark all the functions which are pure mathematical functions and not higher-order as "pure". These functions have no function parameters and do not reference external data. For now, this distinction has no immediate implications, but it will simplify the next change (which tracks where data came from when it originated from an external source -- these pure functions never have any source information, since they only apply pure mathematical transformations to data).
Test Plan: Loaded a burnup chart, nothing seemed obviously broken.
Subscribers: yelirekim
Differential Revision: https://secure.phabricator.com/D20815
Summary: Ref T13279. We currently draw a point on the chart for each datapoint, but this leads to many overlapping circles. Instead, aggregate the raw points into display points ("events") at the end.
Test Plan: Viewed a stacked area chart with many points, saw a more palatable number of drawn dots.
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20814
Summary: This psueudocode should use the result of computation at the end.
Test Plan: Read carefully.
Differential Revision: https://secure.phabricator.com/D20813
Summary: Ref T13411. This is a leftover from recent policy rendering changes.
Test Plan: Viewed feed with application policy stories, no more fatal.
Maniphest Tasks: T13411
Differential Revision: https://secure.phabricator.com/D20811
Summary:
Fixes T13415. Provide a way for subtypes to customize the behavior of "Change Subtype" actions that appear above comment areas.
Subtypes may disable this action by specifying `"mutations": []`, or provide a list of subtypes.
The bulk editor and API can still perform any change.
Test Plan:
- Tried to define an invalid "mutations" list with a bad subtype, got a sensible error.
- Specified a limited mutations list and an empty mutations list, verified that corresponding tasks got corresponding actions.
- Used the bulk editor to perform a freeform mutation.
- Verified that tasks of a subtype with no "mutations" still work the same way they used to (allow mutation into any subtype).
Maniphest Tasks: T13415
Differential Revision: https://secure.phabricator.com/D20810
Summary: See PHI1434. For objects that support subtypes and have subtypes configured, allow Herald rules to act on subtypes.
Test Plan:
- Configured task and project subtypes, wrote Herald rules, saw "Subtypes" as an option, saw appropriate typeahead values and detail page rendering.
- Unconfigured project subtypes, saw field vanish from UI for new rules.
- Wrote a "subtype"-depenent rule that added a comment, interacted with tasks of that subtype and a different subtype. Saw Herald act only on tasks with the correct subtype.
Differential Revision: https://secure.phabricator.com/D20809
Summary:
Fixes T7961. Currently, we present Herald users with actions like "Require legalpad signatures" and "Run build plans" even if Legalpad and Harbormaster are not installed.
Instead, allow fields and actions to be made "unavailable", which means that we won't present them as options when adding to new or existing rules.
If you edit a rule which already uses one of these fields or actions, it isn't affected.
Test Plan:
- Created a rule with a legalpad action, uninstalled legalpad, edited the rule. Action remained untouched.
- Created a new rule, wasn't offered the legalpad action.
- Reinstalled the application, saw the action again.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T7961
Differential Revision: https://secure.phabricator.com/D20808
Summary:
Fixes T13411. This looks like the last case where you hit a policy explanation and have permission to see the policy, but we don't currently show you the policy rules.
This implementation is slightly clumsy, but likely harmless.
Test Plan: {F6856421}
Maniphest Tasks: T13411
Differential Revision: https://secure.phabricator.com/D20807
Summary:
Ref T13411. Currently, if you hit a policy exception because you can't view an object, we disclose details about the view policy of the object, particularly which project's members can see the object for project policies.
Although there's a large amount of grey area here, this feels like a more substantial disclosure than we offer in other contexts. Instead, if you encounter a policy exception while testing "CAN_VIEW" or don't have "CAN_VIEW", present an "opaque" explanation which omits details that viewers who can't view the object shouldn't have access to. Today, this is the name of "Project" policies (and, implicitly, the rulesets of custom policies, which we now disclose in other similar contexts).
Test Plan:
- Hit policy exceptions for "CAN_VIEW" on an object with a project view policy, saw an opaque explanation.
- Hit policy exceptions for "CAN_EDIT" on an object with a project edit policy and a view policy I satisfied, saw a more detailed explanation.
Maniphest Tasks: T13411
Differential Revision: https://secure.phabricator.com/D20806
Summary: Ref T13411. When users click a link to explain a capability (like the policy header on many objects, or the link next to specific capabilities in "Applications", "Diffusion", etc), inline the full ruleset for the custom policy into the dialog if the object has a custom policy.
Test Plan: {F6856365}
Maniphest Tasks: T13411
Differential Revision: https://secure.phabricator.com/D20805
Summary:
Ref T13411. This cleans up policy name rendering. We ultimately render into three contexts:
- Plain text contexts, like `bin/policy show`.
- Transaction contexts, where we're showing a policy change. In these cases, we link some policies (like project policies and custom policies) but the links go directly to the relevant object or a minimal explanation of the change. We don't link policies like "All Users".
- Capability contexts, where we're describing a capability, like "Can Push" or cases in Applicaitons. In these cases, we link all policies to the full policy explanation flow.
Test Plan:
- Used `bin/policy show` to examine the policy of an object with a project policy, no longer saw HTML.
- Viewed the transaction logs of Applications (ModularTransactions) and Tasks (not ModularTransactions) with policy edits, including project and custom policies.
- Clicked "Custom Policy" in both logs, got consistent dialogs.
- Viewed application detail pages, saw all capabities linked to explanatory capability dialogs. The value of having this dialog is that the user can get a full explanation of special rules even if the policy is something mundane like "All Users".
Maniphest Tasks: T13411
Differential Revision: https://secure.phabricator.com/D20804
Summary: Ref T13411. This pathway has an unused "icon" parameter with no callsites. Throw it away to ease refactoring.
Test Plan: Grepped for callsites, found none using this parameter.
Maniphest Tasks: T13411
Differential Revision: https://secure.phabricator.com/D20803
Summary:
Ref T13411. These three applications render an "Editable By: <policy>" field in their descriptions.
The pages that these appear on all have "Edit <thing>" actions which either tell you the policy or allow you to discover the policy, and this field is unusual (the vast majority of objects don't have it). I think it largely got copy/pasted or used as space-filler and doesn't offer much of value.
Remove it to simplify/standardize these pages and make changes to how this field works simpler to implement.
Test Plan: Viewed a Credential, Blog, and Space; no longer saw the "Editable By" field.
Maniphest Tasks: T13411
Differential Revision: https://secure.phabricator.com/D20802
Summary:
Ref T13411. Since circa D19829, transactions have rendered policy changes in a modern way, notably making "Custom Policy" clickable to show the policy rules.
Edit transactions in Applications still use a separate, older approach to render policies. This produces policy renderings which don't use modern quoting rules and don't link in a modern way.
Make Applications use the same rendering code that other transactions (like normal edit/view edits) use.
Test Plan: Edited policies in Applications, saw more useful transactions in the log. Clicked "Custom Policy" in the transaction log and got a useful explanation of the policy.
Maniphest Tasks: T13411
Differential Revision: https://secure.phabricator.com/D20801
Summary:
Ref T13410. See PHI1431. Currently, when you move a document in Phriction, the target shows a "This document was moved from ..." banner until it is edited.
This banner isn't particularly useful, and it's distracting and it isn't obvious how to dismiss it, and making a trivial edit to dismiss it is awkward.
This information is also already available in the transaction log.
Just remove this banner since it doesn't really serve any clear purpose.
Test Plan:
- Moved a page in Phriction, then loaded the destination page. Before change: header banner. After change: nothing.
- Viewed a normal (non-moved) page, saw normal behavior.
- Reviewed transactions, saw "Moved from ..." in the timeline.
Maniphest Tasks: T13410
Differential Revision: https://secure.phabricator.com/D20800
Summary:
Fixes T8808. Currently, all project use the default ("Briefcase") project icon when they appear in a policy dropdown.
Since project policies are separated out into a "Members of Projects" section of the dropdown anyway, there is no reason not to use the actual project icon, which is often more clear.
Test Plan: {F6849927}
Maniphest Tasks: T8808
Differential Revision: https://secure.phabricator.com/D20799
Summary:
Fixes T9136.
- Fix a bug where the name is rendered improperly.
- Put disabled rules at the bottom.
- Always show the rule monogram so you can distingiush between rules with the same name.
Test Plan: {F6849915}
Maniphest Tasks: T9136
Differential Revision: https://secure.phabricator.com/D20798
Summary:
Fixes T8952. These feed stories are not interesting and tend to be generated as collateral damage when a non-story update is made to an old task and someone has a "subscribe me" Herald rule.
Also clean up some of the Herald field/condition indexing behavior slightly.
Test Plan: Wrote a "Subscribe X" herald rule, made a trivial update to a task. Before: low-value feed story; after: no feed story.
Maniphest Tasks: T8952
Differential Revision: https://secure.phabricator.com/D20797
Summary: Fixes T13409. This is a companion to the existing "Mark with flag" rule.
Test Plan: Used a "remove flag" rule on an object with no flag (not removed), the right type of flag (removed), and a different type of flag (not removed).
Maniphest Tasks: T13409
Differential Revision: https://secure.phabricator.com/D20796
Summary:
Fixes T13408. Currently, when a package (or other object) appears in a field (rather than an action), it is not indexed.
Instead: index fields too, not just actions.
Test Plan:
- Wrote a rule like "[ Affected packages include ] ...".
- Updated the search index.
- Saw rule appear on "Affected By Herald Rules" on the package detail page.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13408
Differential Revision: https://secure.phabricator.com/D20795
Summary: Fixes T13412. If you search for "https://phabricator.example.com" with no trailing slash, we currently redirect you to "", which is fouled by a safety check in the redirection flow.
Test Plan:
- Searched for "https://local.phacility.com"; before: fatal in redirection; after: clean redirect.
- Searched for other self-URIs, got normal redirects.
Maniphest Tasks: T13412
Differential Revision: https://secure.phabricator.com/D20794
Summary: Fixes T13405. We currently offer non-global custom saved queries here, but this doesn't make sense as a global default setting.
Test Plan: Saved a global search query, edited global search settings, no longer saw the non-global query as an option.
Maniphest Tasks: T13405
Differential Revision: https://secure.phabricator.com/D20793
Summary: Ref T13405. Some pages don't have a contextual application.
Test Plan: Viewed 404 page, no more fatal.
Maniphest Tasks: T13405
Differential Revision: https://secure.phabricator.com/D20792
Summary:
Ref T13404. Except for one known issue in Multimeter, Phabricator appears to function properly in this mode. It is broadly desirable that we run in this mode; it's good on its own, and enabled by default in at least some recent MySQL.
Additionally, "ONLY_FULL_GROUP_BY" and "STRICT_ALL_TABLES" shared a setup key, so ignoring one would ignore both. Change the key so that existing ignores on "ONLY_FULL_GROUP_BY" do not mask "STRICT_ALL_TABLES" warnings.
Test Plan: Grepped for `ONLY_FULL_GROUP_BY`.
Maniphest Tasks: T13404
Differential Revision: https://secure.phabricator.com/D20791
Summary: Ref T13404. Enabling "STRICT_ALL_TABLES" is good, but if you don't want to bother it doesn't matter too much. All upstream development has been on "STRICT_ALL_TABLES" for a long time.
Test Plan: {F6847839}
Maniphest Tasks: T13404
Differential Revision: https://secure.phabricator.com/D20790
Summary: Fixes T13406. On the logout screen, test for no configured providers and warn users they may be getting into more trouble than they expect.
Test Plan:
- Logged out of a normal install and a fresh (unconfigured) install.
{F6847659}
Maniphest Tasks: T13406
Differential Revision: https://secure.phabricator.com/D20789
Summary:
See D20779, https://discourse.phabricator-community.org/t/3089. `bin/config set` complains about
missing config file as if it's un-writable.
Test Plan: run `bin/config set` with missing, writable, unwritable conf.json and parent dir.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D20788
Summary: Fixes T13405. The default behavior of the global search bar isn't currently configurable, but can be made configurable fairly easily.
Test Plan: Changed setting as an administrator, saw setting reflected as a user with no previous preference. As a user with an existing preference, saw preference retained.
Maniphest Tasks: T13405
Differential Revision: https://secure.phabricator.com/D20787
Summary:
Fixes T13392. If you have 17 load balancers in sequence, Phabricator will receive requests with at least 17 "X-Forwarded-For" components in the header.
We want to select the 17th-from-last element, since prior elements are not trustworthy.
This currently isn't very easy/obvious, and you have to add a kind of sketchy piece of custom code to `preamble.php` to do any "X-Forwarded-For" parsing. Make handling this correctly easier.
Test Plan:
- Ran unit tests.
- Configured my local `preamble.php` to call `preamble_trust_x_forwarded_for_header(4)`, then made `/debug/` dump the header and the final value of `REMOTE_ADDR`.
```
$ curl http://local.phacility.com/debug/
<pre>
HTTP_X_FORWARDED_FOR =
FINAL REMOTE_ADDR = 127.0.0.1
</pre>
```
```
$ curl -H 'X-Forwarded-For: 1.1.1.1, 2.2.2.2, 3.3.3.3, 4.4.4.4, 5.5.5.5, 6.6.6.6' http://local.phacility.com/debug/
<pre>
HTTP_X_FORWARDED_FOR = 1.1.1.1, 2.2.2.2, 3.3.3.3, 4.4.4.4, 5.5.5.5, 6.6.6.6
FINAL REMOTE_ADDR = 3.3.3.3
</pre>
```
```
$ curl -H 'X-Forwarded-For: 5.5.5.5, 6.6.6.6' http://local.phacility.com/debug/
<pre>
HTTP_X_FORWARDED_FOR = 5.5.5.5, 6.6.6.6
FINAL REMOTE_ADDR = 5.5.5.5
</pre>
```
Maniphest Tasks: T13392
Differential Revision: https://secure.phabricator.com/D20785
Summary:
Ref T13336. Currently, "bin/storage destroy" destroys every master. This is wonderfully destructive, but if replication fails it's useful to be able to destroy only a replica.
Operate on a single host, and require "--host" to target the operation in cluster mode, so `bin/storage destroy --host dbreplica001` is a useful operation.
Test Plan: Ran `bin/storage destroy` with various flags locally. Will destroy `secure002` and refresh replication.
Maniphest Tasks: T13336
Differential Revision: https://secure.phabricator.com/D20784
Summary: Ref T13366. The "authorities" mechanism was replaced, but I missed this callsite. Update it to use the request cache mechanism.
Test Plan: As a user without permission to view some initiatives, viewed a list of initiatives.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20783
Summary: Ref T13404. This query is invalid under "sql_mode=ONLY_FULL_GROUP_BY". Rewrite it to avoid interacting with `actorIdentity` at all; this is a little more robust in the presence of weird data and not really more complicated.
Test Plan:
- Enabled "ONLY_FULL_GROUP_BY".
- Hit system actions (e.g., login).
- Before: error.
- After: clean login.
- Tried to login with a bad password many times in a row, got properly limited by the system action rate limiter.
Maniphest Tasks: T13404
Differential Revision: https://secure.phabricator.com/D20782
Summary:
Depends on D20780. Ref T13403. During initial setup, it's routine to run "bin/config" with a bad database config. We start the stack in "config optional" mode to anticipate this.
However, even in this mode, we may emit warnings if the connection fails in certain ways. These warnings aren't useful; suppress them with "@".
(Possibly this message should move from "phlog()" to "--trace" at some point, but it has a certain amount of context/history around it.)
Test Plan:
- Configured MySQL to fail with a retryable error, e.g. good host but bad port.
- Ran `bin/config set ...`.
- Before: saw retry warnings on stderr.
- After: no retry warnings on stderr.
- (Turned off suppression code artificially and verified warnings still appear under normal startup.)
Maniphest Tasks: T13403
Differential Revision: https://secure.phabricator.com/D20781
Summary: Depends on D20779. Ref T13403. Bad parameters may cause this call to fail without setting an error code; if it does, catch the issue and go down the normal connection error pathway.
Test Plan:
- With "mysql.port" set to "quack", ran `bin/storage probe`.
- Before: wild mess of warnings as the code continued below and failed when trying to interact with the connection.
- After: clean connection failure with a useful error message.
Maniphest Tasks: T13403
Differential Revision: https://secure.phabricator.com/D20780
Summary: Ref T13403. We currently emit a useful error message, but it's not tailored and has a stack trace. Since this is a relatively routine error and on the first-time-setup path, tailor it so it's a bit nicer.
Test Plan:
- Ran `bin/config set ...` with an unwritable "local.json".
- Ran `bin/config set ...` normally.
Maniphest Tasks: T13403
Differential Revision: https://secure.phabricator.com/D20779
Summary:
Ref T13286. The current (very safe / conservative) rules for retrying git reads generalize to git writes, so we can use the same ruleset in both cases.
Normally, writes converge rapidly to only having good nodes at the head of the list, so this has less impact than the similar change to reads, but it generally improves consistency and allows us to assert that writes which can be served will be served.
Test Plan:
- In a cluster with an up node and a down node, pushed changes.
- Saw a push to the down node fail, retry, and succeed.
- Did some pulls, saw appropriate retries and success.
- Note that once one write goes through, the node which received the write always ends up at the head of the writable list, so nodes need to be explicitly thawed to reproduce the failure/retry behavior.
Maniphest Tasks: T13286
Differential Revision: https://secure.phabricator.com/D20778
Summary: Ref T13286. When retrying a read request, keep retrying as long as we have canididate services. Since we consume a service with each attempt, there's no real reason to abort early, and trying every service allows reads to always succeed even if (for example) 8 nodes of a 16-node cluster are dead because of a severed network link between datacenters.
Test Plan: Ran `git pull` in a clustered repository with an up node and a down node; saw retry count dynamically adjust to available node count.
Maniphest Tasks: T13286
Differential Revision: https://secure.phabricator.com/D20777
Summary:
Depends on D20775. Ref T13286. When a Git read request fails against a cluster and there are other nodes we could safely try, try more nodes.
We DO NOT retry the request if:
- the client read anything;
- the client wrote anything;
- or we've already retried several times.
Although //some// requests where bytes went over the wire in either direction may be safe to retry, they're rare in practice under Git, and we'd need to puzzle out what state we can safely emit.
Since most types of failure result in an outright connection failure and this catches all of them, it's likely to almost always be sufficient in practice.
Test Plan:
- Started a cluster with one up node and one down node, pulled it.
- Half the time, hit the up node and got a clean pull.
- Half the time, hit the down node and got a connection failure followed by a retry and a clean pull.
- Forced `$err = 1` so even successful attempts would retry.
- On hitting the up node, got a "failure" and a decline to retry (bytes already written).
- On hitting the down node, got a failure and a real retry.
- (Note that, in both cases, "git pull" exits "0" after the valid wire transaction takes place, even though the remote exited non-zero. If the server gave Git everything it asked for, it doesn't seem to care if the server then exited with an error code.)
Maniphest Tasks: T13286
Differential Revision: https://secure.phabricator.com/D20776
Summary:
Ref T13286. To support request retries, allow the service lookup method to return an ordered list of structured service references.
Existing callsites continue to immediately discard all but the first reference and pull a URI out of it.
Test Plan: Ran `git pull` in a clustered repository with an "up" node and a "down" node, saw 50% serivce failures and 50% clean pulls.
Maniphest Tasks: T13286
Differential Revision: https://secure.phabricator.com/D20775
Ref T13401. The checkout UI didn't get fully updated to the new View objects,
and account handles are still manually building a URI that goes to the wrong
place.
Summary:
Ref T13393. See some previous discussion in T13366.
Caching is hard and all approaches here have downsides, but the request cache likely has fewer practical downsides for this kind of policy check than other approaches. In particular, the grant approach (at least, as previously used in Phortune) has a major downside that "Query" classes can no longer fully enforce policies.
Since Phortune no longer depends on grants and they've now been removed from instances, drop the mechanism completely.
Test Plan: Grepped for callsites, found none.
Maniphest Tasks: T13393
Differential Revision: https://secure.phabricator.com/D20754
Summary:
Ref T13393. While doing a shard migration in the Phacility cluster, we'd like to stop writes to the migrating repository. It's safe to continue serving reads.
Add a simple maintenance mode for making repositories completely read-only during maintenance.
Test Plan: Put a repository into read-only mode, tried to write via HTTP + SSH. Viewed web UI. Took it back out of maintenance mode.
Maniphest Tasks: T13393
Differential Revision: https://secure.phabricator.com/D20748
Summary:
Fixes T13336.
- Prevent `--no-indexes` from being combined with `--for-replica`, since combining these options can only lead to heartbreak.
- In `--for-replica` mode, dump caches too. See discussion in T13336. It is probably "safe" to not dump these today, but fragile and not correct.
- Mark the "MarkupCache" table as having "Cache" persistence, not "Data" persistence (no need to back it up, since it can be fully regenerated from other datasources).
Test Plan: Ran `bin/storage dump` with various combinations of flags.
Maniphest Tasks: T13336
Differential Revision: https://secure.phabricator.com/D20743
Summary:
Fixes T13389. Currently, we try to "newSubtypeMap()" unconditionally, even if the underlying object does not support subtypes.
- Only try to build a subtype map if subtype transactions are actually being applied.
- When subtype transactions are applied to a non-subtypable object, fail more explicitly.
Test Plan: Clicked "Make Editable" in a fresh Calendar transaction form, got an editable form instead of a fatal from "newSubtypeMap()". (Calendar events are not currently subtypable.)
Maniphest Tasks: T13389
Differential Revision: https://secure.phabricator.com/D20741
Summary:
Fixes T13390. We have some old code which doesn't dynamically select between "utf8mb4" and "utf8". This can lead to dumping utf8mb4 data over a utf8 connection in `bin/storage dump`, which possibly corrupts some emoji/whales.
Instead, prefer "utf8mb4" if it's available.
Test Plan: Ran `bin/storage dump` and `bin/storage shell`, saw sub-commands select utf8mb4 as the client charset.
Maniphest Tasks: T13390
Differential Revision: https://secure.phabricator.com/D20742
Summary:
Depends on D20739. Ref T13366. Slightly modularize/update components of order views, and make orders viewable from either an account context (existing view) or an external context (new view).
The new view is generally simpler so this mostly just reorganizes existing code.
Test Plan: Viewed orders as an account owner and an external user.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20740
Summary: Depends on D20738. Ref T13366. Fixes T8389. Now that the infrastructure is in place, actually send email to external addresses.
Test Plan: Used `bin/phortune invoice` to generate invoices and saw associated external accounts receive mail in `bin/mail list-outbound`.
Maniphest Tasks: T13366, T8389
Differential Revision: https://secure.phabricator.com/D20739
Summary: Depends on D20737. Ref T13367. Allow external addresses to have their access key rotated. Account managers can disable them, and anyone with the link can permanently unsubscribe them.
Test Plan: Enabled/disabled addresses; permanently unsubscribed addresses.
Maniphest Tasks: T13367
Differential Revision: https://secure.phabricator.com/D20738
Summary: Ref T13366. This gives each account email address an "external portal" section so they can access invoices and receipts without an account.
Test Plan: Viewed portal as user with authority and in an incognito window.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20737
Summary:
Depends on D20734. Ref T13366. This makes the cart/order flow work under the new policy scheme with no "grantAuthority()" calls.
It prepares for a "Void Invoice" action, although the action doesn't actually do anything yet.
Test Plan: With and without merchant authority, viewed and paid invoices and went through the other invoice interaction workflows.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20735
Summary: Refresh the 404 text since it hasn't been updated in a while, and swap the "Save Query" button back to grey since I never got used to blue.
Test Plan: Hit 404 page, saved a query.
Differential Revision: https://secure.phabricator.com/D20734
Summary:
Depends on D20732. Ref T13366. This generally makes the "Merchant" UI look and work like the "Payment Account" UI.
This is mostly simpler since the permissions have largely been sorted out already and there's less going on here and less weirdness around view/edit policies.
Test Plan: Browsed all Merchant functions as a merchant member and non-member.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20733
Summary:
Ref T13366. Depends on D20721. Continue applying UI and policy updates to the last two Phortune objects.
Charges aren't mutable and Carts are already transactional, so this is less involved than prior changes.
Test Plan: Viewed various charge/order interfaces as merchants and account members.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20732
Summary:
Depends on D20720. Ref T13366.
- Use modern policies and policy interfaces.
- Use new merchant authority cache.
- Add (some) transactions.
- Move MFA from pre-upgrade-gate to post-one-shot-check.
- Simplify the autopay workflow.
- Use the "reloading arrows" icon for subscriptions more consistently.
Test Plan: As a merchant-authority and account-authority, viewed, edited, and changed autopay for subscriptions.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20721
Summary:
Depends on D20719. Currently, if a page throws an exception (like a policy exception) and rendering that exception into a response (like a policy dialog) throws another exception (for example, while constructing breadcrumbs), we only show the orginal exception.
This is usually the more useful exception, but sometimes we actually care about the other exception.
Instead of guessing which one is more likely to be useful, throw them both as an "AggregateException" and let the high-level handler flatten it for display.
Test Plan: {F6749312}
Differential Revision: https://secure.phabricator.com/D20720
Summary:
Depends on D20718. Ref T13366. Ref T13367.
- Phortune payment methods currently do not use transactions; update them.
- Give them a proper view page with a transaction log.
- Add an "Add Payment Method" button which always works.
- Show which subscriptions a payment method is associated with.
- Get rid of the "Active" status indicator since we now treat "disabled" as "removed", to align with user expectation/intent.
- Swap out of some of the super weird div-form-button UI into the new "big, clickable" UI for choice dialogs among a small number of options on a single dimension.
Test Plan:
- As a mechant-authority and account-authority, created payment methods from carts, subscriptions, and accounts. Edited and viewed payment methods.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13367, T13366
Differential Revision: https://secure.phabricator.com/D20719
Summary:
Depends on D20717. Ref T13366. Make PhortunePaymentMethod use an extended policy interface for consistency with modern approaches. Since Accounts have hard-coded policy behavior (and can't have object policies like "Subscribers") this should have no actual impact on program behavior.
This leaves one weird piece in the policy dialog UIs, see T13381.
Test Plan: Viewed and edited payment methods as a merchant and account member. Merchants can only view, not edit.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20718
Summary: Depends on D20716. Ref T13366. This implements the new policy behavior cleanly in all top-level Phortune payment account interfaces.
Test Plan: As a merchant with an account relationship (not an account member) and an account member, browsed all account interfaces and attempted to perform edits. As a merchant, saw a reduced-strength view.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20717
Summary:
Depends on D20715. Ref T13366. See that task for discussion.
Replace the unreliable "grantAuthority()"-based check with an actual "can the viewer edit any merchant this account has a relationship with?" check.
This makes these objects easier to use from a policy perspective and makes it so that the `Query` alone can fully enforce permissions properly with no setup, so general infrastructure (like handles and transactions) works properly with Phortune objects.
Test Plan: Viewed merchants and accounts as users with no authority, direct authority on the account, and indirect authority via a merchant relationship.
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20716
Summary:
Depends on D20713. Ref T13366. When a payment account establishes a relationship with a merchant by creating a cart or subscription, create an edge to give the merchant access to view the payment account.
Also, migrate all existing subscriptions and carts to write these edges.
This aims at straightening out Phortune permissions, which are currently a bit wonky on a couple of dimensions. See T13366 for detailed discussion.
Test Plan:
- Created and edited carts/subscriptions, saw edges write.
- Ran migrations, saw edges write.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20715
Summary: Depends on D20697. Ref T8389. Add support for adding "billing@enterprise.com" and similar to Phortune accounts.
Test Plan: Added and edited email addresses for a payment account.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T8389
Differential Revision: https://secure.phabricator.com/D20713
Summary:
Ref T13366. Some of the information architecture is a little muddy here, notably an item called "Billing / History" which contains payment methods.
Split things up a bit to prepare for adding support for "Email Addresses".
Test Plan: {F6676988}
Maniphest Tasks: T13366
Differential Revision: https://secure.phabricator.com/D20697
Summary: See PHI1396. Ideally this would be some kind of general-purpose tie-in to object relationships, but see D18456 for precedent.
Test Plan: Used `maniphest.edit` to edit associated commits for a task.
Differential Revision: https://secure.phabricator.com/D20731
Summary:
Fixes T13386. See PHI1391. These constraints largely exist already, but are not yet exposed to Conduit.
Also, tweak some keys to support the underlying query.
Test Plan: Ran `differential.revision.search` queries with the new constraints.
Maniphest Tasks: T13386
Differential Revision: https://secure.phabricator.com/D20730
Summary: Ref T13382. Currently, the "Make Administrator" action in the web UI does state-based MFA. Convert it to one-shot MFA.
Test Plan: Empowered and unempowered a user from the web UI, got one-shot MFA'd. Empowered a user from the CLI, no MFA issues.
Maniphest Tasks: T13382
Differential Revision: https://secure.phabricator.com/D20729
Summary:
Ref T13386. If you issue `differential.query` with a large offset (like 3000), it can overheat regardless of policy filtering and fail with a nonsensical error message.
This is because the overheating limit is based only on the query limit, not on the offset.
For example, querying for "limit = 100" will never examine more than 1,100 rows, so a query with "limit = 100, offset = 3000" will always fail (provided there are at least that many revisions).
Not all numbers work like you might expect them to becuase there's also a 1024-row fetch window, but basically small limits plus big offsets always fail.
Test Plan: Artificially reduced the internal window size from 1024 to 5, then ran `differential.query` with `offset=50` and `limit=3`. Before: overheated with weird error message. After: clean result.
Maniphest Tasks: T13386
Differential Revision: https://secure.phabricator.com/D20728
Summary:
Ref T13385. Currently, if you run `arc diff` in a CWD with more than 255 characters, the workflow fatals against the length of the `sourcePath` database column.
In the long term, removing this property is likely desirable.
For now, truncate long values and continue. This only meaningfully impacts relatively obscure interactive SVN workflows negatively, and even there, "some arc commands are glitchy in very long working directories in SVN" is still better than "arc diff fatals".
Test Plan:
- Modified `arc` to submit very long source paths.
- Ran `arc diff`.
- Before: Fatal when inserting >255 characters into `sourcePath`.
- After: Path truncated at 255 bytes.
Maniphest Tasks: T13385
Differential Revision: https://secure.phabricator.com/D20727
Summary:
Fixes T13384. Currently, the subtype "disabled" configuration is not respected when selecting fields for `ROLE_EDIT`.
The only meaningful caller for `ROLE_EDIT` is transaction validation, but transaction validation should respect fields being disabled by subtype configuration.
Test Plan:
- Added a "required" Maniphest custom field "F", then "disabled" it in a subtype "S".
- Created a task of subtype "S".
- Before: Form submission fails with error "F is required", even though the field is not actually visible on the form and can not be set.
- After: Form submits cleanly and creates the task.
Maniphest Tasks: T13384
Differential Revision: https://secure.phabricator.com/D20726
Summary:
Fixes T13382. Depends on D20724. These ancient scripts are no longer necessary since we've had a smooth web-based onboarding process for a long time.
I retained `bin/user empower` and `bin/user enable` for recovering from situations where you accidentally delete or disable all administrators. This is normally difficult, but some users are industrious.
Test Plan: Grepped for `accountadmin` and `add_user.php`, found no more hits.
Maniphest Tasks: T13382
Differential Revision: https://secure.phabricator.com/D20725
Summary:
Ref T13382.
- Remove "bin/people profileimage" which previously generated profile image caches but now feels obsolete.
- Replace it with "bin/user", with "enable" and "empower" flows. This command is now focused on regaining access to an install after you lock your keys inside.
- Document the various ways to unlock objects and accounts from the CLI.
Test Plan:
- Ran `bin/user enable` and `bin/user empower` with various flags.
- Grepped for `people profileimage` and found no references.
- Grepped for `bin/people` and found no references.
- Read documentation.
Maniphest Tasks: T13382
Differential Revision: https://secure.phabricator.com/D20724
Summary: Fixes T13383. Provide a basic "drydock.resource.search". Also allow "drydock.lease.search" to be queried by resource PHID.
Test Plan: Called "drydock.resource.search" and "drydock.lease.search" with various constraints.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13383
Differential Revision: https://secure.phabricator.com/D20723
Summary: See PHI1392. This flag is `--all`, not `--all-caches`.
Test Plan: Ran `bin/cache purge --all`.
Differential Revision: https://secure.phabricator.com/D20722
Summary:
Fixes T13378. If we join Ferret tables and page, we can end up with an ambiguous `id` column here.
Explicitly refer to "project.x" in all cases that we're interacting with the project table.
Test Plan:
- Changed page size to 3.
- Issued a Projects query for "~e", matching more than 3 results.
- Clicked "Next Page".
- Before: ambiguous id column fatal.
- After: next page.
Maniphest Tasks: T13378
Differential Revision: https://secure.phabricator.com/D20714
Summary:
Ref T13369. See that task for discussion.
When the discovery daemon finds more than 64 commits to import, demote the worker queue priority of the resulting tasks.
Test Plan:
- Pushed one commit, ran `bin/repository discover --verbose --trace ...`, saw commit import with "at normal priority" message and priority 2500 ("PRIORITY_COMMIT").
- Pushed 3 commits, set threshold to 3, ran `bin/repository discover ...`, saw commist import with "at lower priority" message and priority 4000 ("PRIORITY_IMPORT").
Maniphest Tasks: T13369
Differential Revision: https://secure.phabricator.com/D20712
Summary:
Ref T13373. When you "bin/config set x ..." a value, the success message ("Set x ...") is somewhat ambiguous and can be interpreted as "First, you need to set x..." rather than "Success, wrote x...".
Make the messaging more explicit. Also make this string more translatable.
Test Plan: Ran `bin/config set ...` with various combinations of flags, saw more clear messaging.
Maniphest Tasks: T13373
Differential Revision: https://secure.phabricator.com/D20711
Summary: Fixes T13374. The "Temporary Failures" row is missing a cell definiton from the addition of "Average Queue Time".
Test Plan: Viewed "/daemon/" with some temporary failures and and odd number of rows above the "Temporary Failures" row. Saw cell properly zebra-striped.
Maniphest Tasks: T13374
Differential Revision: https://secure.phabricator.com/D20710
Summary:
Ref T13349. This is almost the same change as D20678, but for project profiles instead of user profiles.
The general reproduction case is "view a project where you can't see more than 50 of the 500 most recent feed stories".
Test Plan:
- Forced all queries to overheat.
- Viewed a project profile page.
- Before: overheating fatal near top level.
- After: damage contained to feed panel.
Maniphest Tasks: T13349
Differential Revision: https://secure.phabricator.com/D20704
Summary: See downstream <https://phabricator.wikimedia.org/T229757>. The "autofocus" attribute mostly just works, so add it to this input.
Test Plan: As a user with TOTP enabled, established a new session. Saw browser automatically focus the "App Code" input on the TOTP prompt screen.
Differential Revision: https://secure.phabricator.com/D20703
Summary:
Fixes T13370. We currently show an "Award Badge" button conditionally, based on whether the viewer can award any badges or not.
The query to test this may overheat and this pattern isn't consistent with other UI anyway. Stop doing this test.
Test Plan:
- Created 12 badges.
- As a user who could not edit any of the badges, viewed the "Badges" section of a user profile.
Maniphest Tasks: T13370
Differential Revision: https://secure.phabricator.com/D20702
Summary: Fixes T13368. Some workflows (like "Move tasks to...") execute board layout without objects to update. In these cases, we can hit a warning because `objectPHIDs` is not initialized to `array()`.
Test Plan: Went through the "Move tasks to..." workflow on a workboard, no longer saw a warning when trying to iterate over an empty `objectPHIDs` list.
Maniphest Tasks: T13368
Differential Revision: https://secure.phabricator.com/D20701
Summary:
Ref T13368. Currently, both visible and hidden columns are shown in the "Move tasks to..." dropdown on workflows from workboards.
When the dropdown contains hidden columns, move them to a separate section to make it clear that they're not likely targets.
Test Plan:
- Used "Move tasks to project..." targeting a board with no hidden columns. Saw a single ungrouped dropdown.
- Used "Move tasks to project..." targeting a board with hidden columns. Saw a dropdown grouped into "Visible" and "Hidden" columns.
Maniphest Tasks: T13368
Differential Revision: https://secure.phabricator.com/D20700
Summary:
Ref T13368. Proxy columns should not be selectable from this workflow. If you want to move tasks to milestone/subproject X, do "Move tasks to project..." and pick X as the project.
(This could be made to work some day.)
Test Plan: Went through a "Move tasks to project..." workflow targeting a project with subprojects. No longer saw subproject columns presented as dropdown options.
Maniphest Tasks: T13368
Differential Revision: https://secure.phabricator.com/D20699
Summary: Ref T13368. The column options presented to the user are currently incorrect because the wrong set of columns are drawn from.
Test Plan: On a workboard, used "Move tasks to project..." to target another board, saw that board's columns.
Maniphest Tasks: T13368
Differential Revision: https://secure.phabricator.com/D20698
Summary:
Fixes T13341. Currently, cart emails (invoices/receipts) are sent to members of the associated merchant account. This was just a simple way to keep an eye on things when this was first written.
The system works fine, and recent changes (almost certainly D20525) stopped these emails from working (presumably because of the slightly weird merchant permissions model).
This could be sorted out in more detail, but it looks like the path forward is to introduce a side channel for email anyway (via T8389), and that's a better way to implement this behavior since it means the normal recipients won't see a bunch of random staff/merchant email addresses on their receipts.
Test Plan: Grepped for `merchant` in this editor.
Maniphest Tasks: T13341
Differential Revision: https://secure.phabricator.com/D20696
Summary:
Ref T13339. If a search pattern matches more than once on a line, we currently render the line incorreclty, duplicating some of the text.
`substr()` is being called as though the third parameter was `end_offset`, but it's actually `length`. Correct the parameter.
Test Plan:
Before:
{F6676625}
After:
{F6676623}
Maniphest Tasks: T13339
Differential Revision: https://secure.phabricator.com/D20695
Summary:
Fixes T8830. Fixes T13364.
- The inability to destroy objects from the web UI is intentional. Make this clear in the messaging, which is somewhat out of date and partly reflects an earlier era when things could be destroyed.
- `bin/remove destroy` can't rewind time. Document expectations around the "put the cat back in the bag" use case.
Test Plan: Read documentation, clicked through both workflows.
Maniphest Tasks: T13364, T8830
Differential Revision: https://secure.phabricator.com/D20694
Summary: Ref T13358. This is very minimal, but technically works. The eventual goal is to generate PDF invoices to make my life easier when I have to interact with Enterprise Vendor Procurement.
Test Plan: {F6672439}
Maniphest Tasks: T13358
Differential Revision: https://secure.phabricator.com/D20692
Summary:
Fixes T13356. This option is supported and works fine, it just isn't documented.
Add documentation and fix the config option to actually link to it to make life a little easier.
Test Plan: Read documentation.
Maniphest Tasks: T13356
Differential Revision: https://secure.phabricator.com/D20691
Summary: Fixes T13355. This didn't appear to be a ton of extra work, we just didn't get it for free in the original implementation in D14635.
Test Plan:
- Saw "date" custom fields appear in Conduit API documentation for "maniphest.edit".
- Set custom "date" field to null and non-null values via the API.
{F6666582}
Maniphest Tasks: T13355
Differential Revision: https://secure.phabricator.com/D20690
Summary: Ref T4900. We may execute a bad query here if the task has no projects at all.
Test Plan: Edited a task with no new or old projects. Instead of an exception, things worked.
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20689
Summary:
Fixes T13353. If you:
- Visit a blog post and save the URI.
- Move the blog post to a different blog.
- Revisit the old URI.
...we currently 404. We know what you're trying to do and should just redirect you to the new URI instead. We already do this if you visit a URI with a noncanonical slug.
Test Plan:
- Created post A.
- Copied the live URI.
- Moved it to a different blog.
- Visited the saved URI from the earlier step.
- Before: 404.
- After: Redirect to the canonical URI.
Maniphest Tasks: T13353
Differential Revision: https://secure.phabricator.com/D20688
Summary: Depends on D20686. Fixes T13350. Now that "slowvote.poll.search" exists, deprecate this old method.
Test Plan: Reviewed method description in Condiut API console in the web UI.
Maniphest Tasks: T13350
Differential Revision: https://secure.phabricator.com/D20687
Summary:
Depends on D20685. Ref T13350. Currently:
- When a SearchEngine parameter is marked as hidden from Conduit, we may still render a table of possible values. Instead, only render the table if the parameter is actually usable.
- The table header is hard-coded to say `'statuses'`, which is just a silly mistake. (Most commonly, this table does have `statuses` constants.)
Test Plan: Viewed the Conduit API documentation for the new "slowvote.poll.search" API method, saw more sensible display behavior.
Maniphest Tasks: T13350
Differential Revision: https://secure.phabricator.com/D20686
Summary: Ref T13350. Add a modern "*.search" API method for Slowvote so "slowvote.info" can be deprecated with a reasonable replacement.
Test Plan: Used Conduit test console to call method, saw reasonable results.
Maniphest Tasks: T13350
Differential Revision: https://secure.phabricator.com/D20685
Summary:
Depends on D20680. Ref T4900. The "BoardLayoutEngine" operates on PHIDs without knowledge of the underlying objects, but this means it has to be sensitive to PHID input order when falling back to a default layout order.
We use "default layout order" on workboards which are sorted by "Natual" order but which have one or more cards which no user has ever reordered. For example, if you add 10 tasks to a project, then create a board, there's no existing order for those tasks in the "Backlog" column. The layout engine uses the input order to place them in the column, with the expectation that input order is ID/creation order, so new cards will end up on top.
I think this code never really made an explicit effort to guarantee that the LayoutEngine received objects in ID order, and it just sort of happened to by coincidence and good fortune. Some recent change has disrupted this, so the edit operation can end up with the PHIDs arranged in arbitrary order.
Explicitly put them in ID order so we always get an implicit default layout order to fall back to. Also, update to `msortv()`.
Test Plan:
- Tagged several tasks with project X, a project without a board yet.
- Created the project X workboard.
- (Did not drag any tasks around on the project X board!)
- Viewed the board in "Natural" order.
This creates a view of the board where tasks are ordered by implicit/virtual/input order. The expectation, and "view" behavior of this board, is that this order is "newest on top".
- Edited one of the cards on the board, changing the title (don't reorder it!)
- Before: page state synchronized with cards in arbitrary/random/different order.
- After: page state synchronized with cards in the same order as before ("newest on top").
Reviewers: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20681
Summary:
Ref T4900. When a card is edited, we currently emit an update notification for all the projects the task is tagged with. This isn't quite the right set:
- We want to emit notifications for projects the task //was previously// tagged with, so it can be removed from boards it should no longer be part of.
- We want to emit notifications for ancestors of projects the task is or was tagged with, so parent project boards can be updated.
- However, we don't need to emit notifications for projects that don't actually have workboards.
Adjust the notification set to align better to these rules.
Test Plan:
- Removal of Parent Project: Edited a task on board "A > B", removing the "B" project tag. Saw board A update in another window.
- Normal Update: Edited a task title on board X, saw board X update in another window.
- Used `bin/aphlict debug` to inspect the notification set, saw generally sensible-seeming data going over the wire.
Reviewers: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20680
Summary: Ref T13350. This ancient API method is missing modern policy checks.
Test Plan:
- Set visibility of vote X to "Only: epriestley".
- Called "slowvote.info" as another user.
- Before: retrieved poll title and author.
- After: policy error.
- Called "slowvote.info" on a visible poll, got information before and after.
Maniphest Tasks: T13350
Differential Revision: https://secure.phabricator.com/D20684
Summary:
Fixes T13348. Currently, the Harbormaster UI shows "Restart All Builds", but it really means "Restart Restartable Builds", which is often fewer than "All" builds (because of autobuilds, permissions, and/or configuration).
Remove the misleading term "All" and make the workflow preview exactly which builds will and will not be affected, and why.
Test Plan:
{F6636313}
{F6636314}
{F6636315}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13348
Differential Revision: https://secure.phabricator.com/D20679
Summary:
Depends on D20673. Ref T13343. Since we're now putting log IDs in email, make the UI a little better for working with log IDs.
Some day, this page might have actions like "report this as suspicious" or whatever, but I'm not planning to do any of that for now.
Test Plan: {F6608631}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20674
Summary:
Depends on D20672. Ref T13343. When a user requests an account access link via email:
- log it in the activity log; and
- reference the log in the mail.
This makes it easier to ban users misusing the feature, provided they're coming from a single remote address, and takes a few steps down the pathway toward a button in the mail that users can click to report the action, suspend account recovery for their account, etc.
Test Plan:
- Requested an email recovery link.
- Saw request appear in the user activity log.
- Saw a reference to the log entry in the mail footer.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20673
Summary: Depends on D20671. Ref T13343. Now that log types are modular, provide a datasource/tokenizer for selecting them since we already have a lot (even after I purged a few in D20670) and I'm planning to add at least one more ("Request password reset").
Test Plan: {F6608534}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20672
Summary:
Depends on D20670. Ref T13343. The user activity message log types are currently hard-coded, so only upstream code can really use the log construct.
Under the theory that we're going to keep this log around going forward (just focus it a little bit), modularize things so the log is extensible.
Test Plan:
Grepped for `UserLog::`, viewed activity logs in People and Settings.
(If I missed something here -- say, misspelled a constant -- the effect should just be that older logs don't get a human-readable label, so stakes are very low.)
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20671
Summary: Fixes T13349. If the user profile page feed query overheats, it currently takes the whole page with it. Contain the blast to a smaller radius.
Test Plan: {F6633322}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13349
Differential Revision: https://secure.phabricator.com/D20678
Summary: Humble user cannot silence/mute project if he/she has no CAN_EDIT permissions in it. You can actually leave it but if project is locked - then you're scr*wed.
Test Plan:
1. On a testing phabricator instance created a dummy project
2. Changed that project permissions CAN_EDIT to be by admin only
3. Added poor soul with no CAN_EDIT permissions
4. Logged it in with poor soul
5. Tried to silence the project
6. The Project is successfully silenced
7. User is happy :)
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin, Pawka
Differential Revision: https://secure.phabricator.com/D20675
Summary:
Depends on D20669. Ref T13343. Currently, the user activity log includes a number of explicit administrative actions which some administrator (not a normal user or a suspicious remote address) takes. In most/all cases, these changes are present in the user profile transaction log too, and that's //generally// a better place for them (for example, it doesn't get GC'd after a couple months).
Some of these are so old that they have no writers (like DELETE and EDIT). I'd generally like to modernize this a bit so we can reference it in email (see T13343) and I'd like to modularize the event types as part of that -- partly, cleaning this up makes that modularization easier.
There's maybe some hand-wavey argument that administrative vs non-administrative events could be related and might be useful to see in a single log, but I can't recall a time when that was actually true, and we could always build that kind of view later by just merging the two log sources, or by restoring double-writes for some subset of events. In practice, I've used this log mostly to look for obvious red flags when users report authentication difficulty (e.g., many unauthorized login attempts), and removing administrative actions from the log is only helpful in that use case.
Test Plan: Grepped for all the affected constants, no more hits in the codebase.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20670
Summary: Depends on D20668. Ref T13343. Just an easy cleanup/simplification while I'm here.
Test Plan: `grep` for `getActionConstant()`
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20669
Summary:
Depends on D20667. Ref T13343. Password auth currently uses an older rate limiting mechanism, upgrade it to the modern "SystemAction" mechanism.
This mostly just improves consistency, although there are some tangential/theoretical benefits:
- it's not obvious that making the user log GC very quickly could disable rate limiting;
- if we let you configure action limits in the future, which we might, this would become configurable for free.
Test Plan:
- With CAPTCHAs off, made a bunch of invalid login attempts. Got rate limited.
- With CAPTCHAs on, made a bunch of invalid login attempts. Got downgraded to CAPTCHAs after a few.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20668
Summary:
Depends on D20666. Ref T13343. In D20666, I limited the rate at which a given user account can be sent account recovery links.
Here, add a companion limit to the rate at which a given remote address may request recovery of any account. This limit is a little more forgiving since reasonable users may plausibly try multiple variations of several email addresses, make typos, etc. The goal is just to hinder attackers from fishing for every address under the sun on installs with no CAPTCHA configured and no broad-spectrum VPN-style access controls.
Test Plan: {F6607846}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20667
Summary:
Depends on D20665. Ref T13343. We support CAPTCHAs on the "Forgot password?" flow, but not everyone configures them (or necessarily should, since ReCAPTCHA is a huge external dependency run by Google that requires you allow Google to execute JS on your domain) and the rate at which any reasonable user needs to take this action is very low.
Put a limit on the rate at which account recovery links may be generated for a particular account, so the worst case is a trickle of annoyance rather than a flood of nonsense.
Test Plan: {F6607794}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20666
Summary:
Depends on D20664. Ref T13343. There's a reasonable value for the default "Email Login" auth message (generic "you reset your password" text) that installs may reasonably want to replace. Add support for a default value.
Also, since it isn't completely obvious where this message shows up, add support for an extended description and explain what's going on in more detail.
Test Plan:
- Viewed message detail page, saw more detailed information.
- Sent mail (got default), overrode message and sent mail (got custom message), deleted message (got default again).
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20665
Summary:
Depends on D20663. Ref T13343. Currently, if an Auth message hasn't been customized yet, clicking the message type takes you straight to an edit screen to create a message.
If an auth message has already been customized, you go to a detail screen instead.
Since there's no detail screen on the "create for the first time" flow, we don't have anywhere to put a more detailed description or a preview of a default value.
Add a view screen that works if a message is "empty" so we can add this stuff.
(The only reason we don't already have this is that it took a little work to build; this also generally improves the consistency and predictability of this interface.)
Test Plan: {F6607665}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20664
Summary:
Depends on D20662. Ref T13343. Installs may reasonably want to change the guidance users receive in "Email Login"/"Forgot Password" email.
(In an upcoming change I plan to supply a piece of default guidance, but Auth Messages need a few tweaks for this.)
There's probably little reason to provide guidance on the "Set Password" flow, but any guidance one might issue on the "Email Login" flow probably doesn't make sense on the "Set Password" flow, so I've included it mostly to make it clear that this is a different flow from a user perspective.
Test Plan:
- Set custom "Email Login" and "Set Password" messages.
- Generated "Email Login" mail by using the "Login via email" link on the login screen.
- Generated "Set Password" email by trying to set a password on an account with no password yet.
- Saw my custom messages in the resulting mail bodies.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20663
Summary:
Ref T13343. This makes "Password Reset" email a little more consistent with other modern types of email. My expectation is that this patch has no functional changes, just organizes code a little more consistently.
The new `setRecipientAddress()` mechanism deals with the case where the user types a secondary (but still verified) address.
Test Plan:
- Sent a normal "login with email" email.
- Sent a "login with email to set password" email by trying to set a password on an account with no password yet.
- Tried to email reset a bot account (no dice: they can't do web logins so this operation isn't valid).
- Tested existing "PeopleMailEngine" subclasses:
- Created a new user and sent a "welcome" email.
- Renamed a user and sent a "username changed" email.
- Reviewed all generated mail with `bin/mail list-outbound`.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20662
Summary:
See D20650. Long ago, this got added as "pastebin", but that's the name of another product/company, not a generic term for paste storage.
Rename the database to `phabricator_paste`.
(An alternate version of this patch would rename `phabricator_search` to `phabricator_bing`, `phabricator_countdown` to `phabricator_spacex`, `phabricator_pholio` to `phabricator_adobe_photoshop`, etc.)
Test Plan:
- Grepped for `pastebin`, now only found references in old patches.
- Applied patches.
- Browsed around Paste in the UI without encountering issues.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D20661
Summary:
Fixes T13345. See D20650. Currently, `PhabricatorCursorPagedPolicyAwareQuery` does a JOIN against the "title" field so it can apply additional ranking/ordering conditions to the query.
This means that documents with no title (which don't have this field) are always excluded from the result set.
We'd prefer to include them, just not give them any bonus ranking/relevance boost. Use a LEFT JOIN so they get included.
Test Plan:
- Applied D20650 (diff 1), made it use raw `getTitle()` as the document title, indexed a paste with no title.
- Searched for a term in the paste body.
- Before change: no results.
- After change: found result.
{F6601159}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13345
Differential Revision: https://secure.phabricator.com/D20660
Summary:
Depends on D20654. Ref T4900. When a task is edited, emit a "workboards" event for all boards it appears on (in a future change, this should also include all boards it //previously// appeared on, and all parents of both sets of boards -- but I'm just getting things working for now).
When we receive a "workboards" event, check if the visible board should be updated.
Aphlict has a complicated intra-window leader/follower election system which could let us process this update event exactly once no matter how many windows a user has open with the same workboard. I'm not trying to do any of this since it seems fairly rare. It makes sense for events like "you have new notifications" where we don't want to generate 100 Ajax calls if the user has 100 windows open, but very few users seem likely to have 100 copies of the same workboard open.
Test Plan:
- Ran `bin/aphlict debug`.
- Opened workboard A in two windows, X and Y.
- Edited and moved tasks in window X.
- Saw "workboards" messages in the Aphlict log.
- Saw window Y update in nearly-real-time (locally, this is fast enough that it feels instantaneous).
Then:
- Stopped the Aphlcit server.
- Edited a task.
- Started the Aphlict server.
- Saw window Y update after a few moments (i.e., update in response to a reconnect).
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20656
Summary:
Fixes T13342. This does a few different things, although all of them seem small enough that I didn't bother splitting it up:
- Support export of "remarkup" custom fields as text. There's some argument here to export them in some kind of structure if the target is JSON, but it's hard for me to really imagine we'll live in a world some day where we really regret just exporting them as text.
- Support export of "date" custom fields as dates. This is easy except that I added `null` support.
- If you built PHP from source without "--enable-zip", as I did, you can hit the TODO in Excel exports about "ZipArchive". Since I had a reproduction case, test for "ZipArchive" and give the user a better error if it's missing.
- Add a setup check for the "zip" extension to try to avoid getting there in the first place. This is normally part of PHP so I believe users generally won't hit it, I just hit it because I built from source. See also T13232.
Test Plan:
- Added a custom "date" field. On tasks A and B, set it to null and some non-null value. Exported both tasks to Excel/JSON/text, saw null and a date, respectively.
- Added a custom "remarkup" field, exported some values, saw the values in Excel.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13342
Differential Revision: https://secure.phabricator.com/D20658
Summary:
Depends on D20653. Ref T4900. Pass ordering details to the reload endpoint so it can give the client accurate ordering/header information in the response.
The removed comment mentions this, but here's why this is a difficult mess:
- In window A, view a board with "Group by: Owner" and no tasks owned by "Alice". Since "Alice" owns no tasks, this means the columns do not have an "Assigned to: Alice" header!
- In window B, edit task T and assign it to Alice.
- In window A, press "R".
Window A now not only needs to update to properly reflect the state of task T, it actually needs to draw a new "Assigned to: Alice" header in every column.
Fortunately, the "group by" code anticipates this being a big mess, is fairly careful about handling it, and the client can handle this state change and the actual code change here isn't too involved. This is just causing a lot of not-very-obvious indirect effects in the pipeline to handle these situations that need complex redraws.
Test Plan:
- After making various normal edits/creates/moves in window A, pressed "R" in window B. Saw ordering reflected correctly after sync.
- Went through the whole "Group by: Owner" + assign to unrepresented owner flow above. After pressing "R", saw "Assigned to: Alice" appear on the board.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20654
Summary:
Depends on D20652. Ref T4900. When the user presses "R", send a list of cards currently visible on the client and their version numbers.
On the server:
- Compare the client verisons to the server versions so we can skip updates for objects which have not changed. (For now, the client version is always "1" and the server version is always "2", so this doesn't do anything meaningful, and every card is always updated.)
- Compare the client visible set to the server visible set and "remove" any cards which have been removed from the board.
I believe this means that "R" always puts the board into the right state (except for some issues with client orderings not being fully handled yet). It's not tremendously efficient, but we can make versioning better (using the largest object transaction ID) to improve that and loading the page in the first place doesn't take all that long so even sending down the full visible set shouldn't be a huge problem.
Test Plan:
- In window A, removed a card from a board.
- In window B, pressed "R" and saw the removal reflected on the client.
- (Also added cards, edited cards, etc., and didn't catch anything exploding.)
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20653
Summary:
Depends on D20639. Ref T4900. Currently, "BoardResponseEngine" has a `setObjectPHID()` method. This is called after edit operations to mean "we just edited object X, so we know it needs to be updated".
Move toward `setUpdatePHIDs(...)` in all cases, with `setUpdatePHIDs(array(the-object-we-just-edited))` as a special case of that. After this change, callers pass:
- An optional list of PHIDs they know need to be updated on the client. Today, this is always be a card we just edited (on edit/move flows), or a sort of made-up list of PHIDs for the moment (when you press "R"). In the future, the "R" endpoint will do a better job of figuring out a more realistic update set.
- An optional list of PHIDs currently visible on the client. This is used to update ordering details and mark cards for removal. This is currently passed by edit/move, but not by pressing "R" (it will be in the future).
- An optional list of objects. The "R" workflow has to load these anyway, so we can save a couple queries by letting callers pass them. For now, the edit/move flows still rely on the engine to figure out what it needs to load.
This does very little to actually change client behavior, it mostly just paves the way for the next update to the "R" workflow to make it handle add/remove cases properly.
Test Plan:
- Edited and moved cards on a workboard.
- Pressed "R" to reload a workboard.
Neither of these operations seem any worse off than they were before. They still don't fully work:
- When you edit a card and delete the current workboard project from it, it remains visible. This is also the behavior on `master`. This is sort of intentional since we don't necessarily want to make these cards suddenly disappear? Ideally, we would probably have some kind of "tombstone" state where the card can still be edited but can't be dragged, and the next explicit user interaction would clean up old tombstones. This interaction is very rare and I don't think it's particularly important to specialize.
- When a card is removed from the board, "R" can't currently figure out that it should be removed from the client. This is because the client does not yet pass a "visiblePHIDs" state. It will in an upcoming change.
- The "R" flow always sends a full set of card updates, and can not yet detect that some cards have not changed.
- There's a TODO, but some ordering stuff isn't handled yet.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20652
Summary:
Depends on D20638. Ref T4900. This is an incremental step toward proper workboard updates.
Currently, the client can mostly update its view because we do updates when you edit or move a card, and the client and server know how to send lists of card updates, so a lot of the work is already done.
However, the code assumes we're only updating/redrawing one card at a time. Make the client accept and process multiple card updates.
In future changes, I'll add versioning (so we only update cards that have actually changed), fix the "TODO" around ordering, and move toward actual Aphlict-based real-time updates.
Test Plan:
- Opened the same workboard in two windows.
- Edited cards in one window, pressed "R" (capital letter, with no modifier keys) to reload the second window.
- Saw edits and moves reflected accurately after sync, except for some special cases of header/order interaction (see "TODO").
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20639
Summary: I was poking around in `PhabricatorAuthProviderViewController` and noticed that none of the subclass-specific rendering was working. Figured out that no one ever calls `PhabricatorAuthProviderConfigTransaction->setProvider()`, so instead of adding all those calls, just pull the provider out of the config object.
Test Plan:
Before: {F6598145}
After: {F6598147}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D20655
Summary: Ref T2784. These are lookin' pretty stable. Subclasses like `DiffusionGetLintMessagesConduitAPIMethod` have their warnings about unstable methods, so just remove this warning in the base class.
Test Plan: Loaded `/conduit`, observed lack of unstable warnings. Only unstable methods are now `diffusion.getlintmessages`, `diffusion.looksoon`, and `diffusion.updatecoverage`.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T2784
Differential Revision: https://secure.phabricator.com/D20651
Summary: Forgot to post this after D20394. Fixes T7667.
Test Plan:
* Edited some providers with the config locked and unlocked.
* Opened the edit form with the config unlocked, locked the config, then saved, and got a sensible error: {F6576023}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T7667
Differential Revision: https://secure.phabricator.com/D20645
Summary: See <https://discourse.phabricator-community.org/t/phd-status-calls-to-undefined-method-when-theres-no-instance/2918>. This call should be `logInfo()`.
Test Plan:
- Purged `PHABRICATOR_INSTANCE` from my environment. In a Phacility development environment, it comes from loading `services/`.
- Ran `bin/phd stop` with all daemons already stopped.
- Before: bad call.
- After: helpful error.
- Ran some other `bin/phd start`, `bin/phd status`, etc., to kick the tires.
- Grepped for remaining `writeInfo()` calls (found none).
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20649
Summary: See rPaacc62463d61. D20551 added some `CAN_INTERACT` checks, but `CAN_INTERACT` needs to be checked with `canInteract()` to fall back to `CAN_VIEW` properly. D20558 cleaned up most of this but missed one callsite; fix that up too.
Test Plan: Removed a comment on a commit.
Reviewers: amckinley, 20after4
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20648
Summary: Ref T13332. This fix isn't terribly satisfying, but resolves the issue: this behavior may attempt to build HTML blocks with metadata after Javascript footer rendering has started. Use `hsprintf()` to flatten the markup earlier.
Test Plan: Put a `T123` reference in the description of a Pholio image, then loaded a mock.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13332
Differential Revision: https://secure.phabricator.com/D20647
Summary:
Ref D20645. Start making this view a little more useful:
{F6573605}
Test Plan: Mk. 1 eyeball
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D20646
Summary:
Depends on D20636. Ref T4900. Previously, some workflows didn't know how to identify the default state for the board, so they needed explicit ("force") parameters.
Everything uses the same state management code now so we can rip out the old stuff.
Test Plan: Changed board filters, selected a custom filter, edited a custom filter.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20637
Summary:
Depends on D20635. Ref T4900. Fixes T13316.
Currently, "Move Tasks to Column..." first prompts you to select a project, then prompts you for a column. The first step is prefilled with the current project, so the common case (moving to another column on the same board) requires you to confirm that you aren't doing an off-project move by clicking "Continue", then you can select a column.
This isn't a huge inconvenience and the workflow isn't terribly common, but it's surprising enough that it has come up a few times as a stumbling block. Particularly, we're suggesting to users that they're about to pick a column, then we're asking them to pick a project. The prompt also says "Project: XYZ", not "Project: Keep in current project" or something like that.
Smooth this out by splitting the action into two better-cued flows:
- "Move Tasks to Project..." is the current flow: pick a project, then pick a column.
- The project selection no longer defaults to the current project, since we now expect you to usually use this flow to move tasks to a different project.
- "Move Tasks to Column..." prompts you to select a column on the same board.
- This just skips step 1 of the workflow.
- This now defaults to the current column, which isn't a useful selection, but is more clear.
In both cases, the action cue ("Move tasks to X...") now matches what the dialog actually asks you for ("Pick an X").
Test Plan:
- Moved tasks across projects and columns within the same project.
- Hit all (I think?) the error cases and got sensible error and recovery behavior.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13316, T4900
Differential Revision: https://secure.phabricator.com/D20636
Summary: Depends on D20634. Ref T4900. Ref T13316. I'm planning to do a bit of additional cleanup here in followups, but this separates the main workflow out of the common controller.
Test Plan:
- Used "Move Tasks to Column..." to move some tasks on a board.
- Tried to move an empty column, hit an error.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13316, T4900
Differential Revision: https://secure.phabricator.com/D20635
Summary: Depends on D20633. Ref T4900. Separate the "Bulk Edit Tasks..." flow out of the main workboard controller.
Test Plan:
- Used "Bulk Edit Tasks" on a column with some tasks, got an appropraite edit operation.
- Used "Bulk Edit Tasks" on an empty column, got an error.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20634
Summary:
Depends on D20632. Ref T4900. As with other workflows on the board controller, this one is currently in the giant main "do everything" method. Move it to a separate controller.
This makes one material improvement: previously, we built the full board and did layout on all the cards before building the query. However, we do not actually need to do this: we don't need the cards. Instead, just do layout without handing over any card PHIDs. This is slightly faster, particularly on large boards.
Test Plan:
- Clicked "View as Query" on a board, got a query page for the column.
- Applied a custom filter, then clicked "View as Query" on a board. Got a query page merging the two filters.
- Applied a custom filter, then clicked "Veiw as Query" on a board, in a subproject column. Got a query page merging the two filters, respecting the project-ness of the column.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20633
Summary:
Depends on D20629. Ref T4900. Currently, the "Advanced Filter..." workflow on workboards (where you build a custom query) is inline in the main board controller.
This is because the filter flow depends on some of the board view state: we want to start with the current filter applied to the board, and preserve other state after you change the filter.
Now that `ViewState` can handle state management, we can separate this stuff out pretty easily.
Test Plan:
- Changed filters on a board.
- Applied a custom filter to a board.
- Changed the ordering of a board, then applied a custom filter. Verified "Cancel" and "Apply Filter" both preserve the order state.
- Changed the ordering of a board, then applied a custom filter, intentionally making a mistake in configuring the filter by entering an invalid date. Saw a dialog with an error. After correcting the error, saw state preserved properly.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20632
Summary:
Depends on D20628. Ref T4900. Currently, the "Save Current Order/Filter As Default" flows on workboards duplicate some state construction, and require parameters to be passed to them explicitly.
Now that state management is separate, they can reuse a bit more code and be made to look more like other similar controllers.
Test Plan:
- Changed the default order of a workboard.
- Changed the default filter of a workboard.
- Changed the order of a board to something non-default, then changed the filter, then saved the new filter as the default. Saw the modified order preserved and the modified filter removed, so I ended up in the right ("most correct") place: on the board, with my custom order in a URI parameter, and no filter URI parameter so I could see my new default filter behavior. This is an edge case that's not terribly important to get right, but we do get it right.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20629
Summary:
Depends on D20627. Ref T4900. If a user orders a board by "Sort by Title", then toggles the visibility of hidden columns, we want to keep the board sorted by title. To accomplish this, we pass the board state around to all the workflows here.
Pull the "bag of state properties" code out of the View controller. This class basically:
- reads state from a request (order, hidden, filter);
- manages defaults;
- provides the application with the current settings; and
- generates URIs with "?order=X&hidden=Y&filter=Z" to preserve state.
This is still a little questionable/transitional since some of the controllers need more cleanup.
Test Plan:
Toggled state, order, filters, clicked around various workflows and saw the filters preserved.
A lot of these workflows are pretty serious edge cases. For example, here's a feature this implements:
- Changed workboard order to "Title".
- Selected "Bulk Edit Tasks..." in an empty column and command-clicked it to open the link in a new window.
- Hovered over "Cancel".
- Saw the link properly generate with "?order=title", preserving the order.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20628
Summary: Depends on D20626. Ref T4900. On this controller, "id" is a separate property, but serves little purpose and complicates separating state management. Remove it.
Test Plan: Bulk edited a column, managed filters, did show/hide on columns, edited a column.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20627
Summary:
Ref T4900. The Workboard view controller currently has a lot of different responsibilities (it's ~1,500 lines long) because it has to manage the board filter/sort state.
I'd like to split it up and make it easier to move some workboard features (like "move all tasks in column...") to other Controllers, so we can have smaller controllers implementing specific workflows.
I think the state handling isn't really all that bad, it just needs to be separated a little better than it currently is.
To start with, remove the unused "slug" property.
Test Plan: Searched for "slug", got no hits. This class is final and the property is private, so this is certainly unused.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T4900
Differential Revision: https://secure.phabricator.com/D20626
Summary: Depends on D20624. Fixes T13330. The OAuth client pages are using some out-of-date rendering conventions; update them to modern conventions.
Test Plan:
Viewed a page, saw a modern header layout + curtain:
{F6534135}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13330
Differential Revision: https://secure.phabricator.com/D20625
Summary:
Ref T13330. Handles for "OAuthServerClient" objects currently do not have a URI, which causes some obscure fallout like a missing "Close" button when examining their transactions.
Add a URI.
Test Plan:
- Viewed an OAuth server client detail page.
- Edited a policy, changing it to a custom policy.
- Clicked "Custom Policy" in the resulting transaction to view a dialog explaining the changes.
- Before change: dialog has no close button.
- After change: dialog has a close button.
{F6534121}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13330
Differential Revision: https://secure.phabricator.com/D20624
Summary:
See D20540. I mistakenly multiplied some strenghts by 100 and others by 1000 when converting them to integers for `PhutilSortVector`.
Multiply them all by 100 (that is, divide the ones which were multiplied by 1000 by 10) to put things back the way they were.
Test Plan: quick mafs
Reviewers: amckinley, richardvanvelzen
Reviewed By: richardvanvelzen
Differential Revision: https://secure.phabricator.com/D20622
Summary: See PHI1319. Ref T13291. Bump the remarkup cache version, since the old JIRA / Asana rules may exist in the partial cached representation of remarkup blocks from older versions.
Test Plan: Typed some comments with various formatting, saw remarkup work fine.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291
Differential Revision: https://secure.phabricator.com/D20619
Summary:
Ref T13328. Currently, we read from `mysqldump` something like this:
```
until (done) {
for (100 ms) {
mysqldump > in-memory-buffer;
}
in-memory-buffer > disk;
}
```
This general structure isn't great. In this use case, where we're streaming a large amount of data from a source to a sink, we'd prefer to have a "select()"-like way to interact with futures, so our code is called after every read (or maybe once some small buffer fills up, if we want to do the writes in larger chunks).
We don't currently have this (`FutureIterator` can wake up every X milliseconds, or on future exit, but, today, can not wake for readable futures), so we may buffer an arbitrary amount of data into memory (however much data `mysqldump` can write in 100ms).
Reduce the update frequency from 100ms to 10ms, and limit the buffer size to 32MB. This effectively imposes an artificial 3,200MB/sec limit on throughput, but hopefully that's fast enough that we'll have a "wake on readable" mechanism by the time it's a problem.
Test Plan:
- Replaced `mysqldump` with `cat /dev/zero` as the source command, to get fast input.
- Ran `bin/storage dump` with `var_dump()` on the buffer size.
- Before change: saw arbitrarily large buffers (300MB+).
- After change: saw consistent maximum buffer size of 32MB.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13328
Differential Revision: https://secure.phabricator.com/D20617
Summary:
Fixes T13326. In D20571, I slightly generalized construction of an iterator over a set of files, but missed some code in other "bin/files ..." commands which was also affected.
Today, basically all of these workflows define their own "--all" and "names" flags. Pull these definitions up and implement them more consistently.
Test Plan: Ran multiple different `bin/files` commands with different combinations of arguments, saw consistent handling of iterator construction.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13326
Differential Revision: https://secure.phabricator.com/D20614
Summary:
Fixes T13327. Currently, when we try to bill an account and all members are disabled, we fail temporarily and the task retries forever.
At least for now, just treat this as a permanent failure.
Test Plan:
- Used `bin/phortune invoice` to generate a normal invoice for a regular subscription.
- Disabled all the account members, then tried again. Got a helpful permanent failure:
```
$ ./bin/phortune invoice --subscription PHID-PSUB-kbedwt5cyepoc6tohjq5 --auto-range
Set current time to Mon, Jun 24, 2:47 PM.
Preparing to invoice subscription "localb.phacility.com" from Fri, May 31, 10:14 AM to Sun, Jun 30, 10:14 AM.
WARNING
Manually invoicing will double bill payment accounts if the range overlaps an
existing or future invoice. This script is intended for testing and
development, and should not be part of routine billing operations. If you
continue, you may incorrectly overcharge customers.
Really invoice this subscription? [y/N] y
[2019-06-24 14:47:57] EXCEPTION: (PhabricatorWorkerPermanentFailureException) All members of the account ("PHID-ACNT-qp54y3unedoaxgkkjpj4") for this subscription ("PHID-PSUB-kbedwt5cyepoc6tohjq5") are disabled. at [<phabricator>/src/applications/phortune/worker/PhortuneSubscriptionWorker.php:88]
arcanist(head=experimental, ref.master=d92fa96366c0, ref.experimental=db4cd55d4673), corgi(head=master, ref.master=6371578c9d32), instances(head=stable, ref.master=ba9e4a19df1c, ref.stable=37fb1f4917c7), libcore(), phabricator(head=master, ref.master=65bc481c91de, custom=11), phutil(head=master, ref.master=7adfe4e4f4a3), services(head=master, ref.master=5424383159ac)
#0 PhortuneSubscriptionWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:124]
#1 PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:163]
#2 PhabricatorWorker::scheduleTask(string, array, array) called at [<phabricator>/src/applications/phortune/management/PhabricatorPhortuneManagementInvoiceWorkflow.php:169]
#3 PhabricatorPhortuneManagementInvoiceWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:457]
#4 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:349]
#5 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/setup/manage_phortune.php:21]
$
```
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13327
Differential Revision: https://secure.phabricator.com/D20613
Summary: Ref T13321. The daemons no longer write PID files, so we no longer need to pass any of this stuff to them.
Test Plan: Grepped for affected symbols.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13321
Differential Revision: https://secure.phabricator.com/D20608
Summary:
Ref T13321. This gets rid of the last pidfile readers in Phabricator; we just use the process list instead.
These commands always only work on the current instance since they don't make much sense otherwise.
Test Plan: Ran `bin/phd start` and `bin/phd reload` with and without daemons running.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13321
Differential Revision: https://secure.phabricator.com/D20606
Summary:
Ref T13321. Fixes T11037. Realign "bin/phd status" to just mean "show daemon processes on this host".
The value of `bin/phd status` as a mixed remote/local command isn't clear, and the current output is a confusing mess (see T11037).
This also continues letting us move away from PID files.
Test Plan: Ran `bin/phd status`, saw sensible local process status.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13321, T11037
Differential Revision: https://secure.phabricator.com/D20604
Summary:
Ref T13321. Previously, the behavior was:
- `bin/phd stop --gently`: Stop all daemons with PID files that belong to the current instance.
- `bin/phd stop`: Stop all daemons with PID files that belong to the current instance. Complain if there are more processes.
- `bin/phd stop --force`: Stop all processes that look like daemons, ignoring instances.
The new behavior is:
- `bin/phd stop`: Stop all processes that look like daemons and belong to the current instance.
- `bin/phd stop --force`: Stop all processes that look like daemons, period.
Test Plan: Grep / documentation only.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13321
Differential Revision: https://secure.phabricator.com/D20602
Summary:
Ref T13321. Depends on D20600. Make `bin/phd stop` mean:
- `bin/phd stop`: Stop all processes which have daemon process titles. If we're instanced, only stop daemons for the current instance.
- `bin/phd stop --force`: Stop all processes which have deamon process titles for any instance.
We no longer read or care about PID files on disk, and this moves us away from PID files.
This makes unusual flag `--gently` do nothing. A followup will update the documentation and flags to reflect actual usage/behavior.
This also removes the ability to stop specific PIDs. This was somewhat useful long, long ago when you might explicitly run different copies of the `PullLocal` daemon with flags to control which repositories they updated, but with the advent of clustering it's no longer valid to run custom daemon loadouts.
Test Plan: Ran `bin/phd start`, then `bin/phd stop`. Saw instance daemons stop. Ran `bin/phd stop --force`, saw all daemons stop.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13321
Differential Revision: https://secure.phabricator.com/D20601
Summary:
See <https://discourse.phabricator-community.org/t/cannot-audit-a-git-commit/2848>. In D20581, I made some audit behavior dependent upon identities, but the actual edit flow doesn't load them. This can cause us to raise an "attach identities first" exception in the bowels of the edit workflow and trigger unexpected behavior at top level.
Load identities when editing a commit so that the transaction flows have access to identity information and can use it to figure out if a user is an author, etc.
Test Plan:
- As an auditor, applied an "Accept Commit" action to an open audit after D20581.
- Before patch: accept no-ops internally since the preconditions throw.
- After patch: accept works properly.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20612
Summary:
Ref T13291. See PHI1312. Currently, if you link to a JIRA or Asana issue with an anchor (`#asdf`) or query parameters (`?a=b`), we:
- treat the link as an external object reference and attempt a lookup on it;
- if the lookup succeeds, we discard the fragment or parameters when re-rendering the rich link (with the issue/task title).
Particularly, the re-rendering part uses the canonical URI of the object, and can discard these parameters/fragments, which is broken/bad.
As a first pass at improving this, just don't apply special behavior for links with anchors or parameters -- simply treat them as links.
In some future change, we could specialize this behavior and permit certain known parameters or anchors or something, but these use cases are likely fairly marginal.
Test Plan:
Before:
{F6516392}
After:
{F6516393}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291
Differential Revision: https://secure.phabricator.com/D20592
Summary: Ref T13319. Ref PHI1302. Migrate `PhabricatorEditEngineConfigurationTransaction` to modular transactions and add some additional transaction rendering to make these edits less opaque.
Test Plan: Hit all the form edit controllers, viewed resulting transaction timeline.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T13319
Differential Revision: https://secure.phabricator.com/D20595
Summary:
Fixes T13324. Ref PHI1288. Currently, if you edit an Owners package that has some paths with no trailing slashes (like `README.md`) so their internal names and display names differ (`/README.md` display, vs `/README.md/` internal), the "Show Details" in the transaction log shows the path as re-normalized even if you didn't touch it.
Instead, be more careful about handling display paths vs internal paths.
(This code on the whole is significantly less clear than it probably could be, but this issue is so minor that I'm hesitant to start ripping things out.)
Test Plan:
- In a package with some paths like `/src/` and some paths like `/src`:
- Added new paths.
- Removed paths.
- Changed paths from `/src/` to `/src`.
- Changed paths from `/src` to `/src/`.
In all cases, the "paths" list and the transaction record identically reflected the edit in the way I expected them to.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13324
Differential Revision: https://secure.phabricator.com/D20596
Summary:
Ref T13319. Currently, transactions about changes to a default form value use a raw internal key for the affected field and don't show the actual value change.
An ideal implementation will likely require us to specialize a great deal of rendering, but we can do much better than we currently do without too much work:
- Try to pull the actual `EditField` object for the key so we can `getLabel()` it and get a human-readable label (like `Visible To` instead of `policy.view`).
- Add a "(Show Changes)" action that dumps the raw values as more-or-less JSON, so you can at least figure out what happened if you're sophisticated enough.
Test Plan:
Before:
{F6516640}
After:
{F6516642}
The quality of "Show Details" varies a lot. For some fields, like "Description", it's pretty good:
{F6516645}
For others, like "Assigned To", it's better than nothing but pretty technical:
{F6516647}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13319
Differential Revision: https://secure.phabricator.com/D20594
Summary:
Fixes T13317. On `admin.phacility.com`, an enterprising user added `noreply@admin.phacility.com` to their account. This caused them to become CC'd on several support issues over the last year, because we send mail "From" this address and it can get CC'd via reply/reply all/whatever else.
The original driving goal here is that if I reply to a task email and CC you on my reply, that should count as a CC in Phabricator, since this aligns with user intent and keeps them in the loop.
This misfire on `noreply@` is ultimately harmless (being CC'd does not grant the user access permission, see T4411), but confusing and undesirable. Instead:
- Don't allow reserved addresses ("noreply@", "ssladmin@", etc) to trigger this subscribe-via-CC behavior.
- Only count verified addresses as legitimate user recipients.
Test Plan:
- Added a `bin/mail receive-test --cc ...` flag to make this easier to test.
- Sent mail as `bin/mail receive-test --to X --as alice --cc bailey@verified.com`. Bailey was CC'd both before and after the change.
- Sent mail as `bin/mail receive-test --to X --as alice --cc unverified@imaginary.com`, an address which Bailey has added to her account but not verified.
- Before change: Bailey was CC'd on the task anyway.
- After change: Bailey is not CC'd on the task.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13317
Differential Revision: https://secure.phabricator.com/D20593
Summary:
Fixes T13312. Currently, {nav Manage > Branches} has a list of branches on the same page. This has a few minor issues:
- Pager is at the top (see T13312), which is weird.
- "Default" icon is mystery meat.
- Table is kind of pointless/redundant in general?
Previously, this table had more information about technical status of each branch (autoclose/track/publish) but most of these details have been simplified/eliminated, and the main "Branches" view now has more information than it did before.
Get rid of this and just link to the main view.
Test Plan: Viewed "Branches" in UI, saw a link to the main view instead of a weird table.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13312
Differential Revision: https://secure.phabricator.com/D20584
Summary:
Fixes T13309. If you void the warranty on a repository on disk and turn it into a shallow clone, Phabricator currently can't serve it.
We don't support hosting shallow working copies, but we should still parse and proxy the protocol rather than breaking in a mysterious way.
Test Plan:
- Created a shallow working copy with `mv X X.full; git clone --depth Y file://.../X.full X` in the storage directory on disk.
- Cloned it with `git clone <uri>`.
- Deleted all the refs inside it so the wire only has "shallow" frames; cloned it.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13309
Differential Revision: https://secure.phabricator.com/D20577
Summary:
See <https://discourse.phabricator-community.org/t/view-task-from-maniphest-e-mail-doesnt-have-url/2827>.
I added "View Task" / "View Commit" buttons recently but the logic for generating URIs isn't quite right. Fix it up.
Test Plan:
- Commented on a task.
- Used `bin/mail show-outbound --id ... --dump-html > out.html` to dump the HTML.
- Previewed the HTML in a browser.
- This time, actually clicked the button to go to the task.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20586
Summary:
Fixes T13304. Shell pipes and redirects do not have robust behavior when errors occur. We provide "--compress" and "--output" flags as robust alternatives, but do not currently recommend their use.
- Recommend their use, since their error handling behavior is more robust in the face of issues like full disks.
- If "--compress" is provided but won't work because the "zlib" extension is missing, raise an explicit error. I believe this extension is very common and this error should be rare. If that turns out to be untrue, we could take another look at this.
- Also, verify some flag usage sooner so we can exit with an error faster if you mistype a "bin/storage dump" command.
Test Plan: Read documentation, hit affected error cases, did a dump and spot-checked that it came out sane looking.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13304
Differential Revision: https://secure.phabricator.com/D20572
Summary:
See <https://discourse.phabricator-community.org/t/unhandled-exception-when-logging-in-with-mfa/2828>. The recent changes to turn `msort()` on a vector an error have smoked out a few more of these mistakes.
These cases do not meaningfully rely on sort stability so there's no real bug being fixed, but we'd still prefer `msortv()`.
Test Plan: Viewed MFA and External Account settings panels. Did a `git grep 'msort(' | grep -i vector` for any more obvious callsites, but none turned up.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20587
Summary:
Fixes T13315. See that task for discussion.
Without `--background`, we currently treat this as a catastrophic failure, but it's relatively routine for some repository states. We can safely continue reparsing other steps.
Test Plan: Ran `bin/repository reparse --all X --message` with commits faked to all be unreachable. Got warnings instead of a hard failure on first problem.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13315
Differential Revision: https://secure.phabricator.com/D20588
Summary:
Depends on D20580. Fixes T13311. When we choose which actions to show a user, we can either show them "auditor" actions (like "raise concern") or "author" actions (like "request verification").
Currently, we don't show "author" actions if you're the author of the commit via an identity mapping, but we should. Use identity mappings where they exist.
(Because I've implemented `getEffectiveAuthorPHID()` in a way that requires `$data` be attached, it's possible this will make something throw a "DataNotAttached" exception, but: probably it won't?; and that's easy to fix if it happens.)
Test Plan:
See D20580. As `@alice`, viewed the commit in the UI.
- Before: got auditor actions presented to me.
- After: got author actions.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13311
Differential Revision: https://secure.phabricator.com/D20581
Summary:
Ref T13311. We currently don't use committer identity mappings when triggering audits, so if a user is only associated with an identity via manual mapping we won't treat them as the author.
Instead, use the identity and manual mapping if they're available.
Test Plan:
- Pushed a commit as `xyz <xyz@example.org>`, an address with no corresponding user.
- In the UI, manually associated that identity with user `@alice`.
- Ran `bin/repository reparse --publish <hash>` to trigger audits and publishing for the commit.
- Before: observed the `$author_phid` was `null`.
- After: observed the `$author_phid` is Alice.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13311
Differential Revision: https://secure.phabricator.com/D20580
Summary:
Fixes T13310. Use cases in the form "users with no access to any spaces can not <do things>" are generally unsupported (that is, we consider this to mean that the install is misconfigured), but "log out" is a somewhat more reasonable sort of thing to do and easy to support.
Drop the requirement that users be logged in to access the Logout controller. This skips the check for access to any Spaces and allows users with no Spaces to log out.
For users who are already logged out, this just redirects home with no effect.
Test Plan:
- As a user with access to no Spaces, logged out. (Before: error; after: worked).
- As a logged-out user, logged out (was redirected).
- As a normal user, logged out (normal logout).
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13310
Differential Revision: https://secure.phabricator.com/D20578
Summary:
Fixes T13313. The "Download Raw Diff" workflow in Differential currently uses an older way of interacting with Files that doesn't engage the chunk engine and can't handle 8MB+ files.
Update to `IteratorFileUploadSource` -- we're still passing in a single giant blob, but this approach can be chunked.
This will still break somewhere north of 8MB (it will break at 2GB with the PHP string limit if nowhere sooner, since we're putting the entire raw diff in `$raw_diff` rather than using a rope/stream) but will likely survive diffs in the hundreds-of-megabytes range for now.
Test Plan:
- Added `str_repeat('x', 1024 * 1024 * 9)` to the `$raw_diff` to create a 9MB+ diff.
- Configured file storage with no engine explicitly configured for >8MB chunks (i.e., "reasonably").
- Clicked "Download Raw Diff".
- Before: misleading file storage engine error ("no engine can store this file").
- After: large, raw diff response.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13313
Differential Revision: https://secure.phabricator.com/D20579
Summary: Ref T13303. See B22967. This should be "msortv()" but didn't get updated properly.
Test Plan: The system works!
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13303
Differential Revision: https://secure.phabricator.com/D20585
Summary: Ref T13303. I upgraded this to a vector-based sort but forgot to type a "v", which means the sort has different stability under PHP 5.5. See D20582 for a root cause fix.
Test Plan: Locally, on PHP7, not much changes. I expect this to fix the odd selection of title stories in mail and notification stories on `secure`, which is running PHP 5.5.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13303
Differential Revision: https://secure.phabricator.com/D20583
Summary:
Fixes T13307. We currently require "CAN_EDIT" to sign actions, but it's fine to sign a comment with only "CAN_INTERACT".
Since the actions like "Accept Revision" already work like this, the fix is one line.
Test Plan: {F6488135}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13307
Differential Revision: https://secure.phabricator.com/D20574
Summary:
Ref T13306. Currently, there's no easy way to import a third-party local-disk file dump into a Phacility instance.
Add some more options to `bin/files migrate` to support this. In particular, this enables:
```
$ ./bin/files --from-engine local-disk --engine amazon-s3 --local-disk-source path/to/backup
```
...to import these files into S3 directly.
These are general-purpose options and theoretically useful in other use cases, although realistically those cases are probably very rare.
Test Plan: Used `bin/files` with the new options to move files in and out of local disk storage in an arbitrary backup directory. Got clean exports/imports.
Reviewers: amckinley
Maniphest Tasks: T13306
Differential Revision: https://secure.phabricator.com/D20571
Summary:
Ref T13298. Add a simple profiler as a starting point to catch any egregiously expensive rules or conditions.
This doesn't profile rule actions, so if "Add subscriber" (or whatever) is outrageously expensive it won't show up on the profile. Right now, actions get evaluated inside the Adapter so they're hard to profile. A future change could likely dig them out without too much trouble. I generally expect actions to be less expensive than conditions.
This also can't pin down a //specific// condition being expensive, but if you see that `H123` takes 20s to evaluate you can probably guess that the giant complicated regex is the expensive part.
Test Plan: {F6473407}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13298
Differential Revision: https://secure.phabricator.com/D20566
Summary:
Depends on D20566. Ref T13298. See PHI1280. Currently, there's no clean way to disable problematic personal rules. This comes up occasionally and sometimes isn't really the best approach to solving a problem, but is a generally reasonable capability to provide.
Allow Herald rules (including personal rules) to be disabled/enabled via `bin/herald rule ... --disable/--enable`.
Test Plan: Used the CLI to disable and enable a personal rule.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: jmeador
Maniphest Tasks: T13298
Differential Revision: https://secure.phabricator.com/D20567
Summary:
Fixes T13300. Currently, if you create a revision and then immediately land it (either using `--draft` or just beating Harbormaster to the punch) it can be stuck in "Draft" forever.
Instead, count landing changes like this as a publishing action.
Test Plan:
- Used `arc diff --hold` to create a revision, then pushed the commit immediately.
- Before change: revision closed, but was stuck in draft.
- After change: revision closed and was promoted out of draft.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13300
Differential Revision: https://secure.phabricator.com/D20565
Summary:
See PHI1118. That issue may describe more than one bug, but the recent ordering changes to the import pipeline likely make this at least part of the problem.
Previously, commits would always close associated revisions before we made it to the "publish" step. This is no longer true, so we might be triggering audits on a commit before the associated revision actually closes.
Accommodate this by counting a revision in either "Accepted" or "Published (Was Previously Accepted)" as "reviewed".
Test Plan:
- With commit C affecting paths in package P with "Audit Unreviewed Commits and Commits With No Owner Involvement", associated with revision R, with both R and C authored by the same user, and "R" in the state "Accepted", used `bin/repository reparse --publish <hash>` to republish the commit.
- Before change: audit by package P triggered.
- After change: audit by package P no longer triggered.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20564
Summary:
Ref T13303. In D20525 I fixed an issue where transaction rendering could use cached values with the wrong viewer by reloading transactions.
However, reloading transactions may also reorder them as a side effect, since `withPHIDs(...)` does not imply an order. This can make transaction rendering order in mail wrong/inconsistent.
Instead, reorder the transactions before continuing so mail transaction order is consistent.
Test Plan: Applied a group of transactions to a task, saw a more consistent rendering order in mail after the change.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13303
Differential Revision: https://secure.phabricator.com/D20563
Summary:
See downstream <https://phabricator.wikimedia.org/T1050>. Some time ago, we added a "View Revision" button to Differential mail. This hasn't created any problems and generally seems good / desirable.
It isn't trivial to just add everywhere since we need a translation string in each case, but at least add it to Maniphest for now. Going forward, we can fill in more applications as they come up.
Test Plan:
Used `bin/mail show-outbound --id <x> --dump-html`:
{F6470461}
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20561
Summary:
See downstream <https://phabricator.wikimedia.org/T88655>. This is very marginal, but we currently allow comments consisting of //only// whitespace.
These are probably always mistakes, so treat them like completely empty comments.
(We intentionally do not trim leading or trailing whitespace from comments when posting them becuase leading spaces can be used to trigger codeblock formatting.)
Test Plan:
- Posted empty, nonempty, and whitespace-only comments.
- Whitespace-only comments now have the same behavior as truly empty comments (e.g., do not actually generate a transaction).
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20562
Summary: These instructions are fairly old and can be a little fancier and more clear in the context of modern Phabricator. Drop the reference to "HPHP", link the actual timezone list, wordsmith a little.
Test Plan: d( O_o )b
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20560
Summary:
See downstream <https://phabricator.wikimedia.org/T902>. Currently, timezones are rendered with their raw internal names (like `America/Los_Angeles`) which include underscores.
Replacing underscores with spaces is a more human-readable (and perhaps meaningfully better for things like screen readers, although this is pure speculation).
There's some vague argument against this, like "administrators may need to set a raw internal value in `phabricator.timezone` and this could mislead them", but we already give a pretty good error message if you do this and could improve hinting if necessary.
Test Plan: Viewed timezone list in {nav Settings} and the timezone "reconcile" dialog, saw a more-readable "Los Angeles".
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20559
Summary:
Ref T13289. See D20551. In D20551, I implemented some "CAN_INTERACT" checks against certain edits, but these checks end up testing "CAN_INTERACT" against objects like Conpherence threads which do not support a distinct "CAN_INTERACT" permission. I misrembered how the "CAN_INTERACT" fallback to "CAN_VIEW" actually works: it's not fully automatic, and needs some explicit "interact, or view if interact is not available" checks.
Use the "interact" wrappers to test these policies so they fall back to "CAN_VIEW" if an object does not support "CAN_INTERACT". Generally, objects which have a "locked" state have a separate "CAN_INTERACT" permission; objects which don't have a "locked" state do not.
Test Plan: Created and edited comments in Conpherence (or most applications other than Maniphest).
Reviewers: amckinley
Maniphest Tasks: T13289
Differential Revision: https://secure.phabricator.com/D20558
Summary:
Ref T13289. If you do this:
- Subscribe to a task (so we don't generate a subscribe side-effect later).
- Prepare a transaction group: sign with MFA, change projects (don't make any changes), add a comment.
- Submit the transaction group.
...you'll get prompted "Some actions don't have any effect (the non-change to projects), apply remaining effects?".
If you confirm, you get MFA'd, but the MFA flow loses the "continue" confirmation, so you get trapped in a workflow loop of confirming and MFA'ing.
Instead, retain the "continue" bit through the MFA.
Also, don't show "You can't sign an empty transaction group" if there's a comment.
See also T13295, since the amount of magic here can probably be reduced. There's likely little reason for "continue" or "hisec" to be magic nowadays.
Test Plan:
- Went through the workflow above.
- Before: looping workflow.
- After: "Continue" carries through the MFA gate.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13289
Differential Revision: https://secure.phabricator.com/D20552
Summary: Ref T11741. See PHI1276. After the switch to "Ferret", this table has no remaining readers or writers.
Test Plan:
- Ran `bin/storage upgrade -f`, no warnings.
- Grepped for class name, table name, `stemmedCorpus` column; got no relevant hits.
- Did a fulltext search.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T11741
Differential Revision: https://secure.phabricator.com/D20549
Summary:
Ref T13289. This tightens up a couple of corner cases around locked threads.
Locking is primarily motivated by two use cases: stopping nonproductive conversations on open source installs (similar to GitHub's feature); and freezing object state for audit/record-keeping purposes.
Currently, you can edit or remove comments on a locked thread, but neither use case is well-served by allowing this. Require "CAN_INTERACT" to edit or remove a comment.
Administrators can still remove comments from a locked thread to serve "lock a flamewar, then clean it up", since "Remove Comment" on a comment you don't own is fairly unambiguously an administrative action.
Test Plan:
- On a locked task, tried to edit and remove my comments as a non-administrator. Saw appropriate disabled UI state and error dialogs (actions were disallowed).
- On a locked task, tried to remove another user's comments as an administrator. This works.
- On a normal task, edited comments normally.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13289
Differential Revision: https://secure.phabricator.com/D20551
Summary:
Ref T13289. See <https://discourse.phabricator-community.org/t/fatal-error-in-pagination-in-drydock-resources-host-logs-all-logs/2735>.
`bin/drydock lease` and the web UI for reviewing all object logs when there is more than one page of logs didn't get fully updated to the new cursors.
- Use a cursor pager in `bin/drydock lease`.
- Implement `withIDs()` in `LeaseQuery` so the default paging works properly.
Test Plan:
- Ran `bin/drydock lease`, got a lease with log output along the way.
- Set page size to 2, viewed host logs with multiple pages, paged to page 2.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13289
Differential Revision: https://secure.phabricator.com/D20553
Summary:
Depends on D20546. Ref T13283. Currently, if you do something (transactions "A", "B") and Herald does some things in response (transaction "C"), Herald acts only on the things you did ("A", "B") since the thing it did ("C") didn't exist yet, until it ran.
However, if you use the test console to test rules against the object we'll pick up all three transactions since they're all part of the same group. This isn't ideal.
To fix this, skip transactions which Herald applied, since it obviously didn't consider them when it was evaluating.
Test Plan:
- Created a revision, in the presence of a Herald rule that adds reviewers.
- Then, ran the revision through the test console.
- Before: saw the "Herald added reviewers: ..." transaction in the transaction group Herald evaluated.
- After: saw only authentic human transactions.
{F6464064}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13283
Differential Revision: https://secure.phabricator.com/D20547
Summary:
Ref T13283. Currently, each Editor sets its own group ID, so if you create a revision and then Herald does some stuff, the two groups of transactions get different group IDs.
This means the test console is slightly misleading (it will only pick up the Herald transactions). It's going to be misleading anyway (Herald obviously can't evaluate Herald transactions) but this is at least a little closer to reality and stops Herald actions from masking non-Herald actions.
Test Plan:
- Created a revision. Herald applied one transaction.
- Used the test console.
- Before: The test console only picked up the single most recent Herald transaction.
- After: The test console picked up the whole transaction group.
{F6464059}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13283
Differential Revision: https://secure.phabricator.com/D20546
Summary:
Ref T13289. When you create a Phriction document, you currently get an email with the whole new content as a "diff".
You also get extra transactions in the email and on the page.
This is because Phriction isn't on EditEngine and doesn't mark "create" transactions in a modern way. Get them marked properly to fix these obviously-broken behaviors. This can all go away once Phriction switches to EditEngine, although I don't have any particular plans to do that in the immediate future.
Test Plan:
- Created a new document, viewed email, no longer saw redundant "edited content" transaction or "CHANGES TO CONTENT" diff.
- Updated a document, viewed email, got interdiff.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13289
Differential Revision: https://secure.phabricator.com/D20548
Summary:
Ref T13290. Prior to recent changes, if we parsed some commit C which was associated with a revision R, but R was already closed, we'd skip the whole set of updates because the "close the revision" transaction would fail and we'd throw because we did not `setContinueOnNoEffect()`.
We now continue on no effect so we can get the edge ("commit has revision" / "revision has commit"), since we want it in all cases, but this means we may also apply an extra "Updated revision to reflect committed changes" transaction and new diff. This can happen even if we're careful about not trying to apply this transaction to closed revisions, since two workers may race. (Today, we aren't too careful about this.)
To fix this, just make this transaction no-op itself if the revision is already closed by the time it tries to apply.
This happened on D20451 because a merge commit with the same hash as the last diff was pushed, but it's easiest to reproduce by just running `bin/repository reparse --message <commit>`, which updates related revisions with a new diff every time.
Test Plan:
- Ran `bin/repository reparse --messsage <commit>` several times, on a commit with an associated revision.
- Before: each run attached a new diff and created a new "updated to reflect committed changes" transaction.
- After: repeated runs had no effects.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13290
Differential Revision: https://secure.phabricator.com/D20545
Summary:
Ref T13290. Ref T13291. Now that a full URI is a "mention", the full URI in "Differential Revision: ..." also triggers a mention.
Stop it from doing that, since these mentions are silly/redundant/unintended.
The API here is also slightly odd; simplify it a little bit to get rid of doing "append" with "get + append + set".
Test Plan: Used `bin/repository reparse --publish` to republish commits with "Differential Revision: ..." and verified that the revision PHID was properly dropped from the mention list.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291, T13290
Differential Revision: https://secure.phabricator.com/D20544
Summary:
See PHI1222. When we publish several transactions to feed at once, we sort them by "action strength" to figure out which one gets to be the title story.
This sort currently uses `msort()`, which uses `asort()`, which is not a stable sort and has inconsistent behavior across PHP versions:
{F6463721}
Switch to `msortv()`, which is a stable sort. Previously, see also T6861.
If all transactions have the same strength, we'll now consistently pick the first one.
This probably (?) does not impact anything in the upstream, but is good from a consistency point of view.
Test Plan:
Top story was published after this change and uses the chronologically first transaction as the title story.
Bottom story was published before this change and uses the chronologically second transaction as the title story.
Both stories have two transactions with the same strength ("create" + "add reviewer").
{F6463722}
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20540
Summary:
Ref T13279. See that task for some discussion.
The accumulations of some of the datasets may be negative (e.g., if more tasks are moved out of a project than into it) which can lead to negative area in the stacked chart.
Introduce `min(...)` and `max(...)` to separate a function into points above or below some line, then mangle the areas to pick the negative and positive regions apart so they at least have a plausible physical interpretation and none of the areas are negative.
This is presumably not a final version, I'm just trying to produce a chart that isn't a sequence of overlapping regions with negative areas that is "technically" correct but not really possible to interpret.
Test Plan: {F6439195}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20506
Summary:
Ref T13279. Currently, we store a fairly low-level description of functions and datasets in a chart. This will create problems with (for example) translating function labels.
If you view a chart someone links you, it should say "El Charto" if you speak Spanish, not "The Chart" if the original viewer speaks English.
To support this, store a slightly higher level version of the chart: the chart engine key, plus configuration parameters. This is very similar to how SearchEngine works.
For example, the burndown chart now stores a list of project PHIDs, instead of a list of `[accumulate [sum [fact task.open <project-phid>]]]` functions.
(This leaves some serialization code with no callsites, but we may eventually have a "CustomChartEngine" which stores raw functions, so I'm leaving it for now.)
As a result, function labels provided by the chart engine are now translatable.
(Note that the actual chart is meaningless since the underlying facts can't be stacked like they're being stacked, as some are negative in some areas of their accumulation.)
Test Plan: {F6439121}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20504
Summary:
Ref T13279. Replace the hard-coded default range with a range computed by examining the chart data.
Instead of having a "Dataset" return a blob of wire data, "Dataset" now returns a structure with raw wire data plus a range. I expect to add more structured data here in future changes (tooltip/hover event data, maybe function labels).
Test Plan: {F6439101}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20503
Summary: Ref T13279. Slightly simplify domain handling by putting all the "[x, y]" stuff in an Interval class. I'm planning to do something similar for ranges next, so this should make that easierr.
Test Plan: Viewed chart, saw same chart as before.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20502
Summary: Ref T13279. Makes charts incrementally more useful by allowing the server to provide labels and colors for functions.
Test Plan: {F6438872}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20501
Summary:
Ref T13279. Adds client-side support for rendering function labels on charts, then labels every function as important data.
Works okay on mobile, although I'm not planning to target mobile terribly heavily for v0.
Test Plan: {F6438860}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20500
Summary:
Ref T13279. This adds support for:
- Datasets can have types, like "stacked area".
- Datasets can have multiple functions.
- Charts can store dataset types and datasets with multiple functions.
- Adds a "stacked area" dataset.
- Makes D3 actually draw a stacked area chart.
Lots of rough edges here still, but the result looks slightly more like it's supposed to look.
D3 can do some of this logic itself, like adding up the area stacks on top of one another with `d3.stack()`. I'm doing it in PHP instead because I think it's a bit easier to debug, and it gives us more options for things like caching or "export to CSV" or "export to API" or rendering a data table under the chart or whatever.
Test Plan: {F6427780}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20498
Summary:
Ref T13279. For now, we need to render burndowns from both Maniphest (legacy) and Projects (new prototype).
Consolidate this logic into a "BurndownChartEngine". I plan to expand this to work a bit like a "SearchEngine", and serve as a UI layer on top of the raw chart features.
The old "ChartEngine" is now "ChartRenderingEngine".
Test Plan:
- Viewed burndowns ("burnups") in Maniphest.
- Viewed burndowns in Projects.
- Saw the same chart.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20496
Summary:
Ref T13279. Since the use cases that have made it upstream are all for relatively complex charts (e.g., requiring aggregation and composition of multiple data series in nontrivial ways) I'm currently looking at an overall approach like this:
- At least for now, Charts provides a low-level internal-only API for composing charts from raw datasets.
- This is exposed to users through pre-built `SearchEngine`-like interfaces that provide a small number of more manageable controls (show chart from date X to date Y, show projects A, B, C), but not the full set of composition features (`compose(scale(2), cos())` and such).
- Eventually, we may put more UI on the raw chart composition stuff and let you build your own fully custom charts by gluing together datasets and functions.
- Or we may add this stuff in piecemeal to the higher-level UI as tools like "add goal line" or "add trend line" or whatever.
This will let the low-level API mature/evolve a bit before users get hold of it directly, if they ever do. Most requests today are likely satisfiable with a small number of chart engines plus raw API data access, so maybe UI access to flexible charting is far away.
Step toward this by adding a "Reports" section to projects. For now, this just renders a basic burnup for the current project. Followups will add an "Engine" layer above this and make the chart it produces more useful.
Test Plan: {F6426984}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20495
Summary:
Depends on D20488. Ref T13279. When installs run `bin/phd start`, start the fact daemon alongside other daemons.
Since "Reports" in Maniphest now relies on Facts data, populate it.
Test Plan: Ran `bin/phd start`, saw the Fact daemon start.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20489
Summary: Depends on D20487. If you `min(1, 2, null)`, you get `null`. We want `1`.
Test Plan: Viewed a "burnup for project X" chart where one dataseries had no datapoints. Saw a sensible domain selected automatically.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Differential Revision: https://secure.phabricator.com/D20488
Summary:
Depends on D20485. Ref T13279. This removes the ad-hoc charting in Maniphest and replaces it with a Facts-based chart.
(To do this, we build a dashboard panel inline and render it.)
Test Plan: {F6412720}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20486
Summary:
Depends on D20484. Ref T13279. Allows a chart to render as a panel.
Configuring these is currently quite low-level (you have to manually copy/paste a chart key in), but works well enough.
Test Plan: {F6412708}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20485
Summary:
Ref T13279. This changes the chart controller:
- if we have no arguments, build a demo chart and redirect to it;
- otherwise, load the specified chart from storage and render it.
This mostly prepares for "Chart" panels on dashboards.
Test Plan: Visited `/fact/chart/`, got redirected to a chart from storage.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20483
Summary: Depends on D20538. Ref T13291. We now recognize full source URIs, but encoding full URIs isn't super human-friendly and we can't do stuff like relative links with them. Add `{src ...}` as a way to get to this behavior that supports options and more flexible syntax.
Test Plan: {F6463607}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291
Differential Revision: https://secure.phabricator.com/D20539
Summary:
Depends on D20530. Ref T13291. When users paste links to files in Diffusion into remarkup contexts, identify them and specialize the rendering.
When the URIs are embedded with `{...}`, parse them in more detail.
This is a lead-up to a `{src ...}` rule which will use the same `View` but give users more options to customize presentation.
Test Plan: {F6463580}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291
Differential Revision: https://secure.phabricator.com/D20538
Summary:
Ref T13289. In Maniphest, you can currently search for "Owner: none()" to find tasks with no owner, but there's no way to search for "Reviewers: none()" in Differential right now.
Add support for this, since it's consistent and reasonable and doesn't seem too weird or niche.
Test Plan: Searched for "Reviewers: none()", found revisions with no reviewers. Searched for "Reviewers: alice, none()", "Reviewers: alice", and "Reviewers: <no constraint>" and got sensible results.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13289
Differential Revision: https://secure.phabricator.com/D20537
Summary: Depends on D20534. Ref T13294. Add export support so you can dump these out, print them on paper, notarize them, and store them in a box under a tree or whatever.
Test Plan: Exported transactions to a flat file, read the file.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13294
Differential Revision: https://secure.phabricator.com/D20535
Summary: Depends on D20533. Allow querying for transactions of a specific object type, so you can run queries like "Show all edits to Herald rules between date X and Y".
Test Plan: {F6463478}
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20534
Summary: Depends on D20531. Ref T13294. Enable finding raw transactions in particular date ranges or with particular authors.
Test Plan: {F6463471}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13294
Differential Revision: https://secure.phabricator.com/D20533
Summary:
Ref T13294. An install is interested in a way to easily answer audit-focused questions like "what edits were made to any Herald rule in Q1 2019?".
We can answer this kind of question with a more granular version of feed that focuses on being exhaustive rather than being human-readable.
This starts a rough version of it and deals with the two major tricky pieces: transactions are in a lot of different tables; and paging across them is not trivial.
To solve "lots of tables", we just query every table. There's a little bit of sleight-of-hand to get this working, but nothing too awful.
To solve "paging is hard", we order by "<dateCreated, phid>". The "phid" part of this order doesn't have much meaning, but it lets us put every transaction in a single, stable, global order and identify a place in that ordering given only one transaction PHID.
Test Plan: {F6463076}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13294
Differential Revision: https://secure.phabricator.com/D20531
Summary:
See PHI1268. We currently do some weird width handling when rendering Diffusion readmes in a document directory view.
I think this came from D12330, which used `PHUIDocumentViewPro` to change the font, but we later reverted the font and were left with the `DocumentView`. Other changes after that modified `DocumentView` to have fixed-width behavior, but it doesn't make much sense here since the content panel is clearly rendered full-width.
Today, the `DocumentView` is a more structural element with methods like `setCurtain()`. Just get rid of it to simplify things, at least as a first step.
Test Plan:
Before:
{F6463493}
After:
{F6463492}
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20536
Summary:
Depends on D20528. Ref T13291. Ref T13285. Currently, we don't put a timeout on external service calls when enriching URIs for external Asana/JIRA tasks.
Add a 15-second timeout so we'll do something reasonable-ish in the face of a downed service provider. Later, I plan to healthcheck Asana/JIRA providers in a generic way (see T13287) so we can stop making calls if they time out / fail too frequently.
Test Plan:
- Linked to JIRA and Asana tasks in comments.
- Set timeout to 0.0001 seconds, saw requests time out.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291, T13285
Differential Revision: https://secure.phabricator.com/D20530
Summary:
Depends on D20527. Ref T13291. Now that we have more flexible support for URI rewriting, use it for Doorkeeper URIs.
These are used when you set up Asana or JIRA and include the URI to an Asana task or a JIRA issue in a comment.
Test Plan:
- Linked up to Asana and JIRA.
- Put Asana and JIRA URIs in comments.
- Saw the UI update to pull task titles from Asana / JIRA using my OAuth credentials.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291
Differential Revision: https://secure.phabricator.com/D20528
Summary:
Ref T13291. Currently, `T123` is a mention and adds an "alice mentioned this on Txxx." to `T123`, but `https://install.com/T123` is not a mention.
Make the full URI a mention.
Test Plan: Commented a full URI, saw the target object get a mention story in its timeline.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291
Differential Revision: https://secure.phabricator.com/D20527
Summary:
See <https://phabricator.wikimedia.org/T179591>. Some time ago, all handle rendering preloaded handles: things emitted a list of PHIDs they'd need handles for, then later used only those PHIDs.
Later, we introduced `HandlePool` and lazy/on-demand handle loading. Modern transactions mostly use this to render object PHIDs.
When we build mail, many newer transactions use an on-demand load to fetch handles to render transactions. This on-demand load may use the original viewer (the acting user) instead of the correct viewer (the mail recipient): we fetch and reset handles using the correct viewer, but do not overwrite the active viewer for on-demand loading. This could cause mail to leak the titles of related objects to users who don't have permission to see them.
Instead, just reload the transactions with the correct viewer when building mail instead of playing a bunch of `setViewer()` and `clone` games. Until we're 100% on modular transactions, several pieces of the stack cache viewer or state information.
Test Plan:
- Created task A (public) with subtask B (private).
- Closed subtask B as a user with access to it.
- Viewed mail sent to subscribers of task A who can not see subtask B.
- Before change: mail discloses title of subtask B.
- After change: mail properly labels subtask B as "Restricted Task".
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20525
Summary:
Depends on D20446. Currently, chart functions are both configured through arguments and evaluated through arguments. This sort of conflates things and makes some logic more difficult than it should be.
Instead:
- Function arguments are used to configure function behavior. For example, `scale(2)` configures a function which does `f(x) => 2 * x`.
- Evaluation is now separate, after configuration.
We can get rid of "sourceFunction" (which was basically marking one argument as "this is the thing that gets piped in" in a weird magical way) and "canEvaluate()" and "impulse".
Sequences of functions are achieved with `compose(u, v, w)`, which configures a function `f(x) => w(v(u(x)))` (note order is left-to right, like piping `x | u | v | w` to produce `y`).
The new flow is:
- Every chartable function is `compose(...)` at top level, and composes one or more functions. `compose(x)` is longhand for `id(x)`. This just gives us a root/anchor node.
- Figure out a domain, through various means.
- Ask the function for a list of good input X values in that domain. This lets function chains which include a "fact" with distinct datapoints tell us that we should evaluate those datapoints.
- Pipe those X values through the function.
- We get Y values out.
- Draw those points.
Also:
- Adds `accumluate()`.
- Adds `sum()`, which is now easy to implement.
- Adds `compose()`.
- All functions can now always evaluate everywhere, they just return `null` if they are not defined at a given X.
- Adds repeatable arguments for `compose(f, g, ...)` and `sum(f, g, ...)`.
Test Plan: {F6409890}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Differential Revision: https://secure.phabricator.com/D20454
Summary: Ref T13272. Since the move to EditEngine, these methods have no callsites.
Test Plan: `grep`
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20484
Summary:
Ref T13283. See PHI1202. See D20519. When we apply a group of transactions, label all of them with the same "group ID".
This allows other things, notably Herald, to figure out which transactions applied together in a faithful way rather than by guessing, even though the guess was probably pretty good most of the time.
Also expose this to `transaction.search` in case callers want to do something similar. They get a list of transaction IDs from webhooks already anyway, but some callers use `transaction.search` outside of webhooks and this information may be useful.
Test Plan:
- Ran Herald Test Console, saw faithful selection of recent transactions.
- Changed hard limit from 1000 to 1, saw exception. Users should be very hard-pressed to hit this normally (they'd have to add 990-ish custom fields, then edit every field at once, I think) so I'm just fataling rather than processing some subset of the transaction set.
- Called `transaction.search`, saw group ID information available.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13283
Differential Revision: https://secure.phabricator.com/D20524
Summary:
See PHI1210. For certain large inputs, we spend more time than we need to replacing tabs with spaces. Add some fast paths:
- When a line only has tabs at the beginning of the line, we don't need to do as much work parsing the rest of the line.
- When a line has no unicode characters, we don't need to vectorize it to get the right result.
Test Plan:
- Added test coverage.
- Profiled this, got a ~60x performance increase on a 36,000 line 3MB text file.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20477
Summary:
See PHI1225. Ref T13277. In Diffusion, show "default", "permanent", or "not permanent" when looking at branches.
For repositories with 100 or fewer branches, put default and permanent branches on top.
Test Plan: {F6426814}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: leoluk
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20493
Summary:
Depends on D20507. See PHI1232. Previously, see T13255 and D20209.
Since nothing seems to have exploded after "projects" was exposed, give "subscribers" the same treatment.
Test Plan: Added, removed, and modified subscribers. Queried transactions with "transaction.search", saw sensible "type" and data.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20508
Summary:
Depends on D20510. Ref T5378. When remarkup includes a hyperlink to the current install in the form "/X123" (which is common), load the corresponding object and specialize the rendering.
This doesn't cover everything (notably, no handling for Diffusion paths yet), but does cover a lot of the most common cases.
The "uri" form preserves the URI as written, but adds an icon, tag, and hovercard.
The "{uri}" form is more similar to `{T123}` and shows the object name.
Test Plan: {F6440367}
Reviewers: amckinley, joshuaspence
Reviewed By: joshuaspence
Subscribers: joshuaspence
Maniphest Tasks: T5378
Differential Revision: https://secure.phabricator.com/D20512
Summary:
Depends on D20509. See PHI1224. Ref T5378. With some frequency, I paste URIs into the global search input (I am dumb).
When I do this dumb thing, redirect to the URI as though the global search was a URI bar.
Maybe only I am dumb like this, but I don't think it'll hurt anything.
Test Plan: pasted a URI and hit return; tried to eat a rock
Reviewers: amckinley, joshuaspence
Reviewed By: joshuaspence
Maniphest Tasks: T5378
Differential Revision: https://secure.phabricator.com/D20510
Summary: Ref T5378. This class was renamed more than a year ago, in D19087. Remove the leftover compatiblity layer.
Test Plan: `grep`
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T5378
Differential Revision: https://secure.phabricator.com/D20509
Summary:
See PHI1232, which describes a reasonable use case for wanting information about the "draft" ("Hold as Draft / Do Not Auto-Promote") flag.
Also, flesh out "testPlan" and "summary". It's possible these "blob of remarkup" fields might have metadata some day (e.g., a rendered version or a list of PHIDs or something), but we could add more keys, and we already have some other transactions which work like this.
Test Plan: Used "transaction.search" to fetch these transaction types, saw type information and metadata.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20507
Summary:
See PHI1229. An install has a somewhat duct-taped registration flow which can dump users on the "Wait for Approval" screen without clear guidance. The desired guidance is something like "this is totally normal, just wait a bit for a bot to approve you".
Adding guidance here is generally reasonable and consistent with the intent of this feature.
Test Plan: {F6426583}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: kylec
Differential Revision: https://secure.phabricator.com/D20492
Summary: Depends on D20519. Ref T13283. See PHI1202. Add a new rule which triggers when the current/most-recent transaction group includes a "content" or "publish" transaction, which means the published document content has changed.
Test Plan:
- Wrote a Herald rule using this field.
- Created a document (rule matched).
- Edited a document (rule matched).
- Edited a document, saving as a draft (no match).
- Edited a draft, updating it (no match).
- Published a draft docuemnt (rule matched).
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13283
Differential Revision: https://secure.phabricator.com/D20520
Summary:
Depends on D20518. Ref T13283. When you use the "Test Console" in Herald to test rules, pass the most recent group of transactions to the Adapter.
This will make it easier to test rules that depend on edit state, since you can make the type of edit you're trying to test and then use the Test Console to actually test if it matches in the way you expect.
The transactions we select may not be exactly the group of transactions that most recently applied. For example, if you make edits A, B, and C in rapid succession and then run the Test Console on the object, it may select "A + B + C" as a transaction group. But we'll show you what we selected and this is basically sane/reasonable and should be fine.
(Eventually, we could include a separate "transaction group ID" on transactions if we want to get this selection to match exactly.)
Test Plan: Ran the Test Console on various objects, saw sensible transaction lists in the transcripts.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13283
Differential Revision: https://secure.phabricator.com/D20519
Summary:
Ref T13283. Since D14575, we already pass applied transactions to Herald, but they exist only as a backwards compatibility layer and have no upstream callsites.
Save the applied transaction PHIDs as part of the object transcript, and show them in the UI.
Test Plan:
- Viewed a modern transcript, saw a list of transactions.
- Viewed an older transcript, saw nothing (since there were no transactions in the transcript).
{F6456431}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13283
Differential Revision: https://secure.phabricator.com/D20518
Summary:
Depends on D20516. See PHI1247. In D20331, I made the crumbs on workboards point at ancestor workboards.
However, this isn't a great destination if an ancestor doesn't actually have a workboard. In this case, point at the normal profile URI instead.
Test Plan:
- Viewed a milestone workboard with a parent that had no workboard. Saw a profile link instead of a workboard link (new behavior).
- Viewed a milestone workboard with a parent that also had a workboard. Saw a workboard link (existing old behavior still works).
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20517
Summary:
Ref T13276. Previously, these edges were added directly with an `EdgeEditor`, so they did not generate transaction stories.
Now, they're added properly, but they aren't terribly useful in feed/mail. Hide them in those contexts, like we already do with other types of similar edges.
Test Plan: Will verify behavior on `secure`.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20491
Summary:
See PHI1209. When a task is in "Hard Lock" mode, it's still possible to apply some changes to it. Notably:
- You can subscribe/unsubscribe.
- You can mention it on another object.
- You can add a relationship from some other object to it (e.g., select it as a "Parent Task" for some other task).
Currently, these types of edits will show a "Lock Overridden" timeline emblem icon. However, they should not: you didn't override a lock to make these changes, they just bypass locks.
For now, special case these cases (self subscribe/unsubscribe + inverse edge edits) so they don't get the little icon, since I think this list is exhaustive today.
Some day we should modularize this, but we'd need code like this anyway (since TYPE_SUBSCRIBE is not modular yet), and this seems unlikely to cause problems even if it's a bit rough.
Test Plan:
- Hard-locked a task.
- Subscribed/unsubscribed, mentioned, relationship'd it as a non-author. No timeline emblems.
- Soft-locked a task.
- Subscribed/unsubscribed, mentioned, relationship'd it, no timeline emblems.
- Clicked "Edit", answered "yes" to the override prompt, edited it. Got a timeline emblem.
- Added some comments and stuff to a normal non-locked task, no emblems.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20513
Summary:
See PHI1247. If you remove the Workboard from a project profile menu, then navigate to the URI, we currently fatal when trying to select the "Workboard" item.
Instead, only try to select the item if some matching item is present.
Test Plan:
- Disabled the workboard on a project, navigated to `/board/`, got a sensible page with no navigation item selected instead of a fatal.
- Viewed a normal workboard, saw the correct selection.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20516
Summary: Ref PHI1166. I'm documenting our dependencies, and we have approximately 5,000 lines of external code to support WePay as a Phortune provider. We don't use it, I'm almost certain it doesn't work, and we have no plans to use it in the near future. If we did pursue it, I'd probably just wrap the API in a 100-line `WePayFuture` anyway since 5K lines of dependencies to make a couple method calls is ridiculous.
Test Plan: Grepped for `wepay`, `httpful`, `restful`.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: aurelijus
Differential Revision: https://secure.phabricator.com/D20521
Summary: Ref T13276. This edge is pointed the wrong way. Point it the right way.
Test Plan: Will verify production works better.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20490
Summary: Depends on D20474. Ref T13272. Provide an easy way to rearrange tabs on a tab panel, by moving them left or right from the context menu.
Test Plan: Moved tabs left and right. Tried to move them off the end of the tab list, no luck.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20475
Summary:
Depends on D20473. Ref T13272. Fixes T7216. If you want to tweak the query a panel uses, you currently have to complete 7 Great Labors.
Instead, add a "Customize Query" action which lets you update the query inline.
Test Plan: {F6402171}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272, T7216
Differential Revision: https://secure.phabricator.com/D20474
Summary:
Depends on D20469. Ref T13276. See PHI1159. See PHI953. See PHI901.
Allow Herald to detect when "arc land" would (or did) warn users about failed or ongoing builds. This respects the "Warn on Landing" build plan behavior.
To accomplish this:
- When we close a revision, set a "wrong build state" flag if it lands in the wrong build state.
- If the revision is closed when we hit Herald, look for the flag.
- If not (common for push rules, can happen for commit rules if we race against the revision update worker), hit Harbormaster ourselves and check the current state.
Test Plan:
- Wrote a "Require Green" rule.
- Ran it against various commits with various build states (good, not good).
- Fiddled with "Warn on Landing" and saw the effect in rule evaluation.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20470
Summary:
Depends on D20468. Ref T13276. See PHI1008.
When a commit or revision "reverts <hash>", and that's the hash of a commit which has a revision, also write a "reverts" edge to the revision.
Test Plan:
Created "reverts X" objects for:
- a revision;
- a commit;
- a commit with a revision (both got marked properly).
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20469
Summary:
Depends on D20467. Ref T13277. Currently, the "MessageParserWorker" writes this property on commits, then Herald and Audit both read it.
Make them share code so this property has one writer and one reader. This property isn't great, but at least now the badness is hidden.
Currently, we can't just use edges because they may not have been written yet. I am likely to just do this, soon:
- Just write the edges (in "MessageParserWorker").
- Hide the edges from mail.
However, we'll sort-of lose the "revisionMatchData" explanation thing if I do that. Maybe this is fine? But when commits match because hashes match, it legitimately isn't obvious.
For now, just reduce the amount of harm/badness here.
Test Plan:
- Ran `bin/repository reparse --publish ...`.
- Ran a Herald "Audit" rule using the "Accepted Differential revision" field.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20468
Summary: See PHI1220. Ref T13272. I accidentally left the ability to set a query limit behind when updating this.
Test Plan: Edited a query panel, set/removed the limit, tried to set an invalid limit.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20472
Summary:
Depends on D20472. Ref T13272. Currently, when you edit a panel from a dashboard, we try to add the panel to the dashboard. This always works since dashboards no longer enforce panel uniqueness, and you can end up with duplicate panels.
Instead, only add panels if we're creating them.
Test Plan:
- Edited an existing panel, no duplication.
- Created a new panel, saw it added to the dashboard.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20473
Summary:
If you edit an auth message in Auth > Customize Messages, then click "Show Details" in the transaction record, the resulting dialog uses the object's handle's URI to generate a "cancel" button.
Since these handles currently have no URI, the dialog currently has no cancel/done button to close it.
Test Plan: Edited an auth message, clicked "Show Details", was now able to click "Done" to close the dialog.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20471
Summary:
See PHI985. I think we pretty much need to start applying language-specific rules, but we can apply at least one more relatively language-agnostic rule: don't match lines which are indented 3+ levels.
In C++, we may have symbols like this:
```
class X {
public:
int m() { ... }
}
```
..but I believe no mainstream language puts symbol definitions 3+ levels deep.
Also clean up some of the tab handling very slightly.
Test Plan: Tests pass, looked at some C++ code and got slightly better (but still not great) matches.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20479
Summary:
See PHI1218. When rendering "A vs B", we currently show the properties of diff A without modification.
Instead, take properties from the same place we're taking change details.
See T12664 for a followup.
Test Plan:
- In diff A, removed "+x" from a file.
- In diff B, changed the file but did not remove "+x" from it.
- Diffed B vs A.
- Before change: UI incorrectly shows "+x" removed (both sides incorrect, just showing the change from diff A).
- After change: UI shows 100644 -> null, which is half right.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20478
Summary:
See <https://discourse.phabricator-community.org/t/unable-to-reload-object-that-hasnt-been-loaded/2677>.
When editing "Config" objects, they currently get a PHID set outside of the TransactionEditor. They probably should not, but fixing that is likely an involved change.
This causes us to incorrectly fail to detect `$is_new` correctly and try to `reload()` and object with no ID.
To work around this, test for new objects with `getID()` instead of `getPHID()`.
Test Plan: Edited any config value with the web UI.
Reviewers: amckinley
Differential Revision: https://secure.phabricator.com/D20482
Summary:
Depends on D20466. Ref T13277. Currently:
- The "Owners" worker writes ownership relationships (e.g., commit X affects package Y, because it touches a path in package Y) -- these are just edges.
- It also triggers audits.
- Then it queues a "Herald" worker.
- This formally publishes the commit and triggers Herald.
These aren't really separate steps and can happen more easily in one shot. Merge them.
Test Plan:
- Ran `bin/repository reparse --publish` to republish various commits, got sensible behavior.
- Grepped for "IMPORTED_OWNERS", "IMPORTED_HERALD", "--herald", "--owners", and "--force-local" flags.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20467
Summary:
Depends on D20465. Ref T13277. Currently, when a commit is unpublished, we put a single line about it on the "Edit Commit" page. This is pretty much impossible to find.
Move it to the main page. This treatment is more big/bold than I'd probably like to end up, but we should probably overshoot on the explanatory text until users get used to this behavior.
Also, allow searching for only published / unpublished commits.
Test Plan: {F6395705}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20466
Summary:
Depends on D20464. Ref T13277. Broadly:
- Move all the "should publish X" and "why aren't we publishing X" stuff to a separate class (`PhabricatorRepositoryPublisher`).
- Rename things to be more consistent with modern terminology ("Publish", "Permanent Refs").
Test Plan:
This could use some trial-by-fire on `secure`, but:
- Grepped for all symbols.
- Viewed various commits.
- Reparsed commits.
- Here's a commit with an explanation:
{F6394569}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20465
Summary: Depends on D20463. Ref T13277. This flag was added some time before 2015 and I don't think I've ever used it. Just get rid of it.
Test Plan: Grepped for `force-autoclose`, `forceAutoclose`, `AUTOCLOSE_FORCED`.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20464
Summary:
Depends on D20462. Ref T13276. Currently, the "Message" parser also updates related tasks and revisions when a commit is published.
For PHI1165, which ran into a race with message parsing, I originally believed we needed to separate this logic and lock + yield to avoid the race. D20462 provides what is probably a better approach for avoiding the race.
Still, I think separating these "update related revisions" and "updated related tasks" chunks into separate workers is a net improvement. There may still be some value in doing lock + yield in the future to deal with other issues, and when we occasionally run into problems with pulling a diff out of the repository to update the revision (usually because the diff is too big) this isolates the problem better and allows the commit to import.
I think the only thing to watch out for here is that Herald may now run before the revision and commit are attached to one another. This is fine for all current Herald rules, we just need to be mindful in implementing new rules.
Test Plan: Used `bin/repository reparse --message` on various commits, including commits that close revisions and close tasks.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20463
Summary:
Depends on D20461. Ref T13276. Ref T13054.
Currently, we acquire the transaction read lock after populating "old values" in transactions and filtering transactions with no effect.
This isn't early enough to prevent all weird chaotic races: if two processes try to apply a "close revision" transaction at the same time, this can happen:
```
PROCESS A PROCESS B
Old Value = Open Old Value = Open
Transaction OK: Yes Transaction OK: Yes
Acquire Read Lock Acquire Read Lock
Got Read Lock! Wait...
Apply Transactions Wait...
New Value = Closed Wait...
Release Lock Wait...
Got Read Lock!
Apply Transactions
New Value = Closed
Release Lock
```
That's not great: both processes apply an "Open -> Closed" transaction since this was a valid transaction from the viewpoint of each process when it did the checks.
We actually want this:
```
PROCESS A PROCESS B
Acquire Read Lock Acquire Read Lock
Got Read Lock! Wait...
Old Value = Open Wait...
Transaction OK: Yes Wait...
Apply Transactions Wait...
New Value = Closed Wait...
Release Lock Wait...
Got Read Lock!
>>> Old Value = Closed
>>> Transaction Has No Effect!
>>> Do Nothing / Abort
Release Lock
```
Move the "lock" part up so we do that.
This may cause some kind of weird second-order effects, but T13054 went through pretty cleanly and we have to do this to get correct behavior, so we can survive those if/when they arise.
Test Plan:
- Added a `sleep(10)` before the lock.
- Ran `bin/repository message --reparse X` in two console windows, where X is a commit that closes revision Y and Y is open.
- Before patch: both windows closed the revision and added duplicate transactions.
- After patch: only one of the processes had an effect.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: jmeador
Maniphest Tasks: T13276, T13054
Differential Revision: https://secure.phabricator.com/D20462
Summary: Depends on D20459. Ref T13276. I'll file a followup to actually destroy the table.
Test Plan:
- Grepped for `TABLE_COMMIT`.
- Ran `bin/storage upgrade -f`, got a clean bill of health.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20461
Summary:
Depends on D20458. Ref T13276. Although I'm not thrilled about "needCommitPHIDs()", it has a few callers, including custom fields. Allow "need + attach + get" to survive for now since they're reasonably modern, at least.
However, use edges instead of "TABLE_COMMIT" and require `need...()` + `get...()`, removing the direct `load...()`.
Also remove `RevisionQuery->withCommitPHIDs(...)`, which has no callers.
Test Plan:
- Grepped for `loadCommitPHIDs` (only two hits, the private `RevisionQuery` method).
- Called "differential.getrevision", got commits.
- Viewed a revision, saw "Commits: ...".
- Grepped for `withCommitPHIDs()`, no callers on `RevisionQuery` (some other query classes have methods with this name).
- Called "differential.query", got commits.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20459
Summary: Depends on D20457. Ref T13276. Kill all remaining callers to this method and delete it.
Test Plan:
- Grepped for `loadIDsByCommitPHIDs`.
- Viewed blame again to make sure I didn't break it.
- Viewed "History" view for commits with revisions.
- Viewed "Graph" view for commits with revisions.
- Viewed "Merged Commits" table for commits with revisions.
- Viewed "Compare" table for commits with revisions.
- Viewed "Repository" main page history table for commits with revisions.
- Grepped for `linkRevision`.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20458
Summary:
Ref T13276. Differential has a pre-edge "TABLE_COMMIT" with about a half-dozen weird callers I'd like to get rid of.
Move blame to use edges instead. (Bonus: this makes blame respect edge edits in the UI.)
Since there are some more callers to clean up this code may move into some "RelatedObjectQueryThing" class or something, but I'm taking it one step at a time for now.
Test Plan: {F6394106}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276
Differential Revision: https://secure.phabricator.com/D20457
Summary:
Depends on D20445. Ref T13279. I'm not sure what the class tree of functions actually looks like, and I suspect it isn't really a tree, so I'm hesitant to start subclassing. Instead, try adding some `isSomethingSomething()` methods.
We have some different types of functions:
# Some functions can be evaluated anywhere, like "constant(3)", which always evaluates to 3.
# Some functions can't be evaluated anywhere, but have values everywhere in some domain. This is most interesting functions, like "number of open tasks". These functions also usually have a distinct set of interesting points, and are constant between those points (any count of anything, like "open points in project" or "tasks closed by alice", etc).
# Some functions can be evaluated almost nowhere and have only discrete values. This is most of the data we actually store, which is just "+1" when a task is opened and "-1" when a task is closed.
Soon, I'd like to be able to show ("all tasks" - "open tasks") and draw a chart of closed tasks. This is somewhat tricky because the two datasets are of the second class of function (straight lines connecting dots) but their "interesting" x values won't be the same (users don't open and close tasks every second, or at the same time).
The "subtract X Y" function will need to be able to know that `subtract "all tasks" 3` and `subtract "all tasks" "closed tasks"` evaluate slightly differently.
To make this worse, the data we actually //store// is of the third class of function (just the "derivative" of the line chart), then we accumulate it in the application after we pull it out of the database. So the code will need to know that `subtract "derivative of all tasks" "derivative of closed tasks"` is meaningless, or the UI needs to make that clear, or it needs to interpret it to mean "accumulate the derivative into a line first".
Anyway, I'll sort that out in future changes. For now, simplify the easy case of functions in class (1), where they're just actual functions.
Add "shift(function, number)" and "scale(function, number)". These are probably like "mul" and "add" but they can't take two functions -- the second value must always be a constant. Maybe these will go away in the future and become `add(function, constant(3))` or something?
Test Plan: {F6382885}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20446
Summary:
Depends on D20444. Ref T13279. Instead of ad-hoc parsing and messages, formalize chart function arguments.
Also, add a whole lot of extra type checking.
Test Plan: Built and charted various functions with various valid and invalid argument lists, got sensible-seeming errors and results.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20445
Summary:
See PHI1213. If we don't have identities for "revision X closed by commit Y" stories, just do the plain non-attribution rendering rather than trying to fall back. Falling back won't work since we don't load the data, which should be OK now since identities seem like they're in generally good shape.
(We could probably just throw out the fallback behavior instead, but we can always clean things up later.)
Test Plan: Forced no commit identity on a revision, loaded it, saw reasonable story rendering.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20460
Summary:
Ref T13266. See <https://discourse.phabricator-community.org/t/notification-page-throws-unrecoverable-fatal-error/2651/>.
The "notifications" query currently uses offset paging for no apparent reason (just a legacy issue?), so some of the paging code is only reachable internally.
- Stop it from using offset paging, since modern cursor paging is fine here (and Feed has used cursor paging for a long time).
- Fix the non-offset paging to work like Feed.
Also:
- Remove a couple of stub methods with no callsites after cursor refactoring.
Test Plan:
- Set things up so I had more than 100 notifications and some in the first 100 were policy filtered, to reproduce the issue (I just made `FeedStory` return `NO_ONE` as a visibility policy).
- Applied the patch, notifications now page cleanly.
- Verified that "Next Page" took me to the right place in the result list.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: hskiba
Maniphest Tasks: T13266
Differential Revision: https://secure.phabricator.com/D20455
Summary:
Ref T13280. In D20421, I changed our observe strategy to `git fetch <uri>` in all cases.
This doesn't work in an ancient, non-bare repository if `master` is checked out and `master` is also fetch: `git` refuses to overwrite the local ref unless we pass `--update-head-ok`. Pass this flag.
Also, remove some code which examines branches and tags in a special way for non-bare working copies. The old `git fetch <origin>` code without explicit revsets meant that `refs/remotes/orgin/heads/xyz` got updated instead of `refs/heads/xyz`. We now update our local refs in all cases (bare and non-bare) so we can throw away this special casing.
Test Plan:
- Replaced a modern bare working copy with a non-bare working copy by explicitly using `git clone` without `--bare`.
- Ran `bin/repository update`, hit the `--update-head-ok` error. Applied the patch, got a clean update.
- Used the "repository.branchquery" API method...
- ...with "contains" to trigger the "git branch" case. Got identical results after removing the special casing.
- ...without "contains" to trigger the "low level ref" case. Got identical results after removing the special casing.
- Grepped for `isWorkingCopyBare()`. The only remaining callsites deal with hook paths, and genuinely need to be specialized.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13280
Differential Revision: https://secure.phabricator.com/D20450
Summary:
Ref T13279. Charting changes alter how the "line-chart" behavior works, but the "Burnup Chart" still relies on the old behavior.
Although I'm intending to remove "Maniphest > Reports" once Facts is a minimally sufficient replacement, copy this behavior to keep it working until we're ready to pull the trigger.
Also fix a leftover typo from D20435.
Test Plan: Viewed a legacy Maniphest burnup rate report.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20449
Summary: This always bugs me when I'm going through `secure` looking for the spiciest macros.
Test Plan: Forced a date to be null, saw reasonable text.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D20453
Summary:
Depends on D20443. Ref T13279. This is probably not terribly useful on its own, but is mostly a function which takes another function as an argument, and a step toward more useful functions like arithmetic and drawing a picture of an owl.
The only structural change here is that functions now read data parameters (domain, sample limit) using a more tailored "ChartDataQuery" instead of reading the actual axis. Mostly, I want a more cohesive representation of query state that can be easily passed to sub-functions, as here.
Test Plan: {F6382432}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20444
Summary:
Depends on D20442. Ref T13279. Add basic support for drawing chart functions that are not based on Facts first-party ETL datasets. Some general goals:
- This might be useful to draw a line like "goal" or "profitability".
- This might be useful to pull data from an external source.
- For composable functions like "add" or "subtract", which are useful in manipulating ETL datasets, these value functions will make testing easier.
Test Plan:
Added a `constant(256)` function:
{F6382408}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20443
Summary:
Depends on D20441. Ref T13279. Currently, we pull all the data, then decide what the X-axis should look like.
Since users will reasonably want to do stuff like "show me march-april 2018" in the future, we need to move toward flipping this around so that we can support cases where the domain is specified by the user.
For actual chart functions (like "constant(3)" or "cos(x)"), we must also know the domain before we pull data, since there are an infinite number of places where we can evaluate the function "constant(3)".
See note in T13279 about continunity.
Test Plan: {F6382356}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20442
Summary:
Depends on D20440. Ref T13279. Create a class to represent a chartable function: something we can get some data points out of.
Then, make the chart chart two functions.
For now, the only supported function is "fact(key)", which pulls data from the Facts ETL pipeline, identified by "key", and takes no other arguments.
In future changes, I plan to support things like "fact(tasks.open.project, PHID-PROJ-xyz)", "constant(1000)" (e.g. to draw a goal line), "sum(fact(...), fact(...))" (to combine data from several projects), and so on.
The UI may not expose this level of power for a while (or maybe ever) but until we get close enough to the UI that these features are a ton of extra work I'm going to try to keep things fairly flexible/modular.
Test Plan: {F6382286}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20441
Summary:
Depends on D20439. Ref T13279. Some day, charts will probably need to reload themselves or do a bunch of defer/request-shaping magic when they're on a dashboard with 900 other charts.
Give the controller separate "HTML placeholder" and "actual data" modes, and make the placeholder fetch the data in a separate request.
Then, make the chart redraw if you resize the window instead of staying at whatever size it started as.
Test Plan:
- Loaded a chart, saw it load data asynchronously.
- Resized the window, saw the chart resize.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20440
Summary:
Ref T13279. I think I'm going to fling some stuff at the wall for a bit here and hope most of it sticks, so this series of changes may not be terribly cohesive or focused. Here:
The range of the chart is locked to "[0, 105% of max]". This is trying to make a pleasing extra margin above the maximum value, but currently just breaks charts with negative values. Later:
- I'll probably let users customize this.
- We should likely select 0 as the automatic minimum for charts with no negative values.
- For charts with positive values, it would be nice to automatically pick a pleasantly round number (25, 100, 1000) as a maximum by default.
We don't have any storage for charts yet. Add some. This works like queries, where every possible configuration gets a short URL slug. Nothing writes or reads this yet.
Rename `fn()` to `css_function()`. This builds CSS functions for D3. The JS is likely to get substantial structural rewrites later on, `fn()` was just particularly offensive.
Test Plan: Viewed a fact series with negative values. Ran `bin/storage upgrade`.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13279
Differential Revision: https://secure.phabricator.com/D20438
Summary:
Depends on D20434. Fixes T5963. Broadly, the issue here is that when:
- You create a new, empty repository.
- Then, you work on some branch other than `master`, without ever creating `master`.
...you get a warning on `git clone`:
> warning: remote HEAD refers to nonexistent ref, unable to checkout
To fix this, point the symbolic-ref HEAD at `refs/heads/<default-branch>` after installing commit hooks.
This fixes the warning, and also means that `git clone` will check out the repository default branch by default, which is nice.
There are a few caveats about this behavior (see T5963 for discussion) but nothing too substantial.
The only real issue is that Git prevents deletion of the default branch without a config setting. Just set that settting.
Test Plan:
See T5963.
In a repository, set `HEAD` to point somewhere invalid. Ran `bin/repository update ...`. Saw HEAD pointed back at the repository default branch.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T5963
Differential Revision: https://secure.phabricator.com/D20435
Summary:
Depends on D20433. Ref T13277. Since "Autoclose" no longer exists, update the documentation.
Currently, this documentation focuses a lot on troubleshooting because users historically had a lot of trouble with figuring out why things were or were not autoclosing. I haven't seen any real confusion about this in years, so I suspect we may have improved the import pipeline and/or UI to make this less of a problem.
It's also possible that this document "fixed" the problem, but usually I expect a documentation fix to not affect the frequency of reports, just make them easier to resolve, so I doubt it.
If unclear things remain //and// documentation really did fix it, maybe we can fix the issues. Or we can just put the troubleshooting documentation back.
Test Plan: Read documentation.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20434
Summary:
Depends on D20432. Ref T13277. Fixes T12967. Removes some "Track Only" hints and warns that the feature is deprecated in favor of "Permanent Refs" and "Fetch Only".
(This "fixes" T12967 by mooting it.)
Test Plan: Viewed "branches" sectino of the manage UI, edited "braches" section of a repository.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277, T12967
Differential Revision: https://secure.phabricator.com/D20433
Summary:
Depends on D20427. Ref T13277. As an optimization, when we discover that a commit which was previously only on a non-permanent ref ("tmp-epriestley-123") is now reachable from a permanent ref ("master"), we currently queue only a new "message" parse step.
This is an optimization because these commits previously got the full treatment (feed, publish, audit, etc) as soon as they were discovered. Now, those steps only happen once a commit is reachable from a permanent ref, so we need to run everything.
Test Plan:
- Pushed local "tmp-123" branch to remote tag "tmp-123".
- Updated repository with "bin/repository update", saw commit import as a "not on any permanent ref" commit, with no herald/audit/etc.
- Merged "tmp-123" tag into "master".
- Pushed new "master".
- Updated repository with "bin/repository refs ... --trace --verbose", saw commit detected as now reachable from a permament ref.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20428
Summary:
Depends on D20426. Ref T13277. The new behavior is to fire Herald only once a commit becomes reachable from a permanent ref (previously, an "Autoclose" branch).
That means that every "Commit" Herald rule implicitly has this field as a condition, and it no longer does anything.
Test Plan: Wrote a Herald rule, saw this as an option in the "Deprecated" section.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20427
Summary:
Depends on D20425. Ref T13277. See PHI1067. There's currently no way to retrieve branch/ref rules over the API, which makes some management operations against a large number of repositories difficult.
Expose these rules to the API.
Test Plan: Called `diffusion.repository.search`, got rules in the result set.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20426
Summary:
Depends on D20424. Ref T13277. Now that the "Actions" panel only has one item ("Publishing"), just move it to the "Basics" panel.
Update the UI to show active/publishing status more clearly and relate them to one another and importing state.
Test Plan: {F6378087}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20425
Summary:
Depends on D20423. Ref T13277. Repositories currently have separate toggles for "Autoclose" and "Publishing".
Merge the "Autoclose" toggle into the "Publishing" toggle. I'm unaware of any valid use case for enabling one but not the other.
(This doesn't fix all the documentation, yet.)
Test Plan: Edited a repository, saw only one publishing option.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20424
Summary:
Depends on D20422. Ref T13277. Currently, "track only", "publish", and "autoclose" are three separate ideas. I'd like to generally merge them into a more natural idea called "permanent refs".
Since "Autoclose" effectively now controls both "autoclose" and "publish", rename it.
This doesn't rename all the methods or internals, and the documentation needs an update, but it renames most of the UI-facing stuff.
(You also can only specify branches as "Permanent Refs" today, but we may let you specify tags and other arbitrary refs in the future.)
Test Plan: Grepped, poked around the UI, saw UI show "Permanent" / "Permanent Refs" more often and "Autoclose" less.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20423
Summary:
Depends on D20421. Ref T13277. I'd generally like to move away from "Track Only".
Some of the use cases for "Track Only" (or adjacent to "Track Only") are better resolved with "Fetch Rules" -- basically, rules to fetch only some subset of refs from the observed remote.
Add configurable "Fetch Rules" for Git repositories. For example, if you only want to fetch `master`, you can now speify:
```
refs/heads/master
```
If you only want to fetch branches and tags, you can use:
```
refs/heads/*
refs/tags/*
```
In theory, this is slightly less powerful in the general case than "Track Only", but gives us better behavior in some cases (e.g., when the remote has 50K random temporary branches). In practice, I think this and a better "Autoclose Only" will let us move away from "Track Only", get default behavior which is better aligned with what users actually expect, and dodge all the "track tags/refs" questions.
Test Plan: Configured repositories with "Fetch Refs" rules, used `bin/repository pull --verbose --trace ...` to run pulls, saw expected pull/fetch behavior.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20422
Summary:
Depends on D20420. Ref T13277. We currently spend substantial effort trying to detect and correct the URL of the "origin" remote in Git repositories.
I believe this is unnecessary, and we can always `git fetch <url> ...` to get the desired result instead of `git muck-with-origin + git fetch origin ...`. We already do this in the more recent parts of the codebase (e.g., intracluster sync) and it works correctly in every case I'm aware of.
Test Plan:
- Grepped for `origin`, ` origin `.
- Ran `bin/repository update ...` to fetch a mirrored repository.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277
Differential Revision: https://secure.phabricator.com/D20421
Summary:
Depends on D20418. Ref T13277. Fixes T11314.
Currently, when you push commits to some arbitrary ref or tag (like `refs/pull/123` on GitHub, `refs/tags/phabricator/diff/123` on Phabricator, or `refs/changes/whatever` on Gerrit), we do not "autoclose" related objects. This means that we don't process `Ref T123` to create links to tasks, and don't process `Differential Revision: xyz` to close revisions.
However, we //do// still publish these commits. "Publish" means: trigger audits, publish feed stories, and run Herald rules.
- Stop publishing these commits.
- In the UI, show these commits as "Not Permanent" with a note that they are "Not [on any permanent branch]."
These commits will publish and autoclose if they ever become reachable from an "autoclose" ref (most commonly, if they are later merged to "master").
Test Plan:
- Pushed a commit to `refs/tags/quack`.
- Before: got a feed story.
- After: no feed story, UI shows commit as "Not Permanent".
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277, T11314
Differential Revision: https://secure.phabricator.com/D20419
Summary:
Depends on D20381. Ref T8093. This makes minor improvements to the protocol proxy to handle cases where we add, remove, or replace refs and may need to move the "capabilities" section.
Rather than invoking a callback on every ref: parse the whole ref list into a data structure, mutate it if necessary (in a future diff), then dump it back into wire format.
This allows us to shift the capabilities data (which needs to be coupled with the first ref) around if we modify the first ref, and reorder the reflist alphabetically like git does.
When the server has no refs, Git sends no capabilities data. This is easy to emulate, just surprising.
Test Plan:
Tested the cases not covered by D20381:
- Fetching where the fetch actually fetches data.
- `ls-remote` when we hide the first ref (capabilities data properly moves to the first visible ref).
- `ls-remote` when the remote is empty (we just drop the capabilities frame completely).
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T8093
Differential Revision: https://secure.phabricator.com/D20436
Summary:
Depends on D20380. Ref T8093. When prototypes are enabled, inject a (hopefully?) no-op proxy into the Git wire protocol.
This proxy decodes "git upload-pack" and allows the list of references to be rewritten, in a similar way to how we already proxy the Subversion protocol to rewrite URIs and proxy the Mercurial protocol to distinguish between read and write operations.
The piece we care about comes at the beginning, and looks like this:
```
<frame-length><ref-hash> <ref-name>\0<server-capabilities>\n
<frame-length><ref-hash> <ref-name>\n
<frame-length><ref-hash> <ref-name>\n
...
<0000>
```
We can add, remove, or modify this section to make it appear that the server has different refs than the refs that exist on disk.
Things I have tried:
- `git ls-remote`
- `git ls-remote` where the server hides some refs.
- `git fetch` where the fetch is a no-op.
Things I have not tried:
- `git fetch` where the fetch is not a no-op.
- Tricking things into doing protocol v2. Or: I tried this, I wasn't successful. In v2, additional "\0" tricks are used to hide data in the capabilities, I think?
- `git ls-remote` where we rewrite/hide the first ref in the list, and need to move the capabilities frame elsewhere.
- `git ls-remote` where the server has no refs at all, or we remove every ref.
So the "interesting" piece of this works, but it almost certainly needs some cleanup to survive interaction with the real world.
Test Plan: See above.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T8093
Differential Revision: https://secure.phabricator.com/D20381
Summary:
Ref T8093. Support dumping the protocol bytes to a side channel logfile, as a precursor to parsing the protocol and rewriting protocol frames to virtualize refs.
The protocol itself is mostly ASCII text so the raw protocol bytes are pretty comprehensible.
Test Plan:
{F6363221}
{F6363222}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T8093
Differential Revision: https://secure.phabricator.com/D20380
Summary: Ref T7667. Also one text fix.
Test Plan: Ran `bin/auth lock`, didn't get a replacement setup issue.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T7667
Differential Revision: https://secure.phabricator.com/D20447
Summary:
See PHI1134. Generally, "alice added a dependent revision: ..." isn't a very interesting story. This relationship itself is valuable, but the creation of the relationship is usually pretty obvious from context.
In the specific case of PHI1134, various scripts are racing one another, but I don't think this story is of much value in the general case anyway.
Test Plan: Edited parent/child revisions, no more feed stories.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20437
Summary:
Depends on D20419. Ref T13277. Fixes T8936. Fixes T9383. Fixes T12300. When you push arbitrary refs to Phabricator, the push log currently complains if those refs are not tags or branches.
Upstream Git now features "notes", and there's no reason to prevent writes to arbitrary refs, particularly beause we plan to start using them soon (see T13278).
Allow these writes as affecting raw refs.
Test Plan:
- Pushed an arbitrary ref.
- Pushed some Git notes.
- Wrote a Herald ref rule, saw "ref" in the dropdown.
{F6376492}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13277, T8936, T9383, T12300
Differential Revision: https://secure.phabricator.com/D20420
Summary:
Depends on D20416. Ref T13269. See D20329.
If you try to save an "Assign to" rule with no assignee, we currently replace the control with an "InvalidRule" control that isn't editable. We'd prefer to give you an empty field back and let you pick a different value.
Differentiate between "bad record format" (i.e., we can't really do anything with this) and "bad record value" (i.e., everything is structurally fine, you just typed the wrong thing). In the latter case, we still build a properly typed rule for the UI, we just refuse to update storage until you fix the problem.
Test Plan:
First, hit the original issue and got a nicer UI with a more consistent control width (note full-width error):
{F6374205}
Then, applied the rest of the patch and got a normal "fix the issue" form state instead of a dead-end:
{F6374211}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13269
Differential Revision: https://secure.phabricator.com/D20417
Summary:
Ref T12164. Ref T13276. Currently, the parsing pipeline copies the author and committer names and PHIDs into the transcaction record as metadata. They are then rendered directly from the metadata.
This makes planned changes to the parsing pipeline (to prevent races when multiple commits matching a single revision are pushed simultaneously) more difficult, and generally won't work with repository identities.
Instead, load the commit and use its identities.
Test Plan: Loaded a revision, saw the same story rendering for a "Closed by commit" story.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13276, T12164
Differential Revision: https://secure.phabricator.com/D20418
Summary:
Ref T13269. See D20329. When we switch trigger rule control types, reset the rule value.
Also, pick slightly nicer defaults for status/priority.
Test Plan:
- Created a "Change Status To: X" rule.
- Saved it.
- Edited it.
- Selected "Assign to" for the existing action's dropdown.
- Before: tokenizer filled with nonsense.
- After: tokenizer cleared.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13269
Differential Revision: https://secure.phabricator.com/D20416
Summary:
Depends on D20414. Ref T13272. Several minor things here:
- Currently, you can drag panels underneath the invisible "there are no items in this column" div and the "Create Panel / Add Existing Panel" buttons. This is silly; stop it.
- Currently, when viewing a tab panel on a dashboard, you can drag the panels inside it. This is extremely silly. Make "movable" off by default and pass it through the async flow only when we actually need it.
- Make the whole "Add Tab..." virtual tab clickable to open the dropdown. This removes the rare exception/todo combo I added earlier. {key F}
- Add or remove some icons or something.
Test Plan: Moved panels around on dashboards. Tried to drag panels inside tab panels. Added tab. Things were less obviously broken.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20415
Summary:
Depends on D20413. Ref T13272. When you search for stuff, you can "Use Results > Add to Dashboard" to generate a query panel.
This needs some updating after the recent refactoring. All the changes are pretty straightforward. Swaps a giant `<select />` for a tokenizer with a datasource.
Test Plan: Used the "Use Results > Add to Dashboard" flow to create a panel on a dashboard using a query.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20414
Summary:
Depends on D20412. See PHI1147.
- Index the targets of "Add Reviewer", "Add Blocking Reviewer", "Add Auditor", "Add Subscriber", and "Remove Subscriber" Herald rules. My major goal is to get Owners packages. This will also hit projects/users, but we just don't read those edges (for now, at least).
- Add a "Related Herald Rules" panel to Owners Package pages.
- Add a migration to reindex Herald rules for the recent build plan stuff and this, now that such a migration is easy to write.
Test Plan:
Ran migration, verified all rules reindexed.
{F6372695}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D20413
Summary:
Depends on D20411. Ref T13272. Dashboards and panels have new indexes (Ferret and usage edges) that need a rebuild.
For large datasets like commits we have the "activity" flow in T11932, but realistically these rebuilds won't take more than a few minutes on any realistic install so we should be able to just queue them up as migrations.
Let migrations insert a job to basically run `bin/search index --type SomeObjectType`, then do that for dashboards and panels.
(I'll do Herald rules in a followup too, but I want to tweak one indexing thing there.)
Test Plan: Ran the migration, ran `bin/phd debug task`, saw everything get indexed with no manual intervention.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20412
Summary:
Ref T7667. On the road to locking the auth config, also clean up some minor UI issues:
* Only show the warning about not Phacility instance auth if the user isn't a manager (see next diff).
* When rendering more than one warning in the guidance, add bullets.
* I didn't like the text in the `auth.config-lock` config setting.
Test Plan: Loaded the page, saw more reasonable-looking guidance: {F6369405}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T7667
Differential Revision: https://secure.phabricator.com/D20400
Summary:
Depends on D20410. Ref T13272. Dashboards/Panels currently use older "ngram" indexing, which is a less-powerful precursor to Ferret. Throw away the ngram index and provide a Ferret index instead. Also:
- Remove the NUX state, which links to the wrong place now and doesn't seem terribly important.
- Add project tags to the search result list.
- Make the "No Tags" tag a little less conspicious.
Test Plan:
- Indexed dashboards and panels.
- Searched for dashboards and panels via SearchEngine using Ferret "query" field.
- Searched for panels via "Add Existing Panel" datasource typeahead.
- Searched for dashboards via "Add Menu Item > Dashboard" on a ProfileMenu via typeahead.
- Viewed dashboard NUX state (no special state, but no more bad link to "/create/").
- Viewed dashboard list, saw project tags.
- Viewed dashboards with no project tags ("No Tags" is now displayed but less visible).
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20411
Summary: Depends on D20409. Ref T13272. Before "ProfileMenu", dashboards were installed on specific objects using this table. Installs are now handled via ProfileMenu and this table no longer has any meaningful readers. Remove references to the table and destroy it.
Test Plan: Grepped for `DashboardInstall`.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20410
Summary: Depends on D20408. Ref T13272. The actual JS is still a little bit iffy, but this makes the server side "move" operation work correctly by updating it to use the same code as everything else.
Test Plan: Moved panels around on single-column and multi-column dashboards, saw them move to reasonable places and stay there when I reloaded the page.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20409
Summary:
Depends on D20407. Ref T13272. This updates the "add panel" (which has two flavors: "add existing" and "create new") and "remove panel" flows to work with the new duplicate-friendly storage format.
- We now modify panels by "panelKey", not by panel PHID, so one dashboard may have multiple copies of the same panel and we can still figure out what's going on.
- We now work with "contextPHID", not "dashboardID", to make some flows with tab panels (or other nested panels in the future) easier.
The only major remaining flow is the Javascript "move panels around with drag-and-drop" flow.
Test Plan:
- Added panels to a dashboard with "Create New Panel".
- Added panels to a dashboard with "Add Existing Panel".
- Removed panels from a dashboard.
- Added and removed duplicate panels, got a correctly-functioning dashboard that didn't care about duplicates.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20408
Summary:
Depends on D20406. Ref T13272. This gets about half of Dashboards working with the new "duplicate panel friendly" storage format. Followups will fix the write pathways.
Collateral damage here includes:
- Remove the old Dashboard/Panel edge type. We have a new, more general edge type for "container X uses panel Y", and we don't need this edge type for anything else.
- Remove "attachPanels()" from Dashboard. Only rendering actually needs this, and it can just load the panels.
- Remove "attachPanelPHIDs()" from Dashboard. We can look at the panel refs to figure this out.
- Remove "attachProjects()" from Dashboard. Nothing uses this and it's not a very modern approach.
- `getPanelPHIDs()` just looks at the config now.
- Deleted some `LayoutConfig`-related code which is broken/obsolete.
Test Plan:
- Viewed various dashboards which were created before the changes, saw them render correctly.
- Viewed a dashboard with two of the same panel! AMAZING!
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20407
Summary:
Depends on D20405. Ref T13272. Currently, the `PhabricatorDashboardLayoutConfig` class uses a lot of `switch()` statements to define layout modes.
Although I'm not planning to add thousands of new layout modes, this (and upcoming changes) can be made substantially cleaner by using a standard modular approach.
(This doesn't fully remove `PhabricatorDashboardLayoutConfig` yet, but that will happen soon.)
Test Plan: Edited a dashboard, saw the same layout modes as before.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20406
Summary:
Depends on D20402. Ref T13272. Replaces an old-school hard-coded "EditController" with a more modern one.
The actual panel stuff is still using a weird mix of legacy manual `save()` calls, but that's up next.
Test Plan: Created Dashboards, edited all dashboard fields via "Edit Dashboard".
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20403
Summary:
Depends on D20399. Ref T13272. I'm moving toward fixing all the "moving panels around on Dashboards breaks the entire world" problems.
On the way there, modularize Dashboard transactions.
Test Plan:
- Created a new Dashboard.
- Edited all fiedls of a dashboard.
- Archived/restored a dashboard.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20402
Summary:
Depends on D20398. Ref T13272. Fixes T6018. Previously, panels showed "used on dashboards: x, y", but this did not include cases where a panel was used by another container panel (today, a tab panel).
Do edge indexing when a dashboard or panel is saved, then pull the edges on the Panel page so we can provide a full list of uses.
Test Plan: {F6369289}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272, T6018
Differential Revision: https://secure.phabricator.com/D20399
Summary:
Depends on D20397. Ref T13272. Similar to the recent "where are Herald rules used" stuff, show which menus Dashboards are installed in.
This is mostly straightforward, except that I pulled some of the Herald logic into a parent class so it could be shared.
Test Plan: {F6369164}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20398
Summary:
Depends on D20396. Ref T13272. Currently, using the dropdowns to edit a tab panel from a dashboard redirects you to the tab panel page.
Instead, redirect back to the context page (usually, a dashboard -- but theoretically a containing tab panel, since you can put tab panels inside tab panels).
Also, fix some JS issues with non-integer panel keys. I've moved panel keys from "0, 1, 2, ..." to having arbitrary keys to make some operations less flimsy/error-prone, but this needs some JS tweaks.
Test Plan: Edited a tab panel from a dashboard, got sent sensibly back to the dashboard.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13272
Differential Revision: https://secure.phabricator.com/D20397
