Summary:
See PHI775. See D19499. Originally, see PHI720.
D19499 broke the standalone "Branches" page for commits. Normally, you reach this by taking these steps:
- View a commit which is contained by 11 or more branches.
- Click the "More Branches..." link in the "Branches" field.
- You should be taken to a list of all branches which contain the commit.
The change to the 'branch' parameter was adjusted in the query that builds the "x, y, z, More Branches..." list, but not on the actual "Branches" list with the full list. Adjust it.
Test Plan:
- Set display limit to 1, viewed a commit on "master" and "stable", clicked "More Branches".
- Before: saw only "master".
- After: saw both "master" and "stable".
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19532
Summary:
Ref T13168. I'm not sure how this worked before, but I ran into this issue on my new laptop.
SiteSource accesses `PhabrictatorEnv::getEnvConfig('phabricator.base-uri')` when local, which may poison the cache and lock the value since we don't later discard the cache.
Specifically, when I access `http://locala.phacility.com`, I was getting an error like "You made a request for locala.phacility.com, but no configured site can serve this request.". This was because the base-uri was being incorrectly frozen as "local.phacility.com". The expectation is that it will match, so the standard PlatformSite will serve the request.
Test Plan:
- Before change: "no configured site" error.
- After change: local instance works properly.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13168
Differential Revision: https://secure.phabricator.com/D19526
Summary: Ref T13168. This is just a small quality-of-life fix: we can disclose which public key we're talking about because public keys are public.
Test Plan:
- Hit public key error (through my own bumbling / not reading or following instructions). Specifically, I haven't associated the key with a device in Almanac.
- Before: vague error.
- After: more specific error with enough key material that I could grep for it.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: yelirekim
Maniphest Tasks: T13168
Differential Revision: https://secure.phabricator.com/D19516
Summary:
See PHI746. See also T11833, perhaps. Ref T13151.
Long ago, parent revisions were called "dependent revisions". This was changed to "parent revisions" in the action UI to improve clarity, but not changed in the timeline stories.
Update the timeline stories to use the same language the actions in the UI use.
Test Plan:
{F5732876}
{F5732877}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19514
Summary:
Ref T13151. See PHI727. Update the dashboard widget/panel datasource to actually query results using what the user typed.
The current approach is blind to what the user typed when pulling results from the database, and gets limited to an artificially small number of results somewhere in the pipeline.
Test Plan:
- Queried for panels with text queries.
- Queried for panels with `W123` queries.
- This is substantially similar to the Owners datasource, which received a similar update in D17142 and has worked well since then.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19511
Summary:
Depends on D19509. Ref T13151. See PHI725. `transaction.search` supports "constraints" and the documentation mentions it in passing, but doesn't really show how to do it, and the method is not automatically self-documenting because it isn't a "real" `*.search` method.
Add an example of how to use the `contraints` parameter to retrieve information about only the relevant transactions.
Also remove a refernece to `requestb.in`, which is now defunct.
Test Plan: Called `transaction.search` with similar parameters.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19510
Summary:
See PHI725. Ref T13151. These actions are somewhat unusual and I considered different ways to represent them (make them look like "status" transactions; build multiple synthetic transactions) but ultimately landed on the simplest approach of just exposing them more or less as they exist internally.
I haven't included data for any of them. Most don't really have any data, but "accept" does. I'm holding off on providing more data until after T731, which may shake up the internal format.
Test Plan: Applied most of these transactions against a revision, queried for it with `transaction.search`, got distinguishable transactions out.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19509
Summary: See PHI725. Ref T13151. We currently try to load comments unconditionally, but not all objects (like projects) have comments. Only try to load comments if an object actually has comments.
Test Plan: Queried for an object with no comments, like project `#masonry`, via `transaction.search`.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19507
Summary:
Ref T13151. See PHI725. By default, "transaction.search" doesn't provide details about transactions because many have bad/weird/policy-violating internal types or fields.
The "create" transaction is simple and straightforward, so label it to allow callers to distinguish it.
Test Plan:
- Created a new task.
- Called `transaction.search` on it.
- Saw the labelled "create" transaction.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: swisspol
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19505
Summary:
Depends on D19503. Ref T13151. See PHI719. If you have something like a script which updates an object in a loop, we can end up queueing many search reindex tasks.
These tasks may reasonably contend for the lock, especially if the object is larger (lots of text and/or lots of comments) and indexing takes a few seconds.
This isn't concerning, and the indexers should converge to good behavior quickly once the updates stop.
Today, they'll spew a bunch of serious-looking lock exceptions into the log. Instead, just yield so it's more clear that there's (normally) no cause for concern here.
Test Plan: Ran `bin/search index Txxx --force` on a large object in multiple windows with a 0 second lock, saw an explicit yield instead of a lock exception.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19504
Summary:
Depends on D19502. Ref T13151. See PHI719. An install ended up with an object with 111,000+ comments on it because someone wrote a script to treat it like a logfile.
Although we seem to do mostly okay with this (locally, it only takes about 30s to index a similar object) we'll hit a wall somewhere (since we need to hold everything in memory), and it's hard to imagine a legitimate object with more than 1,000 comments. Just ignore comments past the first thousand.
(Conpherence threads may legitimately have more than 1,000 comments, but go through a different indexer.)
Test Plan:
- Piped some comments into `maniphest.edit` in a loop to create a task with 100K comments.
- Ran `bin/search index Txxx --force` to reindex it, with `--trace`.
- Before: task indexed in about 30s.
- After: script loaded comments with LIMIT 1000 and indexed in a couple seconds.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19503
Summary:
Ref T13151. See PHI719. One minor hiccup in debugging the issue (which ended up being "revision has 100K comments") was that the `SearchWorker` did not show which object it was indexing.
Add `'objectPHID'` to the queue call so you can see which object is affected from the web UI.
Test Plan:
- Stopped daemons.
- Used `bin/search index D123 --background` to queue a search task.
- Viewed task details in web UI from `/daemon/`.
- Before change: no indication of which object was being indexed.
- After change: page helpfully shows that the task is indexing D123.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19502
Summary:
Fixes T13159. Two issues here:
- When viewing a particular config setting, there's an extra "Config" crumb.
- On the page for a config group, the link to the parent group has an extra "/config/" in it.
Test Plan:
- Viewed a page for a particular setting, no longer saw an extra "Config" crumb.
- Viewed a page for a setting group, clicked parent crumb, got taken to a real page.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13159
Differential Revision: https://secure.phabricator.com/D19501
Summary:
Ref T13151. See PHI720. If you want to test if commit X appears on specific branch Y, `git branch --contains X -- Y` is faster than (effectively) `git branch --contains X | grep Y`.
Since this call has a "branch" parameter anyway, use it as the pattern argument if provided.
Test Plan:
- Called the API method with no parameters, got all branches.
- Called the API method with `master`, got just master.
- Called the API method with `maste*`, got master. This behavior is not officially supported and may change in the future.
- Viewed a commit, still saw all branches.
- Grepped for `diffusion.branchquery` and verified that no remaining callsites pass a default "branch" parameter.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19499
Summary: Fixes T13155. Ref T13151. A recent change (D19455) changed the return format here, but I missed this special case for empty commits.
Test Plan:
- T13155 has a good set of reproduction instructions.
- Pushed an empty commit.
- Before: bunch of warning log spew.
- After: clean logs.
Reviewers: amckinley, avivey
Reviewed By: avivey
Maniphest Tasks: T13155, T13151
Differential Revision: https://secure.phabricator.com/D19500
Summary: I landed D19491 a little aggressively, so allow this field to be null until after the migration goes out.
Test Plan: Loaded commits without identity objects; did not get any errors.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D19496
Summary: Ref T12164. Make it easier to work with identity objects by attaching them to commits and attaching users to identities.
Test Plan: Loaded some commits with `->needIdentities(true)` and checked the resulting objects.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T12164
Differential Revision: https://secure.phabricator.com/D19491
Summary: This never worked.
Test Plan: Ran `bin/repository rebuild-identities` and viewed identity objects with `currentEffectiveUserID`s and no longer got errors about attempting to attach `null` objects instead of `PhabricatorUser` objects.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D19495
Summary:
See <https://discourse.phabricator-community.org/t/commit-6011085b0fcd-breaks-sending-certain-email/1571>. Some mailers get upset if we `setHTMLBody(...)` with an empty string.
There's some possible argument they should be more graceful about this, but it's reasonably pretty ambiguous.
Only try to set the HTML body if we actually have a nonempty HTML body.
Test Plan:
- Configured an "smtp" mailer.
- Ran `echo hi | ./bin/mail send-test --to someone@somewhere.com --subject test`.
- Before: error about empty message body.
- After: no more message body error.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19494
Summary:
Ref T13151. Ref T12164. Two small tweaks:
- If we aren't actually going to change anything, just skip the writes. This makes re-running/resuming a lot faster (~20x, locally).
- Print when we touch a commit so there's some kind of visible status.
This is just a small quality-of-life tweak that I wrote anyway while investigating T13152, and will make finishing off db024, db025 and db010 manually a little easier.
Test Plan:
- Set `authorIdentityPHID` + `committerIdentityPHID` to `NULL`.
- Ran `rebuild-identities`, saw status information.
- Ran `rebuild-identiites` again, saw it go faster with status information.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151, T12164
Differential Revision: https://secure.phabricator.com/D19484
Summary:
Ref T13151. See T11767. See PHI686. Although we limit outbound mail text bodies, the limit doesn't currently apply to attachments, HTML bodies, or headers. T11767 discusses improving this in the general case.
In the wild, an install hit an issue (see PHI686) where edits to Phriction pages generate very large HTML bodies. Check and respect the limit when building HTML bodies.
If we don't have enough room for the HTML body, we just drop it. We have the text body to fall back to, and HTML is difficult to truncate safely.
Test Plan: Added unit tests and made them pass.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19489
Summary:
Ref T13151. See PHI683. Ref T12314.
You can currently change object subtypes via Conduit (`maniphest.edit`) but not via the web UI.
Changing object subtypes is inherently a somewhat-perilous operation that likely has a lot of rough edges we'll need to smooth over eventually, mostly around changing an object from subtype X to subtype Y, where some field exists on one but not the other. This isn't a huge issue, just not entirely intuitive.
It should also, in theory, be fairly rare.
As a reasonable middle ground, provide web UI access via the bulk editor. This makes it possible, but doesn't clutter the UI up with a rarely-used option with rough edges.
Test Plan:
- With subtypes not configured, saw a normal bulk editor with no new option.
- With subtypes configured, swapped tasks subtypes via bulk editor.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151, T12314
Differential Revision: https://secure.phabricator.com/D19490
Summary:
Depends on D19487. Ref T13151. See PHI647. For some objects, like revisions, we can build slightly more useful secure email without actually disclosing anything.
In the general case, the object monogram may disclose information (`#acquire-competitor`) but most do not, so applications can whitelist an acceptable nondisclosing subject and link.
Support doing this, and make Differential do it. When we don't have a whitelisted URI but do know the object the mail is about, include a generic PHID-based URI; these are always nondisclosing.
Test Plan:
- Without the Differential changes, sent normal mail (no changes) and secure mail (new generic PHID-based link).
- With the Differential changes, sent secure mail; got richer subject and body link.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19488
Summary:
Ref T13151. See PHI647. This allows us to link to any object by PHID, without disclosing information in the monogram (like `#fire-steve`).
This capability is relevant when building "secure mail", to provide a link to the object regardless of whether the monogram discloses information or not.
Test Plan: Visited `/object/D123/` (redirect), `/object/xyz/` (404), `/object/PHID-DREV-.../` (redirect).
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19487
Summary:
Ref T13151. See PHI702. An install is interested in a "members of all projects" (vs "members of any project", which is currently implemented) rule.
Although this is fairly niche, I think it's reasonable and doesn't have much of a maintenance cost.
This could already be implemented as an extension, but it would have to copy/paste a bunch of code.
Test Plan:
- Ran unit tests.
- Used the UI to select this policy for a task, with various values. Joined/left projects to satisfy/fail the rule. Behavior seemed correct.
- Used the UI to select the existing policy rule ("any project"), joined/left projects to satisfy/fail the rule. Doesn't look like I broke anything.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19486
Summary:
Ref T13151. See PHI701. Unified diffs are currently missing the logic to apply the "old-full" and "new-full" classes, which results in a too-light coloration for fully added or removed lines.
Make this logic consistent with the two-up renderer so we use the same colors in both.
Test Plan: Viewed diffs and swapped between 1-up and 2-up renderers, now saw the same coloration.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19482
Summary: Ref T13152. The pager does a bit of magic here and doesn't populate `nextPageID` when it knows it got an exact final page. The logic misfired in this case and sent us back to the start.
Test Plan:
- Set page size to 1 to guarantee rows were an exact multiple of page size.
- Ran `rebuild-identities` (I no-op'd the actual logic to make it faster).
- Before: looped forever.
- After: clean exit after processing everything.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13152
Differential Revision: https://secure.phabricator.com/D19479
Summary:
Ref T13151. See PHI654. Depends on D19477. If you have long package names, the table of contents (e.g., in Differential) can end up expanding to be gigantic.
Getting tables to behave nicely is hard (or, at least, I can't figure it out after spending a decent amount of time on it; see also `AphrontTableView::renderSingleDisplayLine()`). I tried a bunch of things and Googled for a bit but didn't make any progress on finding a CSS solution. Just truncate the package names to get reasonable behavior without falling down any kind of CSS rabbit hole.
Test Plan:
- Created a package named "Very long package name...".
- Created a package named "MMMMMMMMMMMMMMMMMMMMMM...".
- Had them own a file in a Differential revision, viewed that revision.
- Before: table is pushed out to several times the browser window width and everything is kind of a mess.
- After: package names get truncated to something reasonable.
{F5652953}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19478
Summary:
Ref T13151. See PHI684. Currently, the `MailableFunction` datasource does not include Owners packages, but they are valid subscribers and the `Mailable` datasource includes them.
Include them in the `MailableFunction` datasource, too.
Test Plan: Searched for revisions with particular package subscribers, got expected results in the UI (tokenizer knew about packages) and response.
Reviewers: amckinley, jmeador
Reviewed By: jmeador
Maniphest Tasks: T13151
Differential Revision: https://secure.phabricator.com/D19476
Summary:
Fixes T13145. The list controllers properly support public access already, but some of the view/detail controllers did not.
Allow logged-out users to browse builds, buildables, plans, etc., provided they can see the corresponding objects.
Test Plan: As a logged-out user, browsed around builds, build plans, logs, etc., without hitting any login pages.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13145
Differential Revision: https://secure.phabricator.com/D19459
Summary:
See PHI689. It can be difficult to distinguish between cards with the same number but different expiration dates (common when the bank sends you a new card).
For now, show the expiration date on the cart checkout screen.
Test Plan: Viewed a cart checkout screen with multiple cards, saw expiration dates.
Reviewers: amckinley
Differential Revision: https://secure.phabricator.com/D19462
Summary:
Depends on D19443. Creates a workflow for populating the new identity table by iterating over commits, either one repo at a time or all at once. Locally caches identities to avoid fetching them `inf` times. An actual migration that invokes this workflow will come in another revision that won't land until at least next week.
Performance is ~2k commits in 4.9s on my local machine.
Test Plan: Ran locally a few times with a few different states of the `repository_identity` table.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: jcox, Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D19446
Summary: Depends on D19429. Depends on D19423. Ref T12164. This creates new columns `authorIdentityPHID` and `committerIdentityPHID` on commit objects and starts populating them. Also adds the ability to explicitly set an Identity's assignee to "unassigned()" to null out an incorrect auto-assign. Adds more search functionality to identities. Also creates a daemon task for handling users adding new email address and attempts to associate unclaimed identities.
Test Plan: Imported some repos, watched new columns get populated. Added a new email address for a previous commit, saw daemon job run and assign the identity to the new user. Searched for identities in various and sundry ways.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T12164
Differential Revision: https://secure.phabricator.com/D19443
Summary: Depends on D19423. Ref T12164. Adds controllers capable of listing and editing `PhabricatorRepositoryIdentity` objects. Starts creating those objects when commits are parsed.
Test Plan: Reparsed some revisions, observed objects getting created in the database. Altered some `Identity` objects using the controllers and observed effects in the database. No attempts made to validate behavior under "challenging" author/committer strings.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T12164
Differential Revision: https://secure.phabricator.com/D19429
Summary: Ref T12164. Start building initial objects for managing `RepositoryIdentity` objects. This won't land until much more of the infrastructure is in place.
Test Plan: Ran `bin/storage upgrade` and observed expected table.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T12164
Differential Revision: https://secure.phabricator.com/D19423
Summary:
Ref T13142. When commits are pushed, we try to handle them on one of two pathways:
- Normal changes: we load these into memory and potentially apply Herald content rules to them.
- "Enormous" changes: we don't load these into memory and skip content rules for them.
The goal is to degrade gracefully when users push huge changes: they should work, just not support all the features.
However, some changes can slip through the cracks right now:
- If you push a lot of commits at once, we'll try to cache all of the changes smaller than 1GB in memory. This can require an arbitrarily large amount of RAM.
- We calculate sizes by just looking at the `strlen()` of the diff, but a changeset takes more RAM in PHP than the raw diff does. So even if a diff is "only" 500MB, it can take much more memory than that. On systems with relatively little memory available, this may result in OOM while processing changes that are close to the "enormous" limit.
This change makes two improvements:
- Instead of caching everything, cache only 64MB of things.
- For most pushes, this is the same, since they have less than 64MB of diffs.
- For pushes of single very large changes, this is a bit slower (more CPU) since we have to do some work twice.
- For pushes of many changes, this is slower (more CPU) since we have to do some work twice, but, critically, doesn't require unlimited memory.
- Instead of flagging changes as "enormous" at 1GB, flag them as "enormous" at 256MB.
- This reduces how much memory is required to process the largest "non-enormous" changes.
- This also gets us under Git's hard-coded 512MB "always binary" cutoff; see T13143.
- This is still completely gigantic and way larger than any normal change should be.
An additional improvement would be to try to reduce the amount of memory we need to use to hold a change in process memory. I think the other changes here alone will fix the immediate issue in PHI657, but it would be nice if the "largest non-enormous change" required only a couple gigs of RAM.
Test Plan:
- Used `ini_set('memory_limit', '1G')` to artificially limit memory to 1GB.
- Pushed a series of two commits which add two 550MB text files (Temporarily, I added a `--binary` flag to trick Git into showing real diffs for these, see T13143.)
- Got a memory limit error.
- Applied the "cache only 64MB of stuff" and "consider 256MB, not 1GB, to be enormous" changes.
- Pushed again, got properly rejected as enormous.
- Added `memory_get_usage()` calls to measure how actual memory size and reported "size" estimate compare. For these changes, saw a 639MB diff require 31,479MB of memory, i.e. a factor of about 50x. This is, uh, pretty not great.
- Allowed enormous changes, pushed again, push went through.
Reviewers: amckinley
Maniphest Tasks: T13142
Differential Revision: https://secure.phabricator.com/D19455
Summary:
Ref T13141. Currently, during first-time setup we don't surface all the details about connection exceptions that we could: the underlying exception is discarded inside cluster connection management.
This isn't a huge issue since the reason for connection problems is usually fairly obvious, but in at least one case (see T13141) we hit a less-than-obvious exception.
Instead, store the original exception and propagate the message up the stack so users have more information about the problem.
Test Plan:
- Configured an intentionally bad MySQL username.
- Restarted Apache and loaded Phabricator.
- Got a more helpful exception with a specific authentication error message.
{F5622361}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13141
Differential Revision: https://secure.phabricator.com/D19454
Summary:
Ref T13137. See that task for discussion.
When we show a diff-of-diffs, we often render stubs for files which didn't change between the diffs. These stubs usually aren't a big deal, but for certain types of changes (like refactors) they can create a lot of clutter.
Instead, hide these stubs and show a notice that we hid them.
Test Plan:
- Created a revision affecting 4 files.
- Updated it with a diff that changed only 1 of the 4 files.
- Added an inline comment to a different file.
- Viewed the diff of diffs.
- Before: 4 changesets with two "nothing changed" stubs.
- After: 2 changesets with the stubs hidden.
{F5621083}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13137
Differential Revision: https://secure.phabricator.com/D19453
Summary:
Ref T13137. The "analyze/cache data about changesets" step is becoming more involved. We recently added detection for generated code to support "Ignore generated changes" in Owners, and I now plan to hash the new file content so we can hide changes which have no effect.
Before adding this new hashing step, pull the "detect copied code" and "detect generated code" stuff out and move them to a separate `ChangesetEngine`. Then support doing a changeset rebuild directly with `bin/differential rebuild-changesets`.
This simplifies things a bit and makes testing easier since you don't need to keep creating new revisions to re-run copy/generated/hash logic.
Test Plan: Ran `bin/differential rebuild-changesets --revision Dxxx`, saw changesets rebuild. See also next change.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13137
Differential Revision: https://secure.phabricator.com/D19452
Summary:
Fixes T13140. See PHI660.
Recent versions of Subversion can send a `(get-file true false false )` protocol frame with extra space between "false" and "false". This is allowed by the protocol spec, but never normally happens, and we do not parse it correctly.
Instead, parse it correctly.
Test Plan:
- Added unit tests.
- Ran `svn proplist svn+ssh://.../diffusion/X/file.c` under SVN 1.10 before and after the change.
- Before: indefinite hang.
- After: completed in finite time.
Reviewers: amckinley, asherkin
Reviewed By: amckinley, asherkin
Maniphest Tasks: T13140
Differential Revision: https://secure.phabricator.com/D19451
Summary:
See D19446. This should make it easier to process larger, more complex result sets in constant memory.
Today, `LiskMigrationIterator` takes constant memory but can't apply `needX()` reqeusts or `withY(...)` constraints.
Using a raw `Query` can handle this stuff, but requires memory proportional to the size of the result set.
Offer the best of both worlds: constant memory and full access to the power of `Query` classes.
Test Plan:
Used this script to iterate over every commit, saw sensible behavior:
```name=list-commits.php
<?php
require_once 'scripts/init/init-script.php';
$viewer = PhabricatorUser::getOmnipotentUser();
$query = id(new DiffusionCommitQuery())
->setViewer($viewer);
$iterator = new PhabricatorQueryIterator($query);
foreach ($iterator as $commit) {
echo $commit->getID()."\n";
}
```
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19450
Summary: Ref PHI662. Viewing a dashboard you don't have permission to view (in the Dashboard application) currently fatals while building crumbs, since we fail to build the ` ... > Dashboard 123 > ...` crumb.
Test Plan:
- Viewed a dashboard I didn't have permission to view in the Dashboards application.
- Before patch, fatal when calling `getID()` on a non-object.
- After patch, sensible policy error page.
- Viewed a dashboard I can view, saw sensible crumbs.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19449
Summary:
See <https://hackerone.com/reports/351361>. We currently require MFA on the screen leading into the user create flow, but not the actual create flow.
That is, `/people/create/` (which is just a "choose a type of account" page) requires MFA, but `/people/new/<type>/` does not, even though this is the actual creation page.
Requiring MFA to create users isn't especially critical: creating users isn't really a dangerous action. The major threat is probably just that an attacker can extend their access to an install by creating an account which they have credentials for.
It also isn't consistently enforced: you can invite users or approve users without an MFA check.
So there's an argument for just removing the check. However, I think the check is probably reasonable and that we'd likely prefer to add some more checks eventually (e.g., require MFA to approve or invite) since these actions are rare and could represent useful tools for an attacker even if they are not especially dangerous on their own. This is also the only way to create bot or mailing list accounts, so this check does //something// on its own, at least.
Test Plan:
- Visited `/people/new/standard/` as an admin with MFA configured.
- Before patch: no MFA prompt.
- After patch: MFA prompt.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19448
Summary: Ref T13137. See PHI592. Depends on D19444. Apply a limit up front to stop patches which are way too big (e.g., 600MB of videos) from generating in the first place.
Test Plan:
- Configured inline patches in git format.
- Created a normal revision, got an inline git patch.
- Created a revision with a 10MB video file, got no inline patch.
- (Added a bunch of debugging stuff to make sure the internal pathway was working.)
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13137
Differential Revision: https://secure.phabricator.com/D19445
Summary:
See <https://discourse.phabricator-community.org/t/maniphest-home-page-crash-after-d19417/1445/3>. These special-token-only searches currently end up populating an empty `ownerPHIDs`, which fatals after the stricter check in D19417.
Make the fatal on `withConstraint(array())` explicit and only set the PHID constraint if we have some PHIDs left.
Test Plan: Searched for "No Owner", "Any Owner", an actual owner, "No Owner + actual user".
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19440
Summary:
See PHI652. When you `echo x | arc paste` today, you end up with a Paste object that has the empty string as its "language".
This is normally not valid. Pastes where the language should be autodetected should have the value `null`, not the empty string.
This behavior likely changed when `paste.create` got rewritten in terms of `paste.edit`. Adjust the implementation so it only adds the LANGUAGE transaction if there's an actual language.
Also, fix an issue where you can't use the "delete" key to delete tokens with the empty string as their value.
Test Plan:
- Created a paste with `echo x | arc paste`, got a paste in autodetect mode instead of with a bogus language value.
- Created a paste with `echo x | arc paste --lang rainbow`, got a rainbow paste.
- Deleted an empty string token with the keyboard.
- Deleted normal tokens with the keyboard.
- Edited subscribers/etc normally with the keyboard and mouse to make sure I didn't ruin anything.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19437
Summary:
See PHI624. Some of the mobile navigation and breadcrumbs in support pacts aren't as good as they could be.
In particular, we generally collapse crumbs on mobile to just the first and last crumbs. The first crumb is the application; the last is the current page.
On `/PHIxxx` pages, the first crumb isn't very useful since the Support landing page is two levels up: you usually want to go back to the pact, not all the way back to the Support landing page.
We also don't need the space since the last crumb (`PHIxxx`) is always small.
Allow Support and other similar applications to tailor the crumb behavior more narrowly if they end up in situations like this.
Test Plan:
- With an additional change to instances (see next diff), viewed a support issue page (`/PHI123`) on mobile and desktop.
- Saw a link directly back to the pact on both mobile and desktop.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19438
Summary:
Ref T13137. See PHI609. An install would like to filter audit requests on a particular branch, e.g. "master".
This is difficult in the general case because we can not apply this constraint efficiently under every conceivable data shape, but we can do a reasonable job in most practical cases.
See T13137#238822 for more detailed discussion on the approach here.
This is a bit rough, but should do the job for now.
Test Plan:
- Filtered commits by various branches, e.g. "master"; "lfs". Saw correct-seeming results.
- Stubbed out the "just list everything" path to hit the `diffusion.internal.ancestors` path, saw the same correct-seeming results.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13137
Differential Revision: https://secure.phabricator.com/D19431
Summary:
See <https://discourse.phabricator-community.org/t/maniphest-non-integer-point-values-in-csv-export/1443>.
We currently export the Maniphest "points" field as an integer, but allow it to accept decimal values (e.g. "6.25").
Also fix a bug where we wouldn't roll over from "..., X, Y, Z, AA, AB, ..." correctly for Excel column names if sheet had more than 26 columns.
Test Plan:
- Set a task point value to 6.25.
- Exported to text, JSON, XLS.
- Saw 6.25 represented accurately in exports.
- Exported an excel sheet with 27+ columns.
- Manually printed the first 200 column names to check that the algorithm looks correct.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19434
Summary:
See PHI251. Ref T13137.
- Replace the perplexing text box with a checkbox that explains what it does.
- Mention this feature in the documentation.
Test Plan:
- Clicked/unclicked checkbox.
- Read documentation.
- Used an existing checkbox control in Slowvote to make sure I didn't break it.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13137
Differential Revision: https://secure.phabricator.com/D19433
Summary:
See <https://discourse.phabricator-community.org/t/hidden-description-field-in-maniphest-task-breaks-form/1432>.
If you hide the "Description" field in Maniphest, we still try to render a remarkup preview for it. This causes a JS error and a nonfunctional element on the page.
Instead, hide the preview panel if the field has been locked or hidden.
Test Plan:
- Hid the field, loaded the form, no more preview panel / JS error.
- Used a normal form with the field visible, saw a normal preview.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19432
Summary:
See PHI638. When a diff is large (between 100 and 1000 files), we collapse content by default unless a change also has inline comments.
This rule isn't explicitly explained anywhere. Although it's not really a critical rule, it fits easily enough into the UI callout.
Also render the UI callout in a slightly more modern way and avoid `hsprintf()`.
Test Plan:
{F5596496}
- Also, clicked the "Expand" link and saw everything expand properly.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19430
Summary:
Depends on D19427. Ref T13130. See PHI251. Support configuring owners packages so they ignore generated paths.
This is still a little rough. A couple limitations:
- It's hard to figure out how to use this control if you don't know what it's for, but we don't currently have a "CheckboxesEditField". I may add that soon.
- The attribute ignore list doesn't apply to Diffusion, only Differential, which isn't obvious. I'll either try to make it work in Diffusion or note this somewhere.
- No documentation yet (which could mitigate the other two issues a bit).
But the actual behavior seems to work fine.
Test Plan:
- Set a package to ignore paths with the "generated" attribute. Saw the package stop matching generated paths in Differential.
- Removed the attribute from the ignore list.
- Tried to set invalid attributes, got sensible errors.
- Queried a package with Conduit, got the ignored attribute list.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19428
Summary:
Depends on D19426. Ref T13130. Ref T13065. While I'm making changes to Owners for "Ignore generated paths", clean up the "mailKey" column.
We recently (D19399) added code to automatically generate and manage mail keys so we don't need a ton of `mailKey` properties in the future. Migrate existing mail keys and blow away the explicit column on packages.
Test Plan: Ran migration, manually looked at the database and saw sensible data. Edited a package to send some mail, which looked good.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13130, T13065
Differential Revision: https://secure.phabricator.com/D19427
Summary:
Depends on D19425. Ref T13130. See PHI251. Now that changesets have a durable "generated" attribute, we can let owners packages check it when we're computing which packages are affected by a revision.
There's no way to actualy configure a package to have this behavior yet.
Test Plan:
- Created a revision affecting a generated file and a non-generated file.
- When I faked `mustMatchUngeneratedPaths()` to `return true;`, saw the non-generated file get no packages owning it.
- Normally: lots of packages owning it).
- Created a revision affecting only generated files.
- When I faked things, saw no Owners actions trigger.
- Normally: some packages added reviewers or subscribers.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19426
Summary:
Ref T13130. See PHI251. Currently, changesets are marked as "generated" (i.e., the file contains generated code and does not normally need to be reviewed) at display time.
An install would like support for having Owners rules ignore generated files. Additionally, future changes anticipate making "generated" and some other similar behaviors more flexible and more general.
To support these, move toward a world where:
- Changesets have "attributes": today, generated. In the future, perhaps: third-party, highlight-as, encoding, enormous-text-file, etc.
- Attributes are either "trusted" (usually: the server assigned the attribute) or "untrusted" (usually: the client assigned the attribute). For attributes like "highlight-as", this isn't relevant, but I'd like to provide tools so that you can't make `arc` mark every file as "generated" and sneak past review rules in the future.
Here, the `differential.generated-paths` config can mark a file as "generated" with a trusted attribute. The `@generated`-in-content rule can mark a file as "generated" with an untrusted attribute.
Putting these attributes on changesets at creation time instead of display time will let Owners interact with changesets cheaply: it won't have to render an entire changeset just to figure out if it's generated or not.
Test Plan:
- Created a revision touching several files, some generated and some not.
- Saw the generated files get marked properly with attribute metadata in the database, and show/fold as "Generated" in the UI.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19425
Summary:
Fixes T13135. See PHI633. For at least some video files with legitimate MIME type "video/quicktime", Chrome can play them but refuses to if the `<source />` tag has a `type="video/quicktime"` attribute.
To trick Chrome into giving these videos the old college try, omit the "type" attribute. Chrome then tries to play the video, seems to realize it can, and we're back on track.
Since the "type" attribute is theoretically only useful to help browsers select among multiple different alternatives and we're only presenting one alternative, this seems likely safe and reasonable. Omitting "type" also validates. It's hard to be certain that this won't cause any collateral damage, but intuitively it seems like it should be safe and I wasn't able to identify any problems.
Test Plan:
- Watched a "video/quicktime" MP4 cat video in Chrome/Safari/Firefox.
- See T13135 for discussion, context, and discussion of the behavior of some smaller reproduction cases.
Reviewers: amckinley, asherkin
Reviewed By: amckinley
Maniphest Tasks: T13135
Differential Revision: https://secure.phabricator.com/D19424
Summary:
Fixes T13132. I removed this branch in D19156 when tightening the logic for the new CSP header, but there's a legitimate need for it: downloading files via `arc download`, or more generally being an API consumer of files.
This is not completely safe, but attacks I'm aware of (particularly, cookie fixation, where an attacker could potentially force a victim to become logged in to an account they control) are difficult and not very powerful. We already issue clear setup advice about the importance of configuring this option ("Phabricator is currently configured to serve user uploads directly from the same domain as other content. This is a security risk.") and I think there's significant value in letting API clients just GET file data without having to jump through a lot of weird hoops.
Test Plan:
- With `security.alternate-file-domain` off, tried to `arc download` a file.
- Before: downloaded an HTML dialog page.
- After: downloaded the file.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13132
Differential Revision: https://secure.phabricator.com/D19421
Summary:
Ref T13130. See <https://discourse.phabricator-community.org/t/unable-to-create-owners-package-with-same-path-in-multiple-repositories/1400/1>.
When you edit paths in Owners, we deduplicate similar paths, like `/x/y` and `/x/y/`. However, this logic currently only examines the paths, and incorrectly deduplicates the same path in different repositories.
Instead, consider the repository before deduplicating.
Test Plan:
- Edited an Owners package and added the path "/" in two different repositories.
- Before: only one surived the edit.
- After: both survived.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19420
Summary: See PHI611 for details.
Test Plan:
Ran a Buildkite build, saw Buildkite confirm receipt of these parameters in the HTTP response:
{F5562054}
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D19419
Summary:
Depends on D19416. Ref T13110. Ref T13130. See PHI598. When rendering a "Very Large" revision (affecting more than 1,000 files) we currently compute the package/changeset ownership map normally.
This is basically a big list of which packages own which of the files affected by the change. We use it to:
# Show which packages own each file in the table of contents.
# Show an "(Owns No Changed Paths)" hint in the reviewers list to help catch out-of-date packages that are no longer relevant.
However, this is expensive to build. We don't render the table of contents at all, so (1) is pointless. The value of (2) is very small on these types of changes, and certainly not worth spending many many seconds computing ownership.
Instead, just skip building out these relationships for very large changes.
Test Plan: Viewed a very large change with package owners; verified it no longer built package map data and rendered the package owners with no "(Owns No Changed Paths)" hints.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130, T13110
Differential Revision: https://secure.phabricator.com/D19418
Summary:
Ref T13110. Ref T13130. When a revision is "large" (100 - 1000 files) we hide the actual textual changes by default. When it is "very large" (more than 1000 files) we hide all the changesets by default.
For "very large" diffs, we currently still show the "large" warning, which doesn't really make sense since there aren't any actual changesets.
When a diff is "very large", don't show the "large" warning.
Test Plan:
- Viewed a small diff (<100 files), saw no warnings.
- Viewed a large diff (100-1000 files), saw just the large warning.
- Viewed a very large diff (>1000 files).
- Before: both "large" and "very large" help warnings.
- After: just "very large" warnings.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130, T13110
Differential Revision: https://secure.phabricator.com/D19416
Summary:
See PHI604. Ref T13130. Ref T13105. There's currently no way to turn blame off in Diffusion. Add a "Hide Blame" option to the "View Options" dropdown so it can be toggled off.
Also fix a couple of bugs around this: for example, if you loaded a Jupyter notebook and then switched to "Source" view, blame would incorrectly fail to activate because the original rendering of the "stage" used an asynchronous engine so `willRenderRef()` wasn't called to populate blame.
Test Plan:
- Viewed a source file, toggled blame off/on, reloaded page to see state stick in URL.
- Viewed a Jupyter notebook, toggled to "Source" view, saw blame.
- Viewed stuff in Files (no blame UI options).
- Tried to do some invalid stuff like toggle blame on a non-blame engine (options disable properly).
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130, T13105
Differential Revision: https://secure.phabricator.com/D19414
Summary: See discussion in D19415.
Test Plan: Searched for some owners, found tasks as expected.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D19417
Summary:
See PHI285. Ref T13130. After recent changes Herald sends email about rules, but the mail doesn't currently actually include a link to the rule.
Include a link for consistency and ease-of-use.
Test Plan: Edited a rule, looked at the resulting mail, saw a link to the rule.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19413
Summary:
Fixes T13128. Ref PHI590. This is a rough-and-ready implementation of a new `PhabricatorPolicyCodex->compareToDefaultPolicy()` method that subclasses can override to handle special cases of policy defaults. Also implements a `PolicyCodex` for Phriction documents, because the default policy of a Phriction document is the policy of the root document.
I might break this change into two parts, one of which maintains the current behavior and another which implements `PhrictionDocumentPolicyCodex`.
Test Plan: Created some Phriction docs, fiddled with policies, observed expected colors in the header. Will test more comprehensively after review for basic reasonable-ness.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, swisspol
Maniphest Tasks: T13128
Differential Revision: https://secure.phabricator.com/D19409
Summary:
See PHI615. Ref T13130. An install is reporting that "Lease Working Copy" build steps always report "Built instantly" after completion.
I'm not 100% sure that this is the fix, but I'm like 99% sure: "Lease Working Copy" build steps yield after they ask Drydock for a lease. They will later reenter `doWork()`, see that the lease is filled, and complete.
Right now, we reset the start time every time we enter `doWork()`. Instead, set it only if it hasn't been set yet.
Test Plan: This is low-risk and a bit tricky to reproduce locally, but I'll run some production builds and see what they look like.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19412
Summary:
Ref T13130. See PHI619. Currently, the Herald "Test Console" doesn't pass a "Content Source" to the adapter, so if any rules of the given type execute a "Content source" field rule, they'll fatal.
Provide a content source:
- If possible, use the content source from the most recent transaction.
- Otherwise, build a default "web" content source from the current request.
Test Plan:
- Wrote a "When [content source][is][whatever]" rule for tasks.
- Ran test console against a task.
- Before: got a fatal trying to interact with the content source.
- After: transcript reports sensible content source.
- Also commented out the "xaction" logic to test the fallback behavior.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19411
Summary:
Ref T13103. Locally, I managed to break the data for a bunch of files by doing `git clean -df` in a working copy that I'd updated to a commit from many many years ago. Since `conf/local.json` wasn't on the gitignore list many years ago, this removed it, and I lost my encryption keyring.
I've symlinked my local config to a version-controlled file now to avoid this specific type of creative self-sabotage in the future, but this has exposed a few cases where we could handle things more gracefully.
One issue is that if your favicon is customized but the file it points at can't actually be loaded, we fail explosively and you really can't do anything to move forward except somehow guess that you need to fix your favicon. Instead, recover more gracefully.
Test Plan:
- Configure file encryption.
- Configure a favicon.
- Remove the encryption key from your keyring.
- Purge Phabricator's caches.
- Before: you pretty much dead-end on a fatal that's hard to understand/fix.
- After: everything works except your favicon.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13103
Differential Revision: https://secure.phabricator.com/D19406
Summary:
See <https://twitter.com/HayleyCAnderson/status/988873585363009536>.
Currently, the action dropdown in Differential shows a heavy "X" after "Request Changes" and a heavy checkmark after "Accept Revision".
Although I'm not convinced that the messaging around "Request Changes" is too strong, I do think these marks are out of place in modern Differential. They came from a simpler time when this dropdown had fewer actions, but feel a little weird and inconsistent to me in the modern UI.
Let's try getting rid of them and see how it goes?
Test Plan:
- Viewed these actions in the dropdown, no longer saw the mark icons.
- Grepped for these unicode sequences without getting any other hits.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19405
Summary:
Depends on D19400. Ref T13130. Currently, when you write Herald rules about other Herald rules, you can't pick a rule type or content type, so there's no way to get notified about edits to just global rules (which is the primary driving use case).
Add a "Content type" field to let the rule match rules that affect revisions, tasks, commits, etc.
Add a "Rule type" field to let the rule match global, personal, or object rules.
Test Plan:
- Wrote a global rule for other rules about global Herald rules:
{F5540307}
{F5540308}
- Ran it against itself which matched:
{F5540309}
- Ran it against another rule (not a global rule about Herald rules), which did not match:
{F5540311}
- Also reviewed the fields in those transcripts in more detail to make sure they were extracting matching correctly.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19403
Summary: This word isn't the right word.
Test Plan: Reading?
Reviewers: avivey, amckinley
Reviewed By: avivey
Differential Revision: https://secure.phabricator.com/D19404
Summary:
Depends on D19399. Ref T13130. This adds basic support for writing Herald rules against Herald rules. See T13130 for a lot more detail.
This needs a bit more work to be useful: for example, there's no way to specify the rule type or subject, so you can't say "notify me when global rules are edited" or "notify me when Maniphest rules are edited". I'll add some fields for that in followup changes to actually solve the original use case.
Test Plan:
- Wrote Herald rules against Herald rules.
- Ran them by editing rules and in the test console.
- Verified they sent some mail with `bin/mail list-outbound`.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19400
Summary:
Ref T13065. `mailKey`s are a private secret for each object. In some mail configurations, they help us ensure that inbound mail is authentic: when we send you mail, the "Reply-To" is "T123+456+abcdef".
- The `T123` is the object you're actually replying to.
- The `456` is your user ID.
- The `abcdef` is a hash of your user account with the `mailKey`.
Knowing this hash effectively proves that Phabricator has sent you mail about the object before, i.e. that you legitimately control the account you're sending from. Without this, anyone could send mail to any object "From" someone else, and have comments post under their username.
To generate this hash, we need a stable secret per object. (We can't use properties like the PHID because the secret has to be legitimately secret.)
Today, we store these in `mailKey` properties on the actual objects, and manually generate them. This results in tons and tons and tons of copies of this same ~10 lines of code.
Instead, just store them in the Mail application and generate them on demand. This change also anticipates possibly adding flags like "must encrypt" and "original subject", which are other "durable metadata about mail transmission" properties we may have use cases for eventually.
Test Plan:
- See next change for additional testing and context.
- Sent mail about Herald rules (next change); saw mail keys generate cleanly.
- Destroyed a Herald rule with a mail key, saw the mail properties get nuked.
- Grepped for `getMailKey()` and converted all callsites I could which aren't the copy/pasted boilerplate present in 50 places.
- Used `bin/mail receive-test --to T123` to test normal mail receipt of older-style objects and make sure that wasn't broken.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13065
Differential Revision: https://secure.phabricator.com/D19399
Summary:
Ref T13130. See PHI483. Currently, "Plan Changes + Draft" uses rules like "Plan Changes", not rules like "Draft", and allows "Accept".
This isn't consistent with how "Draft" and "Accept" work in other cases. Make "Plan Changes + Draft" more like "Draft" for consistency.
Also fix a string that didn't have a natural English version.
Test Plan:
- Added a failing build plan.
- Created a revision.
- Loaded the revision before builds completed, saw a nicer piece of text about "waiting for builds" instead of "waiting for 2 build(s)".
- Builds failed, which automatically demoted the reivsion to "Changes Planned + Draft".
- As the author and as a reviewer, verified all the actions available to me made sense (particularly, no "Accept").
- Abandoned the revision to test "Abandoned + Draft".
- As the author and as a reviewer, verified all the actions available to me made sense.
- Reclaimed the revision, then used "Request Review" to send it to "Needs Review". Verified that actions made sense and, e.g., reviewers could now "Accept" normally.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13130
Differential Revision: https://secure.phabricator.com/D19398
Summary:
See D19394. Currently, during first-time setup before you configure "phabricator.base-uri", we may attempt to generate a setup page, try to generate a CSP header for it, and fail to access the environmental config. This causes a too-severe error page ("configure phabricator.base-uri") instead of preflight guidance (like "can't connect to MySQL").
Instead, treat this more like "security.alternate-file-domain" and just bail on CSP if we can't fetch it.
Test Plan: On a fresh (non-explodey laptop) install with critical setup errors (no MySQL installed yet), loaded Phabricator. Before: error about phabricator.base-uri. After: more helpful guidance about installing/configuring MySQL.
Reviewers: amckinley, avivey
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19396
Summary: Depends on D19391. Ref T13126. See that task for some details on what's going on here.
Test Plan:
- Viewed a file which includes lines that were added during the first commit to the repository.
- Before D19391: fatal.
- After D19391: blank.
- After this patch: accurate blame information.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13126
Differential Revision: https://secure.phabricator.com/D19392
Summary:
Ref T13126. When you view a file using the new document engine view and some lines were introduced in the initial commit to the repository, Git renders "^abc123" in the blame output.
We currently don't do anything about this, and later fail to look it up and fatal.
It's also unlikely-but-conceivably-possible to end up here if a commit has not imported yet or has been nuked with `bin/remove destroy`.
Let the whole thing run without fataling even if a `$commit` is missing. Future refinements could improve this behavior.
Test Plan: Viewed a file with lines introduced in the initial commit, got empty blame instead of a fatal.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13126
Differential Revision: https://secure.phabricator.com/D19391
Summary:
Ref T13124. See PHI584. When you create a draft revision and it automatically demotes to "Changes Planned + Draft" because builds fail, let it promote to "Needs Review" automatically if builds pass. Usually, this will be because someone restarted the builds and they worked the second time.
Although I'm a little wary about adding even more state transitions to the diagram in T13110#237736, I think this one is reasonably natural and not ambiguous.
Test Plan:
- Created a failing build plan with a "Throw Exception" step.
- Created a revision which hit the build plan, saw it demote to "Changes Planned" when Harbormaster failed.
- Edited the build plan to remove the "Throw Exception" step, restarted the build, got a pass.
- Saw revision promote again:
{F5526104}
I didn't exhaustively test that the other 40 state transitions still work properly, but I think the scope of this change is small enough that it's unlikely I did much collateral damage.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13124
Differential Revision: https://secure.phabricator.com/D19380
Summary: See discussion in D19379. The 4-tuple of (device, network, address, port) should be unique.
Test Plan: Created lots of duplicate interfaces, bound those interfaces to various services, observed migration script clean things up correctly.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D19388
Summary:
Ref T13124. See PHI593.
When you `arc diff` in a Git or Mercurial repository, we upload some information about the local commits in your working copy which the change was generated from.
In the future (for example, with T1508) we may increase the prominence of this feature.
Provide a stable way to read this information back via the API. This roughly mirrors the information we provide about commits in "diffusion.commit.search", although the latter is less fleshed-out today.
Test Plan: Used `differential.diff.search` to retrieve commit information about Git, Mercurial, and Subversion diffs. (There's no info for Subversion, but it doesn't crash or anything.)
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13124
Differential Revision: https://secure.phabricator.com/D19386
Summary:
Ref T13127. Users with red/green colorblindness may have difficulty using this element in its current incarnation.
We could give it different behavior if the "Accessibility" option is set for red/green colorblind users, but try a one-size-fits-all approach since the red/green aren't wholly clear anwyay.
Test Plan: {F5530050}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13127
Differential Revision: https://secure.phabricator.com/D19385
Summary:
Ref T13124. Ref T13131. Fixes T8953. See PHI512.
When you receieve a notification about an object and then someone hides that object from you (or deletes it), you get a phantom notification which is very difficult to clear.
For now, test that notifications are visible when you open the menu and clear any that are not.
This could be a little more elegant than it is, but the current behavior is very clearly broken. This unbreaks it, at least.
Test Plan:
- As Alice, configured task stuff to notify me (instead of sending email).
- As Bailey, added Alice as a subscriber to a task, then commented on it.
- As Alice, loaded home and saw a notification count. Didn't click it yet.
- As Bailey, set the task to private.
- As Alice, clicked the notification bell menu icon.
- Before change: no unread notifications, bell menu is semi-stuck in a phantom state which you can't clear.
- After change: bad notifications automatically cleared.
{F5530005}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13131, T13124, T8953
Differential Revision: https://secure.phabricator.com/D19384
Summary: Fixes T13129. This at least makes the existing UI work again before we banish Phlux to the shadow realm.
Test Plan: Edited the visibility for a Phlux variable, didn't get an error. Nothing showed up in the edge tables when I made those changes, but at least it doesn't error out anymore.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13129
Differential Revision: https://secure.phabricator.com/D19387
Summary:
The name of networks should be unique.
Also adds support for exact-name queries for AlamanacNetworks.
Test Plan: Applied migration with existing duplicates, saw networks renamed, attempted to add duplicates, got a nice error message.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam
Differential Revision: https://secure.phabricator.com/D19379
Summary:
Ref T13124. See PHI531. When a revision is updated, builds against the older diff tend to stop being relevant. Add an option to abort outstanding older builds automatically.
At least for now, I'm adding this as a build step instead of some kind of special checkbox. An alternate implementation would be some kind of "Edit Options" action on plans with a checkbox like `[X] When this build starts, abort older builds.`
I think adding it as a build step is a bit simpler, and likely to lead to greater consistency and flexibility down the road, make it easier to add options, etc., and since we don't really have any other current use cases for "a bunch of checkboxes". This might change eventually if we add a bunch of checkboxes for some other reason.
The actual step activates //before// the build queues, so it doesn't need to wait in queue before it can actually act. T13088 discusses some plans here if this sticks.
Test Plan:
- Created a "Sleep for 120 seconds" build plan and triggered it with Herald.
- Added an "Abort Older Builds" step.
- Updated a revision several times in a row, saw older builds abort.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13124
Differential Revision: https://secure.phabricator.com/D19376
Summary:
Depends on D19377. Ref T13125. Ref T13124. Ref T13105. Coverage reporting in Diffusion didn't initially survive the transition to Document Engine; restore it.
This adds some tentative/theoretical support for multiple columns of coverage, but no way to actually produce them in the UI. For now, the labels, codes, and colors are hard coded.
Test Plan:
Added coverage with `diffusion.updatecoverage`, saw coverage in the UI:
{F5525542}
Hovered over coverage, got labels and highlighting.
Double-checked labels for "N" (Not Executable) and "U" (Uncovered). See PHI577.
Faked some multi-column coverage, but you can't currently get this yourself today:
{F5525544}
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13125, T13124, T13105
Differential Revision: https://secure.phabricator.com/D19378
Summary:
Depends on D19372. Ref T13124. See PHI505. Currently, if you `!history` a task with a lot of comments, you get output like this:
> alice added a comment.
> bailey added a comment.
> alice added a comment.
> alice added a comment.
>
> AAAA
>
> BBBB
>
> AAAA
>
> AAAA
This is impossible to read. Put the "alice added a comment." headers above the actual comments for comments after the first.
These types of mail messages are unusual, but occur in several cases:
- The new `!history` command.
- Multiple comments on a draft revision before it promotes out of draft.
- (Probably?) Conduit API updates which submit multiple comment transactions for some reason.
Test Plan: Used `bin/mail receive-test` to send a `!history` command to a task, saw a much more readable rendering of the transaction log in the resulting email.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13124
Differential Revision: https://secure.phabricator.com/D19373
Summary:
See PHI505. Ref T13124. If you're an agent of a hostile state trying to exfiltrate corporate secrets, you might find yourself foiled if Phabricator is secured behind a VPN.
To assist users in this situation, provide a "!history" command which will dump the entire history of an object in a nice text format and get through the troublesome VPN.
Some issues with this:
- You currently get all the "X added a comment." up top, and then all the comments below. This isn't terribly useful.
- This goes through the "Must Encrypt" flag, but possibly should not? (On the other hand, this is a pretty willful way to bypass it the flag.)
Test Plan: Used `bin/mail receive-test ...` to send `!history` commands, got somewhat-useful response mail.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13124
Differential Revision: https://secure.phabricator.com/D19372
Summary:
Depends on D19370. See T13124. See PHI549. The particular install in PHI549 migrated a large amount of data via the fallback hunk migration script, which does not compress hunks.
Add a mode to `bin/differential migrate-hunk` that amounts to "compress all the hunks which would benefit from compression".
Test Plan: Ran `bin/differential migrate-hunk` with `--auto`, `--all`, `--to`, `--id`, and `--dry-run` in various mixtures. Forced a bunch of hunks to raw ("byte") format, saw it cleanly upgrade them to compressed ("gzde") format.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19371
Summary:
Depends on D19369. Ref T13120. Add a flag to migrate every hunk.
This isn't terribly useful on its own, but I'm going to add an `--auto` flag next so that you can run `--auto --all` to migrate hunks to the preferred hunk format.
Test Plan: Ran `bin/differential migrate-hunk --all --to text`.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13120
Differential Revision: https://secure.phabricator.com/D19370
Summary: This is a good spelling, but maybe a better spelling is possible.
Test Plan: hmmm
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19369
Summary: Ref T13076. This will be used by the metric collection system to iterate over the cluster devices.
Test Plan: Created some cluster and non-cluster devices, searched and saw expected results.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T13076
Differential Revision: https://secure.phabricator.com/D19368
Summary:
Ref T13120. See PHI571. Fixes T5024. This adds a "View as Query" action to workboard columns, which builds a query in Maniphest that has the current query constraints plus an additional constraint to select only tasks in the specified column.
This is a normal query and can be turned into a dashboard panel, added to a menu, edited, saved as a link, etc.
Much of the complexity here is that finding tasks in a given column isn't entirely straightforward because of how board layout works: when you create a task, it isn't immediately placed in columns. It's only actually added to the "Backlog" column on any boards when someone looks at the board.
To get the right behavior, we must do "board layout" for any queried columns before we can constrain results. This isn't enormously efficient, but should be OK for reasonable boards.
Test Plan:
- Used "View as Query" for normal columns and milestome columns, got appropriate queries in Maniphest.
- Applied filters to the board (e.g., "Priorities: wishlist"), then used "View As Query" and had my custom filters respected.
- Queried some large boards/columns with more than a thousand tasks, got results back within a second or so.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13120, T5024
Differential Revision: https://secure.phabricator.com/D19366
Summary:
See PHI565. Ref T13120. Although this older log is on the chopping block (see T13088), there's some migration guidance and other complexity around just replacing it.
Until it gets replaced, make clicking the "number of lines" elements respect the current "Build Generation" setting. Prior to this change, clicking the links would lose the generation information and jump you to the most recent build generation.
Also fix some collateral damage from T13105 where we ended up with white text on a white background in some cases.
Test Plan:
- Restarted a build to get multiple generations.
- On each generation, clicked the various "25", "50", etc., links.
- Saw generation and log window sizes both respected by the links.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13120
Differential Revision: https://secure.phabricator.com/D19367
