1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2025-03-02 23:49:19 +01:00
Commit graph

721 commits

Author SHA1 Message Date
epriestley
10162ad43b Support an SSH error log
Summary: Ref T13624. Depends on D21578. In "sshd" subprocess contexts, use "PhutilErrorLog" to direct errors to both stderr and, if configured, a logfile on disk.

Test Plan:
  - Confiugured an error log.
  - Forced `ssh-auth` to fatal.
  - Saw errors on stderr and in log.

Maniphest Tasks: T13624

Differential Revision: https://secure.phabricator.com/D21579
2021-02-26 14:54:54 -08:00
epriestley
04c1f67a02 Add "M<digit>" and "P<digit>" to the default Remarkup ignore list
Summary: Ref T13575. Particularly with the new Apple silicon, I think there are enough domain collisions for `M1`, `M2`, `P1`, etc., to justify adding them to the default ignore list.

Test Plan: Created a mock, then wrote a comment referencing an object on the list (`M1`) and an object not on the list (`T1`). Got text and a link respectively.

Maniphest Tasks: T13575

Differential Revision: https://secure.phabricator.com/D21507
2021-01-11 10:54:41 -08:00
epriestley
a27c83757d Remove ancient "phd.trace" and "phd.verbose" configuration options
Summary:
Ref T13556. These options are very old and effectively obsoleted by "bin/phd debug [--trace]". I haven't used either option diagnostically in many years, and they aren't mentioned in the documentation.

Remove them to simplify configuration, and because "phd.trace" doesn't work anyway and likely hasn't for a long time -- it has specific issues with TTY detection (see T13556).

Test Plan: Grepped for "phd.trace" and "phd.verbose". Ran "bin/phd debug [--trace]" and saw verbose/trace output.

Maniphest Tasks: T13556

Differential Revision: https://secure.phabricator.com/D21426
2020-07-23 12:31:32 -07:00
epriestley
f86d822a37 Update MySQL schema inspection code for deprecation of integer display widths
Summary:
Fixes T13536. See that task for discussion.

Older versions of MySQL (roughly, prior to 8.0.19) emit "int(10)" types. Newer versions emit "int" types. Accept these as equivalent.

Test Plan: Ran `bin/storage upgrade --force` against MySQL 8.0.11 and 8.0.20. Got clean adjustment lists on both versions.

Maniphest Tasks: T13536

Differential Revision: https://secure.phabricator.com/D21265
2020-05-18 12:10:31 -07:00
epriestley
b89e6c0fa9 Add "idea://" to the upstream editor whitelist
Summary: This supports the IntelliJ IDEA editor.

Test Plan:
  - Looked at the editor settings panel, saw "idea://".
  - Set my editor pattern to "idea://a?b".
  - (Did not actually install IntelliJ IDEA.)

Differential Revision: https://secure.phabricator.com/D21206
2020-05-01 12:56:35 -07:00
epriestley
45665dd3b4 Hide "notification.servers" configuration and don't follow redirects from Aphlict
Summary:
See <https://hackerone.com/reports/850114>.

An attacker with administrator privileges can configure "notification.servers" to connect to internal services, either directly or with chosen parameters by selecting an attacker-controlled service and having it issue a "Location" redirect.

Generally, we allow this attack to occur. The same administrator can use an authentication provider or a VCS repository to perform the same attack, and we can't reasonably harden these workflows without breaking things that users expect to be able to do.

There's no reason this particular variation of the attack needs to be allowable, though, and the current behavior isn't consistent with how other similar things work.

  - Hide the "notification.servers" configuration, which also locks it. This is similar to other modern service/server configuration.
  - Don't follow redirects on these requests. Aphlict should never issue a "Location" header, so if we encounter one something is misconfigured. Declining to follow this header likely makes the issue easier to debug.

Test Plan:
  - Viewed configuration in web UI.
  - Configured a server that "Location: ..." redirects, got a followed redirect before and a failure afterward.

{F7365973}

Differential Revision: https://secure.phabricator.com/D21123
2020-04-15 07:00:51 -07:00
epriestley
b52fa96238 Disable automatic decoding of "Content-Encoding" responses during "Accept-Encoding" setup test
Summary:
Ref T13507. Now that we handle processing of "Content-Encoding: gzip" headers by default, this setup check can get a decompressed body back. Since it specifically wants a raw body back, disable this behavior.

Also, "@" a couple things which can get in the way if they fail now that error handling is more aggressive about throwing on warnings.

Test Plan: Ran setup check after other changes in T13507, got clean result.

Maniphest Tasks: T13507

Differential Revision: https://secure.phabricator.com/D21122
2020-04-15 06:28:29 -07:00
epriestley
6b05d2be28 Add a setup warning to detect "SetInputFilter DEFLATE" and other "Content-Encoding" request mangling
Summary: Ref T13507. See that task for discussion.

Test Plan: Faked different response behaviors and hit both variations of this error.

Maniphest Tasks: T13507

Differential Revision: https://secure.phabricator.com/D21116
2020-04-14 14:48:43 -07:00
epriestley
33b73d887a If daemon running-as-user setup check fails its query, don't bother with it
Summary:
See <https://discourse.phabricator-community.org/t/upgrade-from-sep-30-2016/3702/>. A user performing an upgrade from 2016 to 2020 ran into an issue where this setup query is overheating.

This is likely caused by too many rows changing state during query execution, but the particulars aren't important since this setup check isn't too critical and will catch the issue eventually. It's fine to just move on if this query fails for any reason.

Test Plan: Forced the query to overheat, loaded setup issues, got overheating fatal. Applied patch, no more fatal.

Differential Revision: https://secure.phabricator.com/D21057
2020-04-03 16:18:55 -07:00
epriestley
149155ee20 Whitelist "vscode://" as an allowed Editor protocol
Summary:
See PHI1647, which asks for "vscode://" to be a configurable protocol on hosted Phacility instances.

I made the configuration editable in D21008, but this can reasonably just come upstream too.

Test Plan: Viewed config in Config, set my editor URI to `vscode://blahblah`.

Differential Revision: https://secure.phabricator.com/D21009
2020-02-20 12:45:35 -08:00
epriestley
64cc4fe915 Add a test to verify that all routing maps are plausibly valid, and remove some dead routes
Summary:
Previously, see D20999. See also <https://discourse.phabricator-community.org/t/the-phutil-library-phutil-has-not-been-loaded/3543/>.

There are a couple of dead "Config" routes after recent changes. Add test coverage to make sure routes all point somewhere valid, then remove all the dead routes that turned up.

Test Plan: Ran tests, saw failures. Removed dead routes, got clean tests.

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D21000
2020-02-14 18:06:24 -08:00
epriestley
4790a3d94b Stop trying to version-check libphutil in "Config"
Summary:
Ref T13395. No library with this name loads any more, so we can't version check it.

(Ideally, the version check stuff would be more graceful when it fails now, since it's required to load "Config" after I moved it off a separate page.)

Test Plan: Loaded "Config".

Maniphest Tasks: T13395

Differential Revision: https://secure.phabricator.com/D20995
2020-02-14 08:44:50 -08:00
epriestley
530145ba3b Give "Config" a full-width, hierarchical layout
Summary:
Depends on D20933. Ref T13362. This reorganizes Config a bit and attempts to simplify it.

Subsections are now in a landing page console and groupings have been removed. We "only" have 75 values you can edit from the web UI nowadays, which is still a lot, but less overwhelming than it was in the past. And the trend is generally downward, as config is removed/simplified or moved into application settings.

This also gets rid of the "gigantic blobs of JSON in the UI".

Test Plan: Browsed all Config sections.

Maniphest Tasks: T13362

Differential Revision: https://secure.phabricator.com/D20934
2020-02-04 06:59:51 -08:00
epriestley
cb481f36c5 Carve out a separate "Services" section of Config
Summary: Depends on D20930. Ref T13362. Put all the "Services" parts of Config in their own section.

Test Plan: Clicked through each section. This is just an organization / UI change with no significant behavioral impact.

Maniphest Tasks: T13362

Differential Revision: https://secure.phabricator.com/D20931
2020-02-04 06:47:31 -08:00
epriestley
a72ade9475 Carve out a separate "Modules/Extensions" section of Config
Summary:
Ref T13362. Config is currently doing a ton of stuff and fairly overwhelming. Separate out "Modules/Extensions" so it can live in its own section.

(This stuff is mostly useful for development and normal users rarely need to end up here.)

Test Plan: Visited seciton, clicked around. This is just a visual change.

Maniphest Tasks: T13362

Differential Revision: https://secure.phabricator.com/D20930
2020-02-04 06:41:55 -08:00
epriestley
33c534f9b7 Extend Config to full-width
Summary:
Ref T13362. Some applications moved to fixed-width a while ago but I was generally unsatisfied with where they ended up and have been pushing them back to full-width.

Push Config back to full-width. Some of the subpages end up a little weird, but this provides more space to work with to make some improvements, like makign `maniphest.statuses` more legible in the UI>

Test Plan: Grepped for `setFixed(`, updated each page in `/config/`. Browsed each controller, saw workable full-width UIs.

Maniphest Tasks: T13362

Differential Revision: https://secure.phabricator.com/D20925
2019-11-25 16:18:41 -08:00
epriestley
18da346972 Add additional flags to "bin/repository rebuild-identities" to improve flexibility
Summary:
Ref T13444. Repository identities have, at a minimum, some bugs where they do not update relationships properly after many types of email address changes.

It is currently very difficult to fix this once the damage is done since there's no good way to inspect or rebuild them.

Take some steps toward improving observability and providing repair tools: allow `bin/repository rebuild-identities` to effect more repairs and operate on identities more surgically.

Test Plan: Ran `bin/repository rebuild-identities` with all new flags, saw what looked like reasonable rebuilds occur.

Maniphest Tasks: T13444

Differential Revision: https://secure.phabricator.com/D20911
2019-11-19 09:39:48 -08:00
epriestley
535c8e6bdc Remove the "ONLY_FULL_GROUP_BY" SQL mode setup warning and change the setup key for "STRICT_ALL_TABLES"
Summary:
Ref T13404. Except for one known issue in Multimeter, Phabricator appears to function properly in this mode. It is broadly desirable that we run in this mode; it's good on its own, and enabled by default in at least some recent MySQL.

Additionally, "ONLY_FULL_GROUP_BY" and "STRICT_ALL_TABLES" shared a setup key, so ignoring one would ignore both. Change the key so that existing ignores on "ONLY_FULL_GROUP_BY" do not mask "STRICT_ALL_TABLES" warnings.

Test Plan: Grepped for `ONLY_FULL_GROUP_BY`.

Maniphest Tasks: T13404

Differential Revision: https://secure.phabricator.com/D20791
2019-09-09 12:17:51 -07:00
epriestley
f16365ed07 Weaken the guidance recommending that installs enable "STRICT_ALL_TABLES"
Summary: Ref T13404. Enabling "STRICT_ALL_TABLES" is good, but if you don't want to bother it doesn't matter too much. All upstream development has been on "STRICT_ALL_TABLES" for a long time.

Test Plan: {F6847839}

Maniphest Tasks: T13404

Differential Revision: https://secure.phabricator.com/D20790
2019-09-09 12:17:05 -07:00
Aviv Eyal
318e8ebdac Allow bin/config to create config file
Summary:
See D20779, https://discourse.phabricator-community.org/t/3089. `bin/config set` complains about
missing config file as if it's un-writable.

Test Plan: run `bin/config set` with missing, writable, unwritable conf.json and parent dir.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D20788
2019-09-08 00:16:19 +00:00
epriestley
d9badba147 Give "bin/config" a friendlier error message if "local.json" is not writable
Summary: Ref T13403. We currently emit a useful error message, but it's not tailored and has a stack trace. Since this is a relatively routine error and on the first-time-setup path, tailor it so it's a bit nicer.

Test Plan:
  - Ran `bin/config set ...` with an unwritable "local.json".
  - Ran `bin/config set ...` normally.

Maniphest Tasks: T13403

Differential Revision: https://secure.phabricator.com/D20779
2019-09-03 12:47:06 -07:00
epriestley
c6642213d5 Straighten out replication/cache behavior in "bin/storage dump"
Summary:
Fixes T13336.

  - Prevent `--no-indexes` from being combined with `--for-replica`, since combining these options can only lead to heartbreak.
  - In `--for-replica` mode, dump caches too. See discussion in T13336. It is probably "safe" to not dump these today, but fragile and not correct.
  - Mark the "MarkupCache" table as having "Cache" persistence, not "Data" persistence (no need to back it up, since it can be fully regenerated from other datasources).

Test Plan: Ran `bin/storage dump` with various combinations of flags.

Maniphest Tasks: T13336

Differential Revision: https://secure.phabricator.com/D20743
2019-08-28 08:25:40 -07:00
epriestley
006cb659cb Make the success message from "bin/config" more clear
Summary:
Ref T13373. When you "bin/config set x ..." a value, the success message ("Set x ...") is somewhat ambiguous and can be interpreted as "First, you need to set x..." rather than "Success, wrote x...".

Make the messaging more explicit. Also make this string more translatable.

Test Plan: Ran `bin/config set ...` with various combinations of flags, saw more clear messaging.

Maniphest Tasks: T13373

Differential Revision: https://secure.phabricator.com/D20711
2019-08-12 12:50:03 -07:00
epriestley
9ab5f59ca2 Export "date" and "remarkup" custom fields to Excel + "zip" extension check
Summary:
Fixes T13342. This does a few different things, although all of them seem small enough that I didn't bother splitting it up:

  - Support export of "remarkup" custom fields as text. There's some argument here to export them in some kind of structure if the target is JSON, but it's hard for me to really imagine we'll live in a world some day where we really regret just exporting them as text.
  - Support export of "date" custom fields as dates. This is easy except that I added `null` support.
  - If you built PHP from source without "--enable-zip", as I did, you can hit the TODO in Excel exports about "ZipArchive". Since I had a reproduction case, test for "ZipArchive" and give the user a better error if it's missing.
  - Add a setup check for the "zip" extension to try to avoid getting there in the first place. This is normally part of PHP so I believe users generally won't hit it, I just hit it because I built from source. See also T13232.

Test Plan:
  - Added a custom "date" field. On tasks A and B, set it to null and some non-null value. Exported both tasks to Excel/JSON/text, saw null and a date, respectively.
  - Added a custom "remarkup" field, exported some values, saw the values in Excel.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13342

Differential Revision: https://secure.phabricator.com/D20658
2019-07-18 09:59:20 -07:00
epriestley
65bc481c91 Remove "phd.pid-directory" configuration and stop passing "piddir" to daemons
Summary: Ref T13321. The daemons no longer write PID files, so we no longer need to pass any of this stuff to them.

Test Plan: Grepped for affected symbols.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13321

Differential Revision: https://secure.phabricator.com/D20608
2019-06-24 11:28:58 -07:00
epriestley
8747204807 Clean up "phabricator.timezone" configuration instructions a little bit
Summary: These instructions are fairly old and can be a little fancier and more clear in the context of modern Phabricator. Drop the reference to "HPHP", link the actual timezone list, wordsmith a little.

Test Plan: d( O_o )b

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20560
2019-05-30 15:08:11 -07:00
epriestley
c648077625 Don't warn about a locked database value after users run "bin/auth lock"
Summary: Ref T7667. Also one text fix.

Test Plan: Ran `bin/auth lock`, didn't get a replacement setup issue.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T7667

Differential Revision: https://secure.phabricator.com/D20447
2019-04-17 19:54:11 -07:00
Austin McKinley
0583f6dc50 Some formatting changes for showing auth provider config guidance
Summary:
Ref T7667. On the road to locking the auth config, also clean up some minor UI issues:

* Only show the warning about not Phacility instance auth if the user isn't a manager (see next diff).
* When rendering more than one warning in the guidance, add bullets.
* I didn't like the text in the `auth.config-lock` config setting.

Test Plan: Loaded the page, saw more reasonable-looking guidance: {F6369405}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T7667

Differential Revision: https://secure.phabricator.com/D20400
2019-04-17 11:08:16 -07:00
epriestley
5a2d0f0437 Update documentation for "uri.allowed-protocols"
Summary: See <https://discourse.phabricator-community.org/t/download-tasks-and-others-as-excel-throw-exception/>.

Test Plan: Read config.

Reviewers: amckinley, avivey

Reviewed By: avivey

Subscribers: avivey

Differential Revision: https://secure.phabricator.com/D20430
2019-04-16 05:23:13 -07:00
Austin McKinley
0f9776fb58 Add a workflow and a new config option for locking authentication providers
Summary:
Ref T7667. Adds new flows `bin/auth lock` and `bin/auth unlock` to prevent compromised administrator accounts from doing additional damage by altering the authentication provider configuration.

Note that this currently doesn't actually do anything because we aren't checking this config key in any of the edit controllers yet.

Test Plan: Ran `lock` and `unlock`, checked for correct DB state, observed expected setup warning.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T7667

Differential Revision: https://secure.phabricator.com/D20394
2019-04-10 16:14:48 -07:00
epriestley
ea6c0c9bde Refine the "Mangled Webserver Response" setup check
Summary:
Ref T13259. In some configurations, making a request to ourselves may return a VPN/Auth response from some LB/appliance layer.

If this response begins or ends with whitespace, we currently detect it as "extra whitespace" instead of "bad response".

Instead, require that the response be nearly correct (valid JSON with some extra whitespace, instead of literally anything with some extra whitespace) to hit this specialized check. If we don't hit the specialized case, use the generic "mangled" response error, which prints the actual body so you can figure out that it's just your LB/auth thing doing what it's supposed to do.

Test Plan:
  - Rigged responses to add extra whitespace, got "Extra Whitespace" (same as before).
  - Rigged responses to add extra non-whitespace, got "Mangled Junk" (same as before).
  - Rigged responses to add extra whitespace and extra non-whitespace, got "Mangled Junk" with a sample of the document body instead of "Extra Whitespace" (improvement).

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13259

Differential Revision: https://secure.phabricator.com/D20235
2019-03-05 12:58:32 -08:00
epriestley
b28b05342b Simplify one "array_keys/range" -> "phutil_is_natural_list()" in "phabricator/"
Summary: Depends on D20213. Simplify this idiom.

Test Plan: Squinted hard; `grep array_keys | grep range`.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20215
2019-02-28 07:29:36 -08:00
epriestley
5310f1cdd9 Remove all whitespace options/configuration everywhere
Summary:
Depends on D20181. Depends on D20182. Fixes T3498. Ref T13161. My claim, at least, is that D20181 can be tweaked to be good enough to throw away this "feature" completely.

I think this feature was sort of a mistake, where the ease of access to `diff -bw` shaped behavior a very long time ago and then the train just ran a long way down the tracks in the same direction.

Test Plan: Grepped for `whitespace`, deleted almost everything. Poked around the UI a bit. I'm expecting the whitespace changes to get some more iteration this week so I not being hugely pedantic about testing this stuff exhaustively.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13161, T3498

Differential Revision: https://secure.phabricator.com/D20185
2019-02-19 13:09:29 -08:00
epriestley
5892c78986 Replace all "setQueryParam()" calls with "remove/replaceQueryParam()"
Summary: Ref T13250. See D20149. Mostly: clarify semantics. Partly: remove magic "null" behavior.

Test Plan: Poked around, but mostly just inspection since these are pretty much one-for-one.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: yelirekim

Maniphest Tasks: T13250

Differential Revision: https://secure.phabricator.com/D20154
2019-02-14 11:56:39 -08:00
epriestley
eb73cb68ff Raise a setup warning when locked configuration has a configuration value stored in the database
Summary:
Ref T13249. See <https://discourse.phabricator-community.org/t/configuring-the-number-of-taskmaster-daemons/2394/>.

Today, when a configuration value is "locked", we prevent //writes// to the database. However, we still perform reads. When you upgrade, we generally don't want a bunch of your configuration to change by surprise.

Some day, I'd like to stop reading locked configuration from the database. This would defuse an escalation where an attacker finds a way to write to locked configuration despite safeguards, e.g. through SQL injection or policy bypass. Today, they could write to `cluster.mailers` or similar and substantially escalate access. A better behavior would be to ignore database values for `cluster.mailers` and other locked config, so that these impermissible writes have no effect.

Doing this today would break a lot of installs, but we can warn them about it now and then make the change at a later date.

Test Plan:
  - Forced a `phd.taskmasters` config value into the database.
  - Saw setup warning.
  - Used `bin/config delete --database phd.taskmasters` to clear the warning.
  - Reviewed documentation changes.
  - Reviewed `phd.taskmasters` documentation adjustment.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13249

Differential Revision: https://secure.phabricator.com/D20159
2019-02-13 12:27:48 -08:00
epriestley
9f5e6bee90 Make the default behavior of getApplicationTransactionCommentObject() "return null" instead of "throw"
Summary:
Depends on D20115. See <https://discourse.phabricator-community.org/t/transaction-search-endpoint-does-not-work-on-differential-diffs/2369/>.

Currently, `getApplicationTransactionCommentObject()` throws by default. Subclasses must override it to `return null` to indicate that they don't support comments.

This is silly, and leads to a bunch of code that does a `try / catch` around it, and at least some code (here, `transaction.search`) which doesn't `try / catch` and gets the wrong behavior as a result.

Just make it `return null` by default, meaning "no support for comments". Then remove the `try / catch` stuff and all the `return null` implementations.

Test Plan:
  - Grepped for `getApplicationTransactionCommentObject()`, fixed each callsite / definition.
  - Called `transaction.search` on a diff with transactions (i.e., not a sourced-from-commit diff).

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: jbrownEP

Differential Revision: https://secure.phabricator.com/D20121
2019-02-07 14:56:38 -08:00
epriestley
4fa5a2421e Add formal setup guidance warning that "feed.http-hooks" will be removed in a future version of Phabricator
Summary: Depends on D20114. This is on the way out, so make that explicitly clear.

Test Plan: Read setup issue and configuration option.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20115
2019-02-07 14:54:09 -08:00
epriestley
55286d49e8 Clarify "metamta.default-address" instructions and lock the option
Summary: This option no longer needs to be configured if you configure inbound mail (and that's the easiest setup approach in a lot of cases), so stop telling users they have to set it up.

Test Plan: Read documentation and configuration help.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20104
2019-02-06 16:03:49 -08:00
epriestley
87b0ef8839 Remove "iconv" PHP extension dependency
Summary: Depends on D20069. Ref T13232. This is a very, very weak dependency and we can reasonably polyfill it.

Test Plan: Grepped for `iconv` in libphutil, arcanist, and Phabricator.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13232

Differential Revision: https://secure.phabricator.com/D20070
2019-01-30 19:46:58 -08:00
epriestley
a24e6deb57 Read "$_POST" before hooking the profiler, and remove "aphront.default-application-configuration-class"
Summary:
Ref T4369. Ref T12297. Ref T13242. Ref PHI1010. I want to take a quick look at `transaction.search` and see if there's anything quick and obvious we can do to improve performance.

On `secure`, the `__profile__` flag does not survive POST like it's supposed to: when you profile a page and then submit a form on the page, the result is supposed to be profiled. The intent is to make it easier to profile Conduit calls.

I believe this is because we're hooking the profiler, then rebuilding POST data a little later -- so `$_POST['__profile__']` isn't set yet when the profiler checks.

Move the POST rebuild a little earlier to fix this.

Also, remove the very ancient "aphront.default-application-configuration-class". I believe this was only used by Facebook to do CIDR checks against corpnet or something like that. It is poorly named and long-obsolete now, and `AphrontSite` does everything we might reasonably have wanted it to do.

Test Plan: Poked around locally without any issues. Will check if this fixes the issue on `secure`.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13242, T12297, T4369

Differential Revision: https://secure.phabricator.com/D20046
2019-01-30 06:22:41 -08:00
epriestley
98c4cdc5be Make the "PHP 7" setup warning more explicit about what it means
Summary: See <https://discourse.phabricator-community.org/t/minimum-php-versions-supported-by-latest-stable-phabricator/2314/3>.

Test Plan: o~o

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D20027
2019-01-24 15:09:00 -08:00
epriestley
98bf3a950d Add setup warnings for "local_infile" (MySQL Server) and "mysql[i].allow_local_infile" (PHP Client)
Summary: Ref T13238. Warn users about these horrible options and encourage them to defuse them.

Test Plan: Hit both warnings, fixed the issues, issues went away.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13238

Differential Revision: https://secure.phabricator.com/D19999
2019-01-18 19:49:48 -08:00
epriestley
c125ab7a42 Remove "metamta.*.subject-prefix" options
Summary:
In ~2012, the first of these options was added because someone who hates dogs and works at Asana also hated `[Differential]` in the subject line. The use case there was actually //removing// the text, not changing it, but I made the prefix editable since it seemed like slightly less of a one-off.

These options are among the dumbest and most useless config options we have and very rarely used, see T11760. A very small number of instances have configured one of these options.

Newer applications stopped providing these options and no one has complained.

You can get the same effect with `translation.override`. Although I'm not sure we'll keep that around forever, it's a reasonable replacement today. I'll call out an example in the changelog to help installs that want to preserve this option.

If we did want to provide this, it should just be in {nav Applications > Settings} for each application, but I think it's wildly-low-value and "hack via translations" or "local patch" are entirely reasonable if you really want to change these strings.

Test Plan: Grepped for `subject-prefix`.

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19993
2019-01-17 19:18:50 -08:00
epriestley
a37b28ef79 Prevent inbound processing of the "void/placeholder" address and other reserved addresses
Summary:
Depends on D19952. Ref T13222. Never process mail targets if they match:

  - The "default" address which we send mail "From".
  - The "void" address which we use as a placholder "To" when we only have "CC" addresses.
  - Any address from a list of reserved/administrative names.

The first two prevent loops. The third one prevents abuse.

There's a reasonably well-annotated list of reservations and reasons here:

https://webmasters.stackexchange.com/questions/104811/is-there-any-list-of-email-addresses-reserved-because-of-security-concerns-for-a

Stuff like `support@` seems fine; stuff like `ssladmin@` might let you get SSL certs issued for a domain you don't control.

Also, forbid users from creating application emails with these reserved addresses.

Finally, build the default and void addresses somewhat more cleverly.

Test Plan: Added unit tests, tried to configured reserved addresses, hit the default/void cases manually with `bin/mail receive-test`.

Reviewers: amckinley

Reviewed By: amckinley

Subscribers: olexiy.myronenko

Maniphest Tasks: T13222

Differential Revision: https://secure.phabricator.com/D19953
2019-01-16 12:28:53 -08:00
epriestley
e3aa043a02 Allow multiple mail receivers to react to an individual email
Summary:
Fixes T7477. Fixes T13066. Currently, inbound mail is processed by the first receiver that matches any "To:" address. "Cc" addresses are ignored.

**To, CC, and Multiple Receivers**

Some users would like to be able to "Cc" addresses like `bugs@` instead of having to "To" the address, which makes perfect sense. That's the driving use case behind T7477.

Since users can To/Cc multiple "create object" or "update object" addresses, I also wanted to make the behavior more general. For example, if you email `bugs@` and also `paste@`, your mail might reasonably make both a Task and a Paste. Is this useful? I'm not sure. But it seems like it's pretty clearly the best match for user intent, and the least-surprising behavior we can have. There's also no good rule for picking which address "wins" when two or more match -- we ended up with "address order", which is pretty arbitrary since "To" and "Cc" are not really ordered fields.

One part of this change is removing `phabricator.allow-email-users`. In practice, this option only controlled whether users were allowed to send mail to "Application Email" addresses with a configured default author, and it's unlikely that we'll expand it since I think the future of external/grey users is Nuance, not richer interaction with Maniphest/Differential/etc. Since this option only made "Default Author" work and "Default Author" is optional, we can simplify behavior by making the rule work like this:

  - If an address specifies a default author, it allows public email.
  - If an address does not, it doesn't.

That's basically how it worked already, except that you could intentionally "break" the behavior by not configuring `phabricator.allow-email-users`. This is a backwards compatility change with possible security implications (it might allow email in that was previously blocked by configuration) that I'll call out in the changelog, but I suspect that no installs are really impacted and this new behavior is generally more intuitive.

A somewhat related change here is that each receiver is allowed to react to each individual email address, instead of firing once. This allows you to configure `bugs-a@` and `bugs-b@` and CC them both and get two tasks. Useful? Maybe not, but seems like the best execution of intent.

**Sender vs Author**

Adjacently, T13066 described an improvement to error handling behavior here: we did not distinguish between "sender" (the user matching the email "From" address) and "actor" (the user we're actually acting as in the application). These are different when you're some internet rando and send to `bugs@`, which has a default author. Then the "sender" is `null` and the "author" is `@bugs-robot` or whatever (some user account you've configured).

This refines "Sender" vs "Author". This is mostly a purity/correctness change, but it means that we won't send random email error messages to `@bugs-robot`.

Since receivers are now allowed to process mail with no "sender" if they have some default "actor" they would rather use instead, it's not an error to send from an invalid address unless nothing processes the mail.

**Other**

This removes the "abundant receivers" error since this is no longer an error.

This always sets "external user" mail recipients to be unverified. As far as I can tell, there's no pathway by which we send them email anyway (before or after this change), although it's possible I'm missing something somewhere.

Test Plan:
I did most of this with `bin/mail receive-test`. I rigged the workflow slightly for some of it since it doesn't support multiple addresses or explicit "CC" and adding either would be a bit tricky.

These could also be tested with `scripts/mail/mail_handler.php`, but I don't currently have the MIME parser extension installed locally after a recent upgrade to Mojave and suspect T13232 makes it tricky to install.

- Ran unit tests, which provide significant coverage of this flow.
- Sent mail to multiple Maniphest application emails, got multiple tasks.
- Sent mail to a Maniphest and a Paste application email, got a task and a paste.
- Sent mail to a task.
  - Saw original email recorded on tasks. This is a behavior particular to tasks.
- Sent mail to a paste.
- Sent mail to a mock.
- Sent mail to a Phame blog post.
- Sent mail to a Legalpad document.
- Sent mail to a Conpherence thread.
- Sent mail to a poll.
- This isn't every type of supported object but it's enough of them that I'm pretty confident I didn't break the whole flow.
- Sent mail to an object I could not view (got an error).
- As a non-user, sent mail to several "create an object..." addresses.
  - Addresses with a default user worked (e.g., created a task).
  - Addresses without a default user did not work.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13066, T7477

Differential Revision: https://secure.phabricator.com/D19952
2019-01-16 12:28:02 -08:00
Austin McKinley
b98d46ce7d Resurrect setup check for cluster.mailers
Summary:
D19940 removed this file entirely, which has led to at least one user who was unsure how to proceed now that `cluster.mailers` is required for outbound mail: https://discourse.phabricator-community.org/t/invalid-argument-supplied-for-foreach-phabricatormetamtamail-php/2287

This isn't //always// a setup issue for installs that don't care about sending mail, but this at least this gives a sporting chance to users who don't follow the changelogs.

Also, I'm not sure if there's a way to use `pht()` to generate links; right now the phurl is just in plain text.

Test Plan: Removed `cluster.mailers` config; observed expected setup issue.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D19964
2019-01-16 12:14:36 -08:00
epriestley
0b8f24dfd3 Fix bad "SMTP" and "cluster.mailers" default value
Summary: See note in D19964.

Test Plan: O_o

Reviewers: amckinley

Reviewed By: amckinley

Differential Revision: https://secure.phabricator.com/D19966
2019-01-14 13:10:33 -08:00
epriestley
84f94994ad Remove "metamta.insecure-auth-with-reply-to" Config option
Summary:
Ref T7477. This option was added in D842 in 2011, to support a specific narrow use case at Quora with community moderators using some kind of weird Gmail config.

I don't recall it ever coming up since then, and a survey of a subset of hosted instances (see T11760) reveals that no instances are using this option today. Presumably, even Quora has completed the onboarding discussed in D842, if they still use Phabricator. This option generally does not seem very useful outside of very unusual/narrow cases like the one Quora had.

This would be relatively easy to restore as a local patch if installs //do// need it, but I suspect this has no use cases anywhere.

Test Plan: Grepped for option, blame-delved to figure out why we added it in the first place, surveyed instances for usage.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T7477

Differential Revision: https://secure.phabricator.com/D19949
2019-01-04 13:56:39 -08:00
epriestley
1f4cf23455 Remove "phabricator.csrf-key" and upgrade CSRF hashing to SHA256
Summary:
Ref T12509.

  - Remove the "phabricator.csrf-key" configuration option in favor of automatically generating an HMAC key.
  - Upgrade two hasher callsites (one in CSRF itself, one in providing a CSRF secret for logged-out users) to SHA256.
  - Extract the CSRF logic from `PhabricatorUser` to a standalone engine.

I was originally going to do this as two changes (extract logic, then upgrade hashes) but the logic had a couple of very silly pieces to it that made faithful extraction a little silly.

For example, it computed `time_block = (epoch + (offset * cycle_frequency)) / cycle_frequency` instead of `time_block = (epoch / cycle_frequency) + offset`. These are equivalent but the former was kind of silly.

It also computed `substr(hmac(substr(hmac(secret)).salt))` instead of `substr(hmac(secret.salt))`. These have the same overall effect but the former is, again, kind of silly (and a little bit materially worse, in this case).

This will cause a one-time compatibility break: pages loaded before the upgrade won't be able to submit contained forms after the upgrade, unless they're open for long enough for the Javascript to refresh the CSRF token (an hour, I think?). I'll note this in the changelog.

Test Plan:
  - As a logged-in user, submitted forms normally (worked).
  - As a logged-in user, submitted forms with a bad CSRF value (error, as expected).
  - As a logged-out user, hit the success and error cases.
  - Visually inspected tokens for correct format.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12509

Differential Revision: https://secure.phabricator.com/D19946
2019-01-04 13:49:47 -08:00
epriestley
93e6dc1c1d Upgrade object reply addresses to SHA256 and remove "phabricator.mail-key"
Summary:
Ref T12509.

  - Upgrade an old SHA1 to SHA256.
  - Replace an old manually configurable HMAC key with an automatically generated one.

This is generally both simpler (less configuration) and more secure (you now get a unique value automatically).

This causes a one-time compatibility break that invalidates old "Reply-To" addresses. I'll note this in the changelog.

If you leaked a bunch of addresses, you could force a change here by mucking around with `phabricator_auth.auth_hmackey`, but AFAIK no one has ever used this value to react to any sort of security issue.

(I'll note the possibility that we might want to provide/document this "manually force HMAC keys to regenerate" stuff some day in T6994.)

Test Plan: Grepped for removed config. I'll vet this pathway more heavily in upcoming changes.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T12509

Differential Revision: https://secure.phabricator.com/D19945
2019-01-04 13:47:35 -08:00