1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-01 03:02:43 +01:00
Commit graph

648 commits

Author SHA1 Message Date
Jakub Vrana
a778151f28 Fix errors found by PHPStan
Test Plan: Ran `phpstan analyze -a autoload.php phabricator/src`.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, hach-que

Differential Revision: https://secure.phabricator.com/D17371
2017-02-17 10:10:15 +00:00
epriestley
56b1ff833b Fix some outdated help text about "Reply All" in "metamta.one-mail-per-recipient"
Summary: Ref T12240. When you "Reply All" to a Phabricator mail, we make an effort not to send the response to recipients who you hit with the original message. This isn't perfect and we can't always get it right, but the old description implies it's a bigger problem than it should be in practice.

Test Plan: Read text.

Reviewers: chad, eadler

Reviewed By: chad

Maniphest Tasks: T12240

Differential Revision: https://secure.phabricator.com/D17331
2017-02-10 07:17:10 -08:00
epriestley
3d44208e4f Clarify that "account.editable" no longer extends to profile pictures
Summary: Fixes T12216. I'd like to remove this option eventually, but just narrow its scope in the config description for now.

Test Plan: Read config description.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12216

Differential Revision: https://secure.phabricator.com/D17317
2017-02-06 08:37:26 -08:00
Austin Seipp
ab923e0a75 Implement new reCAPTCHA interface
Summary:
Fixes T12195. For the past few years, Recaptcha (now part of Google) has supported
a new, "no captcha" one-click user interface. This new UI is stable, doesn't
require any typing or reading words, and can even work without JavaScript (if
the administrator enables it on the Recaptcha side).

Furthermore, the new Recaptcha has a completely trivial API that can be dealt
with in a few lines of code. Thus, the external `recaptcha` php library is now
gone.

This API is a complete replacement for the old one, and does not require any
upgrade path for users or Phabricator administrators - public and secret keys
for the "new" Recaptcha UI are the exact same as the "classic" Recaptcha. Any
old Recaptcha keys for a domain will continue to work.

Note that Google is currently testing Yet Another new Captcha API, called
"Invisible reCAPTCHA", that will not require user interaction at all. In fact,
the user will not even be aware there //is even a captcha form//, as far as I
understand. However, this new API is 1) in beta, 2) requires new Recaptcha keys
(so it cannot be a drop-in replacement), and 3) requires more drastic API
changes, as form submission buttons must instead invoke JavaScript code, rather
than a token being passed along with the form submission. This would require far
more extensive changes to the controllers. Maybe when it's several years old, it
can be considered.

Signed-off-by: Austin Seipp <aseipp@pobox.com>

Test Plan:
Created a brand-new Phabricator installation, saw the new Captcha UI
on administrator sign up. Logged out, made 5 invalid login attempts, and saw the
new Captcha UI. Reworked the conditional to invert the condition, etc to test
and make sure the API responded properly.

Reviewers: epriestley, #blessed_reviewers, chad

Reviewed By: epriestley, #blessed_reviewers

Subscribers: avivey, Korvin

Maniphest Tasks: T12195

Differential Revision: https://secure.phabricator.com/D17304
2017-02-03 20:06:29 +00:00
epriestley
a2cd3d9a89 Change PHP 7 setup warning to complain about 7.0 only, not 7.1+
Summary: Ref T9640. On 7.0 we had signal handling issues so we can never support it, but async signals should resolve them on 7.1 or newer.

Test Plan: On PHP 7.1, got through the setup warning.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9640

Differential Revision: https://secure.phabricator.com/D17197
2017-01-12 15:59:28 -08:00
epriestley
4a34f26a44 Don't warn about "always_populate_raw_post_data" on PHP7
Summary: Ref T9640. This option was removed in PHP7, so there's no reason to warn about it.

Test Plan: No longer saw a setup warning on PHP7.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9640

Differential Revision: https://secure.phabricator.com/D17196
2017-01-12 15:59:14 -08:00
epriestley
065d865bce In the "Version Information" panel, try to include branchpoints
Summary:
Fixes T12040. In T12039, a user running local patches followed the report instructions as far as grabbing version information, but didn't update or revert their local changes or try against a clean install before reporting.

This obviously isn't ideal for us, but it's understandable (grabbing version information is much easier than upgrading/reverting), and we can do better about making this information useful: when compiling version information, try to figure out the branchpoint from a known upstream `master` branch by listing remotes, then running `git merge-base` against them.

Additionally, explicitly document that we want upstream hashes. We have to have a fallback case in this document anyway (for when you can't get to Config) so hopefully this makes it more likely that we get useful information in initial reports.

Test Plan: {F2229574}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12040

Differential Revision: https://secure.phabricator.com/D17103
2016-12-23 11:42:20 -08:00
epriestley
39b618039f Remove a very old piece of config documentation
Summary: Ref T571. This was accidentally left behind in D12266.

Test Plan: Used {key command F} to search for "bulk".

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T571

Differential Revision: https://secure.phabricator.com/D17034
2016-12-12 23:22:21 +00:00
epriestley
f3d9a0b930 Fix two cache issues (global settings; initial setup)
Summary:
  - Fixes T11995. This got moved but I missed renaming this callsite.
  - Fixes T11993. If you have valid credentials, but haven't run `storage upgrade` yet, we can hit this exception during setup. Just ignore it instead.

Test Plan:
  - Saved global settings, no more fatal.
  - Changed `storage-namespace` to junk, loaded web UI with valid database credentials.

{F2106358}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11993, T11995

Differential Revision: https://secure.phabricator.com/D17024
2016-12-11 08:28:10 -08:00
epriestley
9017bb9925 Add a setup check for installation on a burstable instance type
Summary: Fixes T11544. Attempt to detect if we're on a tiny, burstable-CPU AWS instance and complain.

Test Plan:
  - Completely faked this locally.
  - Hit the URI on an EC2 instance to check that it's correct (got back "m3.large", since that was the instance class).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11544

Differential Revision: https://secure.phabricator.com/D17014
2016-12-09 08:32:16 -08:00
epriestley
ffdc082852 Add a wide range of HTTP-request-based setup checks
Summary:
Ref T11553. With some regularity, users make various configuration mistakes which we can detect by making a request to ourselves.

I use a magical header to make this request because we want to test everything else (parameters, path).

  - Fixes T4854, probably. Tries to detect mod_pagespeed by looking for a header. This is a documentation-based "fix", I didn't actually install mod_pagespeed or formally test this.
  - Fixes T6866. We now test for parameters (e.g., user somehow lost "QSA").
  - Ref T6709. We now test that stuff is decoded exactly once (e.g., user somehow lost "B").
  - Fixes T4921. We now test that Authorization survives the request.
  - Fixes T2226. Adds a setup check to determine whether gzip is enabled on the web server, and attempts to enable it at the PHP level.
  - Fixes `<space space newline newline space><?php` in `preamble.php`.

Test Plan: Tested all of these setup warnings, although mostly by faking them.

Reviewers: joshuaspence, chad

Reviewed By: chad

Subscribers: Korvin

Maniphest Tasks: T4854, T4921, T6709, T6866, T11553, T2226

Differential Revision: https://secure.phabricator.com/D12622
2016-12-08 15:46:23 -08:00
epriestley
5a060b34df Add IPv6 reserved addresses to the default outbound blacklist
Summary:
Ref T11939. Depends on D16984. Now that CIDRLists can contain IPv6 addresses, blacklist all of the reserved IPv6 space.

This reserved blacklist is used to prevent users from accessing internal services via "Import Calendar" or "Add Macro".

They can't actually reach IPv6 addresses via these mechanisms yet because we need to do more work to support outbound IPv6 requests, but make sure reserved IPv6 space is blacklisted already when that support eventaully arrives.

Also, clean up some error messages (e.g., for trying to hit a bad URI in "Add Macro").

Test Plan:
  - Loaded pages with default blacklist.
  - Tried to make requests into IPv6 space.
  - Currently, this is impossible because of `parse_url()` and `gethostynamel()` calls.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11939

Differential Revision: https://secure.phabricator.com/D16986
2016-12-05 11:20:13 -08:00
epriestley
dc73785c4f Add a "--force" argument to "bin/config done"
Summary:
Ref T11922. When we deploy on Saturday I need to rebuild all the cluster indexes, but some instances won't have anything indexed so they won't actually trigger the activity.

Add a `--force` flag that just clears an activity even if the activity is not required.

Test Plan: Ran `bin/config done reindex --force` several times.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11922

Differential Revision: https://secure.phabricator.com/D16970
2016-12-01 13:53:33 -08:00
epriestley
29a3cd5121 Add "Manual Activities", to tell administrators to rebuild the search index
Summary:
Ref T11922. After updating to HEAD of `master`, you need to manually rebuild the index. We don't do this during `bin/storage upgrade` because it can take a very long time (`secure.phabricator.com` took roughly an hour) and can happen while Phabricator is running.

However, if we don't warn users about this they'll just get a broken index unless they go read the changelog (or file an issue, then we tell them to go read the changelog).

This adds a very simple table for notes to administrators so we can write a "you need to go rebuild the index" note, then adds one.

Administrators clear the note by completing the activity and running `bin/config done reindex`. This isn't automatic because there are various strategies you can use to approach the issue, which I'll discuss in greater detail in the linked documentation.

Also, fix an issue where `bin/storage upgrade --apply <patch>` could try to re-mark an already-applied patch as applied.

Test Plan:
  - Ran storage ugrades.
  - Got instructions to rebuild search index.
  - Cleared instructions with `bin/config done reindex`.

Reviewers: chad

Reviewed By: chad

Subscribers: avivey

Maniphest Tasks: T11922

Differential Revision: https://secure.phabricator.com/D16965
2016-11-30 11:23:54 -08:00
epriestley
48a34eced2 Prepare for InnoDB FULLTEXT support
Summary:
Ref T11741. This makes everything work if we switch to InnoDB, but never actually switches yet.

Since the default minimum word length (3) and stopword list (36 common English words) in InnoDB are generally pretty reasonable, I just didn't add any setup advice for them. I figure we're better off with simpler setup until we identify some real problem that the builtin stopwords create.

Test Plan: Swapped the `false` to `true`, ran `storage adjust`, got InnoDB fulltext indexes, searched for stuff, got default "AND" behavior.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11741

Differential Revision: https://secure.phabricator.com/D16942
2016-11-25 15:18:26 -08:00
epriestley
9d0752063e Allow bin/storage adjust to adjust table engines
Summary:
Ref T11741. On recent-enough versions of MySQL, we would prefer to use InnoDB for fulltext indexes instead of MyISAM.

Allow `bin/storage adjust` to read actual and expected table engines, and apply adjustments as necessary.

We have one existing bad table that uses the wrong engine, `metamta_applicationemail`. This change corrects that table.

Test Plan:
  - Ran `bin/storage upgrade`.
  - Saw the adjustment phase apply this change properly:

```
>>>[463] <query> ALTER TABLE `local_metamta`.`metamta_applicationemail` COLLATE = 'utf8mb4_bin', ENGINE = 'InnoDB'
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11741

Differential Revision: https://secure.phabricator.com/D16941
2016-11-25 15:13:40 -08:00
epriestley
a956047989 Use PhutilQueryCompiler in Phabricator fulltext search
Summary:
Ref T11741. Fixes T10642. Parse and compile user queries with a consistent ruleset, then submit queries to the backend using whatever ruleset MySQL is configured with.

This means that `ft_boolean_syntax` no longer needs to be configured (we'll just do the right thing in all cases).

This should improve behavior with RDS immediately (T10642), and allow us to improve behavior with InnoDB in the future (T11741).

Test Plan:
  - Ran various queries in the UI, saw the expected results.
  - Ran bad queries, got useful errors.
  - Searched threads in Conpherence.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10642, T11741

Differential Revision: https://secure.phabricator.com/D16939
2016-11-25 14:46:10 -08:00
epriestley
f199243104 Clean up another insufficiently-general exception
Summary:
Ref T11044. This is still catching the older exceptions, which are now more general.

If you loaded the web UI without MySQL running, this meant you got a less-helpful error.

Test Plan: Stopped MySQL, loaded web UI, got a more-helpful error.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11044

Differential Revision: https://secure.phabricator.com/D16930
2016-11-23 10:41:19 -08:00
epriestley
e6bfa1bd23 Remove "mysql.configuration-provider" configuration option
Summary:
Ref T11044. This was old Facebook cruft for reading configuration from SMC (and maybe doing some other questionable things). See D183.

(See also D175 for discussion of this from 2011.)

In modern Phabricator, you can subclass `SiteConfig` to provide dynamic configuration, and we do so in the Phacility cluster. This lets you change any config, and change in response to requests (e.g., for instancing) and is generally more powerful than this mechanism was.

This configuration provider theoretically let you roll your own replication or partitioning, but in practice I believe no one ever did, and no one ever could have anyway without more support in the upstream (for migrations, read-after-write, etc).

Test Plan:
  - Grepped for removed option.
  - Browsed around with clustering off.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11044

Differential Revision: https://secure.phabricator.com/D16911
2016-11-22 09:24:46 -08:00
epriestley
4da74166fe When storage is partitioned, refuse to serve requests unless web and databases agree on partitioning
Summary:
Ref T11044. One popular tool in a modern operations environment is Puppet. The primary purpose of this tool is to randomly revert hosts to older or different configurations.

Introducing an element of chaotic unpredictability into operations trains staff to be on high alert at all times, rather than lulled into complacency by predictability or consistency.

When Puppet reverts a Phabricator host's configuration to an older version, we might start writing data to a lot of crazy places where it shouldn't go. This will create a big sticky mess that is virtually impossible to undo, mostly because we'll get two files with ID 123 or two tasks with ID 456 or whatever else and good luck with that.

Instead, after changing the partition layout, require `bin/storage partition` to be run. This writes a copy of the config everywhere.

Then, when we start serving web requests, make sure every database has the exact same config. This will foil Puppet by refusing to run requests on hosts it has reverted.

Test Plan:
  - Changed partition configuration.
  - Ran Phabricator.
  - FOILED!
  - Ran `bin/storage partition` to sync config.
  - Things worked again.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11044

Differential Revision: https://secure.phabricator.com/D16910
2016-11-22 04:15:46 -08:00
epriestley
bac27fb403 Remove "mysql.implementation" configuration
Summary:
Ref T11044. Fixes T10931. This option has essentially never been useful for anything, and we've picked the best implementation for a long time (MySQLi if available, MySQL if not).

I am not aware of any reason to ever set this manually. If someone comes up with some bizarre but legitimate use case that I haven't thought of, we can modularize it.

Test Plan: Browsed around. Grepped for `mysql.implementation`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10931, T11044

Differential Revision: https://secure.phabricator.com/D16909
2016-11-22 04:15:34 -08:00
epriestley
bcfd515b32 Run all minor setup checks on all configured database hosts
Summary:
Fixes T10759. Fixes T11817. This runs all the general sanity/configuration checks on all the active servers.

None of these warnings are very important, and this doesn't change any logical stuff.

Depends on D16904.

Test Plan: Painstakingly triggered each warning, verified that they rendered correctly and that messages told me which host was affected.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10759, T11817

Differential Revision: https://secure.phabricator.com/D16905
2016-11-21 15:55:54 -08:00
epriestley
326d5bf800 Detect replicating masters and fatal (also, warn on nonreplicating replicas)
Summary:
Ref T10759. Check master/replica status during startup.

After D16903, this also means that we check this status after a database comes back online after being unreachable.

If a master is replicating, fatal (since this can do a million kinds of bad things).

If a replica is not replicating, warn (this just means the replica is behind so some data is at risk).

Also: if your masters were actually configured properly (mine weren't until this change detected it), we would throw away patches as we applied them, so they would only apply to the //first// master. Instead, properly apply all migration patches to all masters.

Test Plan:
  - Started Phabricator with a replicating master, got a fatal.
  - Stopped replication on a replica, got a warning.
  - With two non-replicating masters, upgraded storage.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10759

Differential Revision: https://secure.phabricator.com/D16904
2016-11-21 15:55:22 -08:00
epriestley
bc4187d709 When we "discover" new fatal setup issues, stop serving traffic
Summary:
Ref T10759. We may "discover" the presence of a fatal setup error later, after starting Phabricator.

This can happen in a few ways, but most are unlikely. The one I'm immediately concerned about is:

  - Phabricator starts up during a disaster with some databases unreachable.
  - We start with warnings (unreachable databases are generally not fatal, since it's OK for some subset of hosts to be down in replicated/partitioned setups).
  - The unreachable databases later recover and become accessible again.
  - When we run checks against them, we discover that they are misconfigured.

Currently, "fatal" setup issues are not truly fatal if we're "in flight" -- we've survived setup checks at least once in the past. This is bad in the scenario above.

Especially with partitioning, it could lead to mangled data in a disaster scenario where operations staff makes a small configuration mistake while trying to get things running again.

Instead, if we "discover" a fatal error while already "in flight", reset the whole setup process as though the webserver had just restarted. Don't serve requests again until we can make it through setup without hitting fatals.

Test Plan:
  - Started Phabricator with multiple masters, one of which was down and broken.
  - Got a warning about the bad master.
  - Revived the master.
  - Before: Phabricator detects the fatal, but keeps serving requests.
  - After: Phabricator detects the fatal, resets the webserver, and stops serving requests until the fatal is resolved.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10759

Differential Revision: https://secure.phabricator.com/D16903
2016-11-21 15:54:40 -08:00
epriestley
78040e0ff5 Run "DatabaseSetup" checks against all configured hosts
Summary:
Ref T10759. Currently, these checks run only against configured masters. Instead, check every host.

These checks also sort of cheat through restart during a recovery, when some hosts will be unreachable: they test for "disaster" by seeing if no masters are reachable, and just skip all the checks in that case.

This is bad for at least two reasons:

  - After recent changes, it is possible that //some// masters are dead but it's still OK to start. For example, "slowvote" may have no master, but everything else is reachable. We can safely run without slowvote.
  - It's possible to start during a disaster and miss important setup checks completely, since we skip them, get a clean bill of health, and never re-test them.

Instead:

  - Test each host individually.
  - Fundamental problems (lack of InnoDB, bad schema) are fatal on any host.
  - If we can't connect, raise it as a //warning// to make sure we check it later. If you start during a disaster, we still want to make sure that schemata are up to date if you later recover a host.

In particular, I'm going to add these checks soon:

  - Fatal if a "master" is replicating.
  - Fatal if a "replica" is not replicating.
  - Fatal if a database partition config differs from web partition config.
  - When we let a database off with a warning because it's down, and later upgrade it to a fatal because we discover it is broken after it comes up again, fatal everything. Currently, we keep running if we "discover" the presence of new fatals after surviving setup checks for the first time.

Test Plan:
  - Configured with multiple masters, intentionally broke one (simulating a disaster where one master is lost), saw Phabricator still startup.
  - Tested individual setup checks by intentionally breaking them.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10759

Differential Revision: https://secure.phabricator.com/D16902
2016-11-21 15:49:07 -08:00
epriestley
55e21565b5 Support application partitioning across multiple masters
Summary:
Ref T11044. I'm going to hold this until after the release cut, but I think it's good to go.

This allows installs to configure multiple masters in `cluster.databases` and partition applications across them (for example, put Maniphest on a dedicated database).

When we make a Maniphest connection we go look up which master we should be hitting first, then connect to it.

This has at least approximately been planned for many years, so the actual change is largely just making sure that your config makes sense.

Test Plan:
  - Configured `db001.epriestley.com` and `db002.epriestley.com` as master/master.
  - Partitioned applications between them.
  - Interacted with various applications, saw writes go to the correct host.
  - Viewed "Database Servers" and saw partitioning information.
  - Ran schema upgrades.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11044

Differential Revision: https://secure.phabricator.com/D16876
2016-11-19 14:14:39 -08:00
epriestley
558d194302 Update bin/storage workflows to accommodate multiple masters
Summary: Depends on D16847. Ref T11044. This updates the remaining storage-related workflows from the CLI to accommodate multiple masters.

Test Plan:
  - Configured multiple masters.
  - Ran all `bin/storage` workflows.
  - Ran `arc unit --everything`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11044

Differential Revision: https://secure.phabricator.com/D16848
2016-11-12 16:37:47 -08:00
epriestley
bc15eee3f2 Update SchemaQuery and the web UI to accommodate multiple master databases
Summary:
Depends on D16115. Ref T11044. In the brave new world of multiple masters, we need to check the schemata on each master when looking for missing storage patches, keys, schema changes, etc.

This realigns all the "check out what's up with that schema" calls to work for multiple hosts, and updates the web UI to include a "Server" column and allow you to browse per-server.

This doesn't update `bin/storage`, so it breaks things on its own (and unit tests probably won't pass). I'll update that in the next change.

Test Plan: Configured local environment in cluster mode with multiple masters, saw both hosts' status reported in web UI.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11044

Differential Revision: https://secure.phabricator.com/D16847
2016-11-12 16:36:52 -08:00
epriestley
ecc598f18d Support multiple database masters and convert easy callers
Summary:
Ref T11044. This moves toward partitioned application databases:

  - You can define multiple masters.
  - Convert all the easily-convertible code to become multi-master aware.

This doesn't convert most of `bin/storage` or "Config > Database (Stuff)" yet, as both are quite involved. They still work for now, but only operate on the first master instead of all masters.

Test Plan: Configured multiple masters, browsed around, ran `bin/storage` commands, ran `bin/storage --host ...`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11044

Differential Revision: https://secure.phabricator.com/D16115
2016-11-12 16:30:20 -08:00
epriestley
706c21375e Remove empty implementations of describeAutomaticCapabilities()
Summary:
This has been replaced by `PolicyCodex` after D16830. Also:

  - Rebuild Celerity map to fix grumpy unit test.
  - Fix one issue on the policy exception workflow to accommodate the new code.

Test Plan:
  - `arc unit --everything`
  - Viewed policy explanations.
  - Viewed policy errors.

Reviewers: chad

Reviewed By: chad

Subscribers: hach-que, PHID-OPKG-gm6ozazyms6q6i22gyam

Differential Revision: https://secure.phabricator.com/D16831
2016-11-09 15:24:22 -08:00
epriestley
ce14338081 When setup issues raise opcache configuration errors, point at the opcache configuration page
Summary:
Fixes T11746. The opcache docs are on a different page, so point there if we're raising opcache issues.

(It's possible for a setup issue to say "configure X, or configure Y", where X is opcache and Y is non-opcache, so we may want to render both links.)

Test Plan: {F1867109}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11746

Differential Revision: https://secure.phabricator.com/D16685
2016-10-07 08:23:13 -07:00
Chad Little
87ebb80059 Revert "Clean up more Quicksand"
Summary: This reverts commit 5eb4bc6ca9.

Test Plan: Reload homepage, no scrollbars

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D16645
2016-10-01 12:58:30 -07:00
Chad Little
5eb4bc6ca9 Clean up more Quicksand
Summary: Creates a background that renders inside the Quicksand frame, through sorcery.

Test Plan: Turn on Quicksand, visit lots of pages. See correct background colors. This probably blows something up I'm not testing.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D16642
2016-10-01 11:22:42 -07:00
epriestley
38b10f05a2 For now, disable persistent connections and the "max_connections" setup warning
Summary:
Ref T11672. At low loads, this causes us to use more connections, which is pushing some installs over the default limits.

Rather than trying to walk users through changing `max_connections`, `open_files_limit`, `fs.file-max`, `ulimit`, etc., just put things back for now. After T11044 we should have headroom to use persistent connections within the default limits on all reasonable systems..

Test Plan: Loaded Phabricator, poked around.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11672

Differential Revision: https://secure.phabricator.com/D16591
2016-09-23 12:42:26 -07:00
epriestley
396be07c15 Add a setup issue about small "max_connections" settings
Summary:
Fixes T11683. Likely as a result of the persitent connections change, more users are seeing MySQL connection limit errors.

The persistent connections change means we use //fewer// connections at the high end, but I'm guessing PHP is keeping some more connections around in the pool, so while high-traffic hosts use fewer connections, low-traffic hosts now use more.

Raise an explicit setup warning about this. Users should be adjusting it anyway, there's no value to leaving it at extremely low default and connections are baiscally free until you run out of outbound ports.

Test Plan: {F1844630}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11683

Differential Revision: https://secure.phabricator.com/D16586
2016-09-22 13:09:16 -07:00
Daniel Stone
03d323e9fd Fix config-migration text for dashboard options
Summary:
The commit which added checks for the old homepage options (now in
Dashboard) in rP9d9a47e9cf, added them to the auth section, where they
would present:
  This option has been migrated to the "Auth" application. Your old
  configuration is still in effect, but now stored in "Auth" instead of
  configuration. Going forward, you can manage authentication from the
  web UI.

Remove them from the moved-to-Auth list, and coalesce the multiple
definitions of the help text into one.

Test Plan:
 - set maniphest.priorities.unbreak-now to something
 - observe the setup issue reported
 - hope it tells you the right thing

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley, chad

Differential Revision: https://secure.phabricator.com/D16576
2016-09-20 10:47:30 +00:00
epriestley
105be01d5f Just fatal for any setting of open_basedir
Summary:
Fixes T11627.

Beyond being complex, I have no real reason to believe these checks even work (and they don't test repositories, file storage, logfiles, etc).

Test Plan:
Faked the error:

{F1813433}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11627

Differential Revision: https://secure.phabricator.com/D16544
2016-09-12 14:49:48 -07:00
epriestley
1ee426e4ac Add a specialized cache for storing "has setup ever worked?"
Summary:
Ref T11613. In D16503/T11598 I refined the setup flow to improve messaging for early-stage setup issues, but failed to fully untangle things.

We sometimes still try to access a cache which uses configuration before we build configuration, which causes an error.

Instead, store "are we in flight / has setup ever worked?" in a separate cache which doesn't use the cache namespace. This stops us from trying to read config before building config.

Test Plan:
Hit bad extension error with a fake extension, got a proper setup help page:

{F1812803}

Solved the error, reloaded, broke things again, got a "friendly" page:

{F1812805}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11613

Differential Revision: https://secure.phabricator.com/D16542
2016-09-12 08:04:32 -07:00
epriestley
f82adbf0c8 Add a friendlier "in flight" error page
Summary:
Ref T11589. When we hit a fatal setup issue (essentially always a connection failure) //after// we've already survived them on at least one request, we can be pretty sure a server went down and that the problem is not a setup/configuration issue.

In this case, show a friendlier error page instead of the fairly detailed technical one.

Test Plan:
  - Broke MySQL config.
  - Restarted Apache.
  - Got the "admin/setup" error page:

{F1803268}

  - Fixed the MySQL config.
  - Loaded any page, to put us "in flight".
  - Broke MySQL config.
  - Loaded any page.
  - Got the friendly "in flight" error page:

{F1803271}

If you want to design this better, easiest way to get to it is:

  - Set `mysql.port` to `9999` in `conf/local/local.json`.
  - Reload any page while already running (don't restart).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11589

Differential Revision: https://secure.phabricator.com/D16503
2016-09-07 06:59:48 -07:00
epriestley
00bb0c9071 Raise setup warnings immediately when failing to load configuration from the database
Summary:
Ref T11589. Previously, when we failed to load database configuration we just continued anyway, in order to get to setup checks so we could raise a better error.

There was a small chance that this could lead to pages running in a broken state, where ONLY that connection failed and everything else worked. This was accidentally fixed by narrowing the exceptions we continue on in D16489.

However, this "fix" meant that users no longer got helpful setup instructions. Instead:

  - Keep throwing these exceptions: it's bad to continue if we've failed to connect to the database.
  - However, catch them and turn them into setup errors.
  - Share all the setup code so these errors and setup check errors work the same way.

Test Plan:
  - Intentionally broke `mysql.host` and `mysql.pass`.
  - Loaded pages.
  - Got good setup errors.
  - Hit normal setup errors too.
  - Put everything back.
  - Swapped into cluster mode.
  - Intentionally broke cluster mode, saw failover to readonly.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11589

Differential Revision: https://secure.phabricator.com/D16501
2016-09-06 14:20:31 -07:00
epriestley
3099601463 Split setup check phases into "preflight" and "post-config"
Summary:
Ref T11589. This runs:

  - preflight checks (critical checks: PHP version stuff, extensions);
  - configuration;
  - normal checks.

The PHP checks are split into critical ("bad version") and noncritical ("sub-optimal config").

I tidied up the extension checks slightly, we realistically depend on `cURL` nowadays.

Test Plan:
  - Faked a preflight failure.
  - Hit preflight check.
  - Got expected error screen.
  - Loaded normal pages.
  - Hit a normal setup check.
  - Used DarkConsole "Startup" tab to verify that preflight checks take <1ms to run (we run them on every page without caching, at least for now, but they only do trivial checks like PHP versions).

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11589

Differential Revision: https://secure.phabricator.com/D16500
2016-09-06 14:20:11 -07:00
epriestley
b1932f1f56 Split setup checks into "preflight" and "normal" checks
Summary:
Ref T11589. Currently, initialization order is a bit tangled: we load configuration from the database, then later test if we can connect to the database.

Instead, I'm going to do: preflight checks ("PHP Version OK?", "Extensions installed?"), then configuration, then normal setup checks.

To prepare for this, flag core checks as "preflight" and add a setup panel to visually confirm that I didn't miss anything.

Test Plan: {F1803210}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11589

Differential Revision: https://secure.phabricator.com/D16499
2016-09-06 11:57:18 -07:00
epriestley
4dc37bcee0 Ignore repository versions on inactive devices in "Repository Servers" panel in Config
Summary:
Fixes T11590. Currently, we incorrectly consider cluster repository versions that are (or were) on devices which are no longer part of the active cluster service when building this status screen.

Instead, ignore them. This is just a display bug; the actual `ClusterEngine` already had similar logic.

Test Plan:
  - Added a bad leader record to `repository_workingcopyversion`.
  - Before patch, got a bad "Partial (1w)" sync:

{F1802292}

  - After patch, got a good "Sycnchronized":

{F1802293}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11590

Differential Revision: https://secure.phabricator.com/D16492
2016-09-05 11:10:16 -07:00
Chad Little
36fa4e5380 Use new Guide layout in Config->Welcome
Summary: Ref T11132, swaps in new UI for welcome page using guide modules

Test Plan: Test instance and non instance guides. Test each setting. Unclear on how to test people / Phacility. Just change the URL link?

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T11132

Differential Revision: https://secure.phabricator.com/D16482
2016-09-03 07:32:22 -07:00
Chad Little
4dacf4d3ad Add a basic first feed story on /home/
Summary: Ref T11132. Adds a text panel to feed if no stories are present and the user is an admin. Seems ok-ish for 15 minutes. Happy to take content suggestions.

Test Plan: Make a new install, see panel. Log in as new user, don't see panel.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T11132

Differential Revision: https://secure.phabricator.com/D16479
2016-08-31 13:57:57 -07:00
Chad Little
eac8171a63 Add some icons/color to Config Settings UI
Summary: This adds status icons, locked, hidden, editable, customized, to the list of options in config. Makes it easier to read and assertain state.

Test Plan:
View a hidden, customized, editable, and locked.

{F1796320}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D16475
2016-08-30 12:58:15 -07:00
epriestley
5504f37eb2 Add a summary view of all repository errors to the repository cluster screen
Summary: Ref T11559. This makes managing large numbers of repositories slightly easier.

Test Plan: {F1796119}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11559

Differential Revision: https://secure.phabricator.com/D16472
2016-08-30 09:21:12 -07:00
Chad Little
60d1762a85 Redesign Config Application
Summary: Ref T11132, significantly cleans up the Config app, new layout, icons, spacing, etc. Some minor todos around re-designing "issues", mobile support, and maybe another pass at actual Group pages.

Test Plan: Visit and test every page in the config app, set new items, resolve setup issues, etc.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam, Korvin

Maniphest Tasks: T11132

Differential Revision: https://secure.phabricator.com/D16468
2016-08-29 15:49:49 -07:00
Chad Little
00796e592b Move Setup Issues into it's own notification style menu
Summary: Ref T11132. This gets rid of the red bar for admins and instead shows a new menu item next to notifications/chat if there are unresolved configuration issues. Menu goes away if there are no issues. May move this later into the bell icon, but think think might be the right place to start especially for NUX and updates. Maybe limit the number of items?

Test Plan:
Tested with some, lots, and no config issues.

{F1790156}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T11132

Differential Revision: https://secure.phabricator.com/D16461
2016-08-29 10:43:30 -07:00
epriestley
72a03dc03e Add a setup warning for "always_populate_raw_post_data"
Summary:
Fixes T9235. When the stars align, PHP 5.6 or newer emits a deprecation warning on startup about "always_populate_raw_post_data" which occurs too early for us to intercept and can break responses by adding garbage to the output.

These settings appear to be sufficient:

```
always_populate_raw_post_data = 1
display_errors = 1
display_startup_errors = 1
error_reporting = -1
```

Then make a request with an unusual content type:

```
$ curl -X POST -H "Content-Type: application/json" -d "{foo: bar}" http://phabricator.example.com/
```

This triggers the warning:

```
<br />
<b>Deprecated</b>:  Automatically populating $HTTP_RAW_POST_DATA is deprecated and will be removed in a future version. To avoid this warning set 'always_populate_raw_post_data' to '-1' in php.ini and use the php://input stream instead. in <b>Unknown</b> on line <b>0</b><br />
<br />
...
```

To avoid this, just instruct administrators to set this value to "-1", which completely disables the feature and silences the warning.

Test Plan:
  - Reproduced this issue by following the instructions above.
  - Triggered the setup issue locally and read all the captivating prose:

{F1786911}

  - Made the configuration change it directed me to, saw the setup issue resolve.

Reviewers: jcox

Reviewed By: jcox

Maniphest Tasks: T9235

Differential Revision: https://secure.phabricator.com/D16454
2016-08-26 07:38:08 -07:00
Chad Little
9d9a47e9cf Add setup checks for unused homepage options
Summary: Ref T11533, Fixes T5315. Remove and add extra setup checks for removed homepage options.

Test Plan: Review text.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T5315, T11533

Differential Revision: https://secure.phabricator.com/D16453
2016-08-25 12:08:02 -07:00
Josh Cox
a88dc2afc2 Added a setup check for empty REMOTE_ADDR
Summary: Fixes T8850. Previously, if a user's preamble script mangled `$_SERVER['REMOTE_ADDR']` or somehow set it to `null`, the user would get errors when performing certain actions. Now those errors shouldn't occur, and instead the user will be warned that there is a setup issue related to their preamble script.

Test Plan: Create a preamble script that contains `$_SERVER['REMOTE_ADDR'] = null;` then navigate to /config/issue/. There should be a warning there about `REMOTE_ADDR` not being available.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, yelirekim, epriestley

Maniphest Tasks: T8850

Differential Revision: https://secure.phabricator.com/D16450
2016-08-25 13:04:12 -04:00
Chad Little
15ed2b936c Update Config Application UI
Summary: Switches over to new property UI boxes, splits core and apps into separate pages. Move Versions into "All Settings". I think there is some docs I likely need to update here as well.

Test Plan: Click on each item in the sidebar, see new headers.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D16429
2016-08-22 10:40:24 -07:00
Chad Little
e7aa874f5e Fix getIcon calls in PHUIObjectListItem
Summary: Fixes T11501. Let's you pass in a full PHUIIconView or just the icon name to give ObjectListItem a large icon.

Test Plan: Alamanac, Applications, Drydock, Settings, Search Typeahead, Config page...

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, PHID-OPKG-gm6ozazyms6q6i22gyam

Maniphest Tasks: T11501

Differential Revision: https://secure.phabricator.com/D16421
2016-08-19 09:35:09 -07:00
Chad Little
f50e550c9e Correct various spelling errors
Summary: Fixes T11477.

Test Plan: Grep for Mulitple

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T11477

Differential Revision: https://secure.phabricator.com/D16399
2016-08-15 10:47:51 -07:00
epriestley
67861ec197 List both hosted and observed repositories in "Cluster Repository Status" configuration console
Summary:
When I wrote this the first time, only hosted repositories could be clustered.

This check wasn't removed when I allowed observed repositories to be clustered in D15986.

Test Plan:
Reloaded {nav Config > Repository Servers} page, saw more stuff locally.

Reviewed the cardinal digits between 1 and 17, inclusive.

Reviewers: chad, avivey

Reviewed By: avivey

Differential Revision: https://secure.phabricator.com/D16392
2016-08-11 16:42:30 -07:00
epriestley
e8083ad63a Increase the storage size for commit summaries
Summary:
Fixes T11453. Currently, commit message summaries are limited to 80 bytes. This may only be 20-40 characters for CJK languages or langauges with Cyrillic script.

Increase storage size to 255, then truncate to the shorter of 255 bytes or 80 glyphs. This preserves the same behavior for latin languages, but is less tight for Russian, etc.

Some minor additional changes:

  - Provide a way to ask "how much data fits in this column?" so we don't have to duplicate column lengths across summary checks or UI errors like "title too long".
  - Remove the `text80` datatype, since no other columns use it and we have no use cases (or likely use cases) for it.

Test Plan:
  - Made a commit with a Cyrillic title, saw reasonable summarization in UI:

{F1757522}

  - Added and ran unit tests.
  - Grepped for removed `SUMMARY_MAX_LENGTH` constant.
  - Grepped for removed `text80` data type.

Reviewers: avivey, chad

Reviewed By: avivey

Subscribers: avivey

Maniphest Tasks: T11453

Differential Revision: https://secure.phabricator.com/D16385
2016-08-10 11:12:45 -07:00
Chad Little
679fbada44 Remove PHUIDocumentView
Summary: Converts final call site to PHUIDocumentViewPro.

Test Plan: grep for PHUIDocumentView, view new Welcome Page

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D16379
2016-08-08 09:39:48 -07:00
epriestley
3a002b6b83 Make new logo and wordmark more reasonably configurable by human users
Summary: Fixes T11437. Provides a normal form for configuring this, instead of weird "look up the PHID and adjust things in the database" stuff.

Test Plan:
{F1753651}

{F1753652}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11437

Differential Revision: https://secure.phabricator.com/D16377
2016-08-07 12:00:21 -07:00
Chad Little
52c0ec2700 Update Phabricator logo
Summary:
This updates the eye logo and removes the formal wordmark "Phabricator" as an image. Instead we'll use the new updated eye logo and plain text for "Phabricator", both of which are more friendly and less industrial.

Installs that already use the `header-logo` customization setting will need to rebuild their logo to 80px x 80px. They will then also get to use plain text to whitebox their install as they see fit.

Test Plan:
Tested new logo at desktop, tablet, and mobile sizes. Set a random instance name, saw new wordmark. Created a really long wordmark of MMMMMMMMMMMM, saw text cut off so UI doesn't break. May need some additional tweaking, but I think we covered the most edge cases here.

{F1751791, size=full}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: edibiase, bjshively, yelirekim, Korvin

Maniphest Tasks: T4214, T11096

Differential Revision: https://secure.phabricator.com/D16373
2016-08-07 11:35:21 -07:00
epriestley
b53831101f Raise explicit setup issues about PHP 7
Summary: Ref T9640. Fixes T9888. Decline to support PHP 7 until the async signal handling issue in T11270 is resolved.

Test Plan: Faked local version, got helpful error message.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9640, T9888

Differential Revision: https://secure.phabricator.com/D16231
2016-07-05 05:33:26 -07:00
epriestley
67084a6953 Support AES256 at-rest encryption in Files
Summary:
Ref T11140. This makes encryption actually work:

  - Provide a new configuation option, `keyring`, for specifying encryption keys.
  - One key may be marked as `default`. This activates AES256 encryption for Files.
  - Add `bin/files generate-key`. This is helps when generating valid encryption keys.
  - Add `bin/files encode`. This changes the storage encoding of a file, and helps test encodings and migrate existing data.
  - Add `bin/files cycle`. This re-encodes the block key with a new master key, if your master key leaks or you're just paraonid.
  - Document all these options and behaviors.

Test Plan:
  - Configured a bad `keyring`, hit a bunch of different errors.
  - Used `bin/files generate-key` to try to generate bad keys, got appropriate errors ("raw doesn't support keys", etc).
  - Used `bin/files generate-key` to generate an AES256 key.
  - Put the new AES256 key into the `keyring`, without `default`.
  - Uploaded a new file, verified it still uploaded as raw data (no `default` key yet).
  - Used `bin/files encode` to change a file to ROT13 and back to raw. Verified old data got deleted and new data got stored properly.
  - Used `bin/files encode --key ...` to explicitly convert a file to AES256 with my non-default key.
  - Forced a re-encode of an AES256 file, verified the old data was deleted and a new key and IV were generated.
  - Used `bin/files cycle` to try to cycle raw/rot13 files, got errors.
  - Used `bin/files cycle` to cycle AES256 files. Verified metadata changed but file data did not. Verified file data was still decryptable with metadata.
  - Ran `bin/files cycle --all`.
  - Ran `encode` and `cycle` on chunked files, saw commands fail properly. These commands operate on the underlying data blocks, not the chunk metadata.
  - Set key to `default`, uploaded a file, saw it stored as AES256.
  - Read documentation.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11140

Differential Revision: https://secure.phabricator.com/D16127
2016-06-16 08:08:56 -07:00
epriestley
cb7560d301 Remove "re prefix" and "vary subjects" config
Summary: Ref T11098. There is no reason to maintain these as separate values now that they can be configured in global settings.

Test Plan:
  - Hit and read setup issue.
  - Fiddled with settings.
  - I'll vet this more throughly in the next diff since I need to fix an issue with global defaults in mail and can explicitly test this at the same time.

Reviewers: chad

Reviewed By: chad

Subscribers: eadler

Maniphest Tasks: T11098

Differential Revision: https://secure.phabricator.com/D16117
2016-06-14 12:35:12 -07:00
epriestley
c694bd651b Make base64 the default SMTP encoding
Summary: Ref T11120. If this works, I'll just remove this option completely.

Test Plan: ¯\_(ツ)_/¯

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11120

Differential Revision: https://secure.phabricator.com/D16095
2016-06-09 13:22:17 -07:00
epriestley
edfc6a6934 Convert some loadPreferences() to getUserSetting()
Summary:
Ref T4103. This doesn't get everything, but takes care of most of the easy stuff.

The tricky-ish bit here is that I need to move timezones, pronouns and translations to proper settings. I expect to pursue that next.

Test Plan:
- Grepped for `loadPreferences` to identify callsites.
- Changed start-of-week setting, loaded Calendar, saw correct start.
- Visited welcome page, read "Adjust Settings" point.
- Loaded Conpherence -- I changed behavior here slightly (switching threads drops the title glyph) but it wasn't consistent to start with and this seems like a good thing to push to the next version of Conpherence.
- Enabled Filetree, toggled in Differential.
- Disabled Filetree, no longer visible in Differential.
- Changed "Unified Diffs" preference to "Small Screens" vs "Always".
- Toggled filetree in Diffusion.
- Edited a task, saw sensible projects in policy dropdown.
- Viewed user profile, uncollapsed/collapsed side nav, reloaded page, sticky'd.
- Toggled "monospaced textareas", used a comment box, got appropriate fonts.
- Toggled durable column.
- Disabled title glyphs.
- Changed monospaced font to 18px/36px impact.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4103

Differential Revision: https://secure.phabricator.com/D16004
2016-06-02 06:29:20 -07:00
epriestley
74a36f9d7b Read "Database Status" page connection information from cluster config if present
Summary:
Fixes T11043. This page was still reading the old information directly instead of going through the cluster-aware stuff.

Have it ask the cluster-aware stuff for information instead.

Test Plan:
  - Nuked MySQL on localhost.
  - Configured cluster hosts.
  - Loaded "Database Status" page -- worked after patch.
  - Grepped for any remaining `mysql.configuration-provider` stragglers, came up empty.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11043

Differential Revision: https://secure.phabricator.com/D15982
2016-05-26 12:14:39 -07:00
epriestley
8a98868bfb Remove "days fresh" / "days stale" indictator in Differential revision list
Summary:
Ref T10939. I'm not //totally// opposed to the existence of this element, but I think it's the kind of thing that would never make it upstream today. I think this should just be a T418 custom sort of thing in the long run, not a mainline upstream feature.

Overall, I think this thing is nearly useless and just adds visual clutter. My dashboard is about 100% red. This also sort of teaches users that it's fine to let revisions sit for a couple of days, which isn't what I'd like the UI to teach. Finally, removing it helps the UI feel a little less cluttered after the visually busy changes in D15926.

Test Plan: Grepped for removed config. Viewed revision list.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10939

Differential Revision: https://secure.phabricator.com/D15927
2016-05-16 10:47:19 -07:00
epriestley
cd86bf0174 Remove metamta.differential.unified-comment-context and explain it in ExtraConfigSetupCheck
Summary: Ref T10694. This setting no longer has any effect: we always show a limited amount of context now.

Test Plan: `grep`

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10694

Differential Revision: https://secure.phabricator.com/D15886
2016-05-11 06:47:58 -07:00
epriestley
3fda965288 When multiple web hosts are in service, don't require setup warnings to be dismissed on each one
Summary:
Fixes T10876. Currently, we can end up with a setup warning banner sticking on each web device, since the state is stored in local cache.

Instead:

  - When we actually run the setup checks, save the current state in the database.
  - Before we show a cached banner, make sure the database still says the checks are a problem.

This could lead to some inconsistencies if setup checks legitimately pass on some hosts but not on others. For example, if you have `git` installed on one machine but not on another, we may raise a setup warning ("No Git Binary!") about it on one host only.

For now, assume users have their operational environments in some sort of reasonable shape and can install the same stuff everywhere. In the future, we could split the issues into "global" and "per-host" issues if we run into problems with this.

Test Plan:
This is somewhat tricky to test locally since you really need multiple webservers to test it properly, but I:

  - Created some setup issues, saw banner.
  - Ignored/cleared them, saw banner go away.
  - Verified database cache writes were occurring properly.

Then I sort of faked it like this:

  - Created a setup issue.
  - Manually set the database cache value to `[]` ("no issues").
  - Reloaded page.
  - No more banner.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10876

Differential Revision: https://secure.phabricator.com/D15802
2016-04-26 10:03:45 -07:00
epriestley
9656fe48bc Add a "Repository Servers" cluster administration panel
Summary: Ref T4292. This adds a new high-level overview panel.

Test Plan: {F1238854}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15772
2016-04-21 11:56:44 -07:00
epriestley
c30fe65ee9 Remove the warning about the Git 2GB pathname issue
Summary:
Ref T10832. In practice, `git --version` is not a useful test for this issue:

  - Vendors like Debian have backported the patch into custom versions like `0.0.0.1-debian-lots-of-patches.3232`.
  - Vendors like Ubuntu distribute multiple different versions which report the same string from `git --version`, some of which are patched and some of which are not.

In other cases, we can perform an empirical test for the vulnerability. Here, we can not, because we can't write a 2GB path in a reasonable amount of time.

Since vendors (other than Apple) //generally// seem to be on top of this and any warning we try to raise based on `git --version` will frequently be incorrect, don't raise this warning.

I'll note this in the changelog instead.

Test Plan: Looked at setup issues, no more warning for vulnerable git version.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10832

Differential Revision: https://secure.phabricator.com/D15756
2016-04-19 07:01:45 -07:00
epriestley
d844e51127 Warn users about remote code execution in older Git
Summary: Ref T10832. Raise a setup warning for out-of-date versions of `git`.

Test Plan: {F1224632}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10832

Differential Revision: https://secure.phabricator.com/D15745
2016-04-18 07:54:13 -07:00
epriestley
cd8491ae93 Fix "daemons running as wrong user" setup issue
Summary:
Fixes T9385. This was accidentally mangled a bit a long time ago by D12797, which was a 1,000-file change which got almost everything right.

Simplify the message and fix all the `%s` conversions and how they map to parameters.

Test Plan: {F1220400}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9385

Differential Revision: https://secure.phabricator.com/D15722
2016-04-15 10:03:15 -07:00
epriestley
2930733ac9 Complete modernization of Aphlict configuration
Summary:
Fixes T10697. This finishes bringing the rest of the config up to cluster power levels.

Phabricator is now given an arbitrarily long list of notification servers.

Each Aphlict server is given an arbitrarily long list of ports to run services on.

Users are free to make them meet in the middle by proxying whatever they want to whatever else they want.

This should also accommodate clustering fairly easily in the future.

Also rewrote the status UI and changed a million other things. 🐗

Test Plan:
{F1217864}

{F1217865}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15703
2016-04-14 04:57:00 -07:00
epriestley
c6b0925954 Move Aphlict logging and PID configuration options to config file
Summary: Ref T10697. Mostly straightforward. Also allow the server to have multiple logs and log options in the future (e.g., different verbosities or separate admin/client logs or whatever). No specific plans for this, but the default log is pretty noisy today.

Test Plan: Set up a couple of logs, started server, saw it log to them.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15702
2016-04-14 04:55:19 -07:00
epriestley
c84dee522b Move server-related Aphlict options to a configuration file
Summary: Ref T10697. This isn't everything but starts generalizing options and moving us toward a cluster-ready state of affairs.

Test Plan: Started server in various configurations, hit most (all?) of the error cases with bad configs, sent test notifications.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10697

Differential Revision: https://secure.phabricator.com/D15701
2016-04-14 04:54:42 -07:00
epriestley
66366137ff Don't apply security.require-https to intracluster requests
Summary:
Ref T10784. Currently, if you terminate SSL at a load balancer (very common) and use HTTP beyond that, you have to fiddle with this setting in your premable or a `SiteConfig`.

On the balance I think this makes stuff much harder to configure without any real security benefit, so don't apply this option to intracluster requests.

Also document a lot of stuff.

Test Plan: Poked around locally but this is hard to test outside of a production cluster, I'll vet it more thoroughly on `secure`.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10784

Differential Revision: https://secure.phabricator.com/D15696
2016-04-13 12:51:41 -07:00
epriestley
58eef68b7c Rough cut of repository cluster status panel
Summary:
Ref T4292. This adds some very basic cluster/device data to the new management view. Nothing interesting yet.

Also deal with disabled bindings a little more cleanly.

Test Plan: {F1214619}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4292

Differential Revision: https://secure.phabricator.com/D15685
2016-04-12 05:38:10 -07:00
epriestley
ac35246d0d Never sever non-cluster database; write more read-only documentation
Summary:
Ref T4571. Write more of the missing documentation sections and clarify a few things.

Since the "replicating master" check needs a special permission, imposes a performance penalty, is probably very difficult to misconfigure, and likely not a big deal anyway, just drop the idea of trying to automatically detect + prevent it. We still show if it's an issue on the status page, provided we have permission to check.

When you don't have any cluster databases configured, never stop trying to connect to the default master database. We might want to do this eventually as load reduction, but just don't muddy the waters too much for now while things stabilize.

Test Plan:
  - Tested functionality in cluster, non-cluster, and degraded-cluster modes.
  - Used status console to monitor a health check cycle.
  - Read docs.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15679
2016-04-11 08:44:11 -07:00
epriestley
ebff07d019 Automatically sever databases after prolonged unreachability
Summary:
Ref T4571. When a database goes down briefly, we fall back to replicas.

However, this fallback is slow (not good for users) and keeps sending a lot of traffic to the master (might be bad if the root cause is load-related).

Keep track of recent connections and fully degrade into "severed" mode if we see a sequence of failures over a reasonable period of time. In this mode, we send much less traffic to the master (faster for users; less load for the database).

We do send a little bit of traffic still, and if the master recovers we'll recover back into normal mode seeing several connections in a row succeed.

This is similar to what most load balancers do when pulling web servers in and out of pools.

For now, the specific numbers are:

  - We do at most one health check every 3 seconds.
  - If 5 checks in a row fail or succeed, we sever or un-sever the database (so it takes about 15 seconds to switch modes).
  - If the database is currently marked unhealthy, we reduce timeouts and retries when connecting to it.

Test Plan:
  - Configured a bad `master`.
  - Browsed around for a bit, initially saw "unrechable master" errors.
  - After about 15 seconds, saw "major interruption" errors instead.
  - Fixed the config for `master`.
  - Browsed around for a while longer.
  - After about 15 seconds, things recovered.
  - Used "Cluster Databases" console to keep an eye on health checks: it now shows how many recent health checks were good:

{F1213397}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15677
2016-04-11 08:43:52 -07:00
epriestley
6a4a9bb2d2 When cluster.databases is configured, read the master connection from it
Summary:
Ref T4571. Ref T10759. Ref T10758. This isn't complete, but gets most of the job done:

  - When `cluster.databases` is set up, most things ignore `mysql.host` now.
  - You can `bin/storage upgrade` and stuff works.
  - You can browse around in the web UI and stuff works.

There's still a lot of weird tricky stuff to navigate, and this has real no advantages over configuring a single server yet (no automatic failover, etc).

Test Plan:
  - Configured `cluster.databases` to point at my `t1.micro` hosts in EC2 (master + replica).
  - Ran `bin/storage upgrade`, got a new install setup on them properly.
  - Survived setup warnings, browsed around.
  - Switched back to local config, ran `bin/storage upgrade`, browsed around, went through setup checks.
  - Intentionally broke config (bad hosts, no masters) and things seemed to react reasonably well.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571, T10758, T10759

Differential Revision: https://secure.phabricator.com/D15668
2016-04-10 12:18:42 -07:00
epriestley
0439645d5b Add a "Database Cluster Status" console in Config
Summary: Ref T4571. The configuration option still doesn't do anything, but add a status panel for basic setup monitoring.

Test Plan:
Here's what a good version looks like:

{F1212291}

Also faked most of the errors it can detect and got helpful diagnostic messages like this:

{F1212292}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15667
2016-04-09 20:34:13 -07:00
epriestley
3f51b78539 Lay cluster.databases configuration groundwork for database clustering
Summary:
Ref T4571. This adds a new option which allows you to upgrade your one-host configuration to a multi-host configuration by configuring it.

Doing this currently does nothing. I wrote a lot of words about what it is //supposed// to do in the future, though.

Test Plan:
  - Tried to configure the option in all the possible bad ways, got errors.
  - Read documentation.

Reviewers: chad

Reviewed By: chad

Subscribers: eadler

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15663
2016-04-09 13:41:16 -07:00
epriestley
49d93dcf98 Add a cluster.read-only option
Summary:
Ref T4571. There will be a very long path beyond this, but add a basic read-only mode. You can explicitly enable this to put Phabricator in a sort of "maintenance" mode today if you're swapping databases or something.

In the long term, we'll automatically degrade into this mode if the master database is down.

Test Plan:
  - Enabled read-only mode.
  - Browsed around.
  - Didn't immediately see anything that was totally 100% broken.

Most stuff is 80-90% broken right now. For example:

  - Stuff like submitting comments doesn't work, and gives you a confusing, unhelpful error.
  - None of the UI really knows that it's read-only. EditEngine stuff should all hide itself and say "you can't add new comments while an install is in read-only mode", for example, but currently does not.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4571

Differential Revision: https://secure.phabricator.com/D15662
2016-04-09 13:40:47 -07:00
Chad Little
57e606b395 Update Settings/Config UI
Summary: Testing out a new 'nav' layout in Settings / Config. Spent a few days here and couldn't find much better overall.

Test Plan: View each page in Settings and in Config. Save some config options. Test mobile, desktop, tablet.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15659
2016-04-08 22:00:38 +00:00
Chad Little
839e7e2fc6 Update Config to new UI
Summary: Converts Config to new UI, updates to `newPage`

Test Plan: Review all pages in Config, setup issues, ignore an issue, edit a config option

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15587
2016-04-03 00:27:39 +00:00
epriestley
4a6589524b Add amazon-ses.endpoint configuration
Summary: Fixes T5116.

Test Plan: Will test in production.

Reviewers: chad

Maniphest Tasks: T5116

Differential Revision: https://secure.phabricator.com/D15515
2016-03-23 12:28:59 -07:00
epriestley
7d4b323da2 Store Almanac "service types" instead of "service classes"
Summary:
Ref T10449. Currently, we store classes (like "AlmanacClusterRepositoryServiceType") in the database.

Instead, store types (like "cluster.repository").

This is a small change, but types are a little more flexible (they let us freely reanme classes), a little cleaner (fewer magic strings in the codebase), and a little better for API usage (they're more human readable).

Make this minor usability change now, before we unprototype.

Also make services searchable by type.

Also remove old Almanac API endpoints.

Test Plan:
  - Ran migration, verified all data migrated properly.
  - Created, edited, rebound, and changed properties of services.
  - Searched for services by service type.
  - Reviewed available Conduit methods.

Reviewers: chad

Reviewed By: chad

Subscribers: yelirekim

Maniphest Tasks: T10449

Differential Revision: https://secure.phabricator.com/D15346
2016-02-26 06:21:50 -08:00
Chad Little
f35509e30e Update to use PHUIRemarkupView everywhere possible
Summary: Moves all the one off object calls to PHUIRemarkupView, adds a "Document" call as well (future plans).

Test Plan: Visited most pages I could get access to, but may want extra careful eyes on this diff.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15281
2016-02-16 14:05:53 -08:00
epriestley
9d125b459e Use large text columns to store IP addresses
Summary: Fixes T10259. There was no real reason to do this `ip2long()` stuff in the first place -- it's very slightly smaller, but won't work with ipv6 and the savings are miniscule.

Test Plan:
  - Ran migration.
  - Viewed logs in web UI.
  - Pulled and pushed.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10259

Differential Revision: https://secure.phabricator.com/D15165
2016-02-02 10:13:14 -08:00
Chad Little
fe5cd4ca2c Move FontIcon calls to Icon
Summary: Normalizes all `setFontIcon` calls to `setIcon`.

Test Plan: UIExamples, Almanac, Apps list, etc.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, hach-que, yelirekim

Differential Revision: https://secure.phabricator.com/D15129
2016-01-28 08:48:45 -08:00
epriestley
c00cd5c2a3 Make hidden and locked configuration even more explicit
Summary:
A user in IRC seemed very confused by this, and worked extremely hard to shoot themsevles in the foot by manually writing locked configuration to the database.

Try to explain why configuration is locked better.

Test Plan:
Mostly reading.

{F1078905}

{F1078906}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D15128
2016-01-28 08:18:24 -08:00
Chad Little
36158dbdc0 Convert all calls to 'IconFont' to just 'Icon'
Summary: Mostly for consistency, we're not using other forms of icons and this makes all classes that use an icon call it in the same way.

Test Plan: tested uiexamples, lots of other random pages.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15125
2016-01-27 20:59:27 -08:00
Chad Little
c9119306d7 Extend PHUITheme to include Profile Nav
Summary: Reasonable first pass, removes the "light" header, due to pain of upkeep. Reinforces UI color into the Profile Nav (and later likely dropmenu hovers). Most of this is reasonably easy to maintain now, but I may do a more accurate color pass after I get some more time together with it. For now this feels pretty good if you're developing in a different color UI.

Test Plan:
Switch between all the colors, hover over all the states.

{F1076766}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15120
2016-01-27 13:56:04 -08:00
epriestley
1e69f06d74 Remove redundant restructured text entry in default Paste highlighting options
Summary: Fixes T10217. This extra entry is just a mistake because of `rst` + `rest` both being valid suffixes. We don't need both entries.

Test Plan: Edited a paste, only saw one entry in dropdown for restructured text.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10217

Differential Revision: https://secure.phabricator.com/D15114
2016-01-25 06:47:53 -08:00
epriestley
d37153f003 Make bin/storage upgrade and bin/storage adjust emit detailed messages if the user has no access to databases
Summary:
Ref T10195. Distinguish between "database does not exist" and "database exists, you just don't have permission to access it".

We can't easily get this information out of INFORMATION_SCHEMA but can just `SHOW TABLES IN ...` every database that looks like it's missing and then look at the error code.

Test Plan:
  - Created a user `limited` with limited access.
  - Ran `bin/storage adjust`.
  - Got hopefully more helpful messages about access problems, instead of "Missing" errors.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10195

Differential Revision: https://secure.phabricator.com/D15079
2016-01-21 13:06:00 -08:00
Chad Little
550793f9a4 Remove header gradients for flat colors
Summary: Removes header gradient images for flat, CSS controlled colors. I didn't convert the "pony" colors over, going with few options for easier theme-ability.

Test Plan:
Test each color choice.

{F1063828}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D15052
2016-01-19 07:17:02 -08:00
epriestley
7e0558612b Fix excessively harsh validation of certain complex configuration
Summary:
See IRC. We're supposed to repair configuration, but if custom validators throw a generic `Exception` or use `PhutilTypeSpec` to do a check, we may explode way harder than we intend to.

Instead, soften these exceptions into validation exceptions so we repair configuration, raise a setup issue, and continue.

Test Plan: {F1059609}

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14998
2016-01-11 14:20:11 -08:00
epriestley
cb08757032 Swap S3 to first-party client
Summary:
Ref T5155. Swaps Phabricator over to the new first-party S3 client using the v4 authentication API so it works in all regions.

The API requires an explicit region, so the new `amazon-s3.region` is now required. I'll write guidance about this.

Test Plan:
  - Uploaded files to S3.
  - Migrated ~1GB of files to S3.
  - Loaded a bunch of files off S3.
  - Browsed around the S3 bucket.
  - Deleted a file, verified the data on S3 was destroyed.
  - Hit new setup warning.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T5155

Differential Revision: https://secure.phabricator.com/D14982
2016-01-10 07:55:27 -08:00
epriestley
9ab22e21b3 Allow installs to customize project icons
Summary:
Ref T10010. Ref T5819. General alignment of the stars:

  - There were some hacks in Conduit around stripping `fa-...` off icons when reading and writing that I wanted to get rid of.
  - We probably have room for a subtitle in the new heavy nav, and using the icon name is a good starting point (and maybe good enough on its own?)
  - The project list was real bad looking with redundant tag/names, now it is very slightly less bad looking with non-redundant types?
  - Some installs will want to call Milestones something else, and this gets us a big part of the way there.
  - This may slightly help to reinforce "tag" vs "policy" vs "group" stuff?

---

I'm letting installs have enough rope to shoot themselves in the foot (e.g., define 100 icons). It isn't the end of the world if they reuse icons, and is clearly their fault.

I think the cases where 100 icons will break down are:

  - Icon selector dialog may get very unwieldy.
  - Query UI will be pretty iffy/huge with 100 icons.

We could improve these fairly easily if an install comes up with a reasonable use case for having 100 icons.

---

The UI on the icon itself in the list views is a little iffy -- mostly, it's too saturated/bold.

I'd ideally like to try either:

  - rendering a "shade" version (i.e. lighter, less-saturated color); or
  - rendering a "shade" tag with just the icon in it.

However, there didn't seem to be a way to do the first one right now (`fa-example sh-blue` doesn't work) and the second one had weird margins/padding, so I left it like this for now. I figure we can clean it up once we build the thick nav, since that will probably also want an identical element.

(I don't want to render a full tag with the icon + name since I think that's confusing -- it looks like a project/object tag, but is not.)

Test Plan:
{F1049905}

{F1049906}

Reviewers: chad

Reviewed By: chad

Subscribers: 20after4, Luke081515.2

Maniphest Tasks: T5819, T10010

Differential Revision: https://secure.phabricator.com/D14918
2016-01-08 14:01:53 -08:00