Summary:
Ref T10759. Check master/replica status during startup.
After D16903, this also means that we check this status after a database comes back online after being unreachable.
If a master is replicating, fatal (since this can do a million kinds of bad things).
If a replica is not replicating, warn (this just means the replica is behind so some data is at risk).
Also: if your masters were actually configured properly (mine weren't until this change detected it), we would throw away patches as we applied them, so they would only apply to the //first// master. Instead, properly apply all migration patches to all masters.
Test Plan:
- Started Phabricator with a replicating master, got a fatal.
- Stopped replication on a replica, got a warning.
- With two non-replicating masters, upgraded storage.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10759
Differential Revision: https://secure.phabricator.com/D16904
Summary:
Ref T10759. We may "discover" the presence of a fatal setup error later, after starting Phabricator.
This can happen in a few ways, but most are unlikely. The one I'm immediately concerned about is:
- Phabricator starts up during a disaster with some databases unreachable.
- We start with warnings (unreachable databases are generally not fatal, since it's OK for some subset of hosts to be down in replicated/partitioned setups).
- The unreachable databases later recover and become accessible again.
- When we run checks against them, we discover that they are misconfigured.
Currently, "fatal" setup issues are not truly fatal if we're "in flight" -- we've survived setup checks at least once in the past. This is bad in the scenario above.
Especially with partitioning, it could lead to mangled data in a disaster scenario where operations staff makes a small configuration mistake while trying to get things running again.
Instead, if we "discover" a fatal error while already "in flight", reset the whole setup process as though the webserver had just restarted. Don't serve requests again until we can make it through setup without hitting fatals.
Test Plan:
- Started Phabricator with multiple masters, one of which was down and broken.
- Got a warning about the bad master.
- Revived the master.
- Before: Phabricator detects the fatal, but keeps serving requests.
- After: Phabricator detects the fatal, resets the webserver, and stops serving requests until the fatal is resolved.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10759
Differential Revision: https://secure.phabricator.com/D16903
Summary:
Ref T10759. Currently, these checks run only against configured masters. Instead, check every host.
These checks also sort of cheat through restart during a recovery, when some hosts will be unreachable: they test for "disaster" by seeing if no masters are reachable, and just skip all the checks in that case.
This is bad for at least two reasons:
- After recent changes, it is possible that //some// masters are dead but it's still OK to start. For example, "slowvote" may have no master, but everything else is reachable. We can safely run without slowvote.
- It's possible to start during a disaster and miss important setup checks completely, since we skip them, get a clean bill of health, and never re-test them.
Instead:
- Test each host individually.
- Fundamental problems (lack of InnoDB, bad schema) are fatal on any host.
- If we can't connect, raise it as a //warning// to make sure we check it later. If you start during a disaster, we still want to make sure that schemata are up to date if you later recover a host.
In particular, I'm going to add these checks soon:
- Fatal if a "master" is replicating.
- Fatal if a "replica" is not replicating.
- Fatal if a database partition config differs from web partition config.
- When we let a database off with a warning because it's down, and later upgrade it to a fatal because we discover it is broken after it comes up again, fatal everything. Currently, we keep running if we "discover" the presence of new fatals after surviving setup checks for the first time.
Test Plan:
- Configured with multiple masters, intentionally broke one (simulating a disaster where one master is lost), saw Phabricator still startup.
- Tested individual setup checks by intentionally breaking them.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T10759
Differential Revision: https://secure.phabricator.com/D16902
Summary:
Fixes T11902.
- Periods now work in short names.
- If you try to name something ".git", no dice.
Test Plan:
- Tried to name something "quack.git", was politely rejected.
- Named something "quack.notgit", and it worked fine.
- Cloned Mercurial and Git repositories over SSH with ".git" and non-".git" variants without hitting any issues.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11902
Differential Revision: https://secure.phabricator.com/D16908
Summary:
Ref T11044. I'm going to hold this until after the release cut, but I think it's good to go.
This allows installs to configure multiple masters in `cluster.databases` and partition applications across them (for example, put Maniphest on a dedicated database).
When we make a Maniphest connection we go look up which master we should be hitting first, then connect to it.
This has at least approximately been planned for many years, so the actual change is largely just making sure that your config makes sense.
Test Plan:
- Configured `db001.epriestley.com` and `db002.epriestley.com` as master/master.
- Partitioned applications between them.
- Interacted with various applications, saw writes go to the correct host.
- Viewed "Database Servers" and saw partitioning information.
- Ran schema upgrades.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11044
Differential Revision: https://secure.phabricator.com/D16876
Summary:
Ref T11893. Previously, we excuted all `bin/storage` queries through `StorageManagementAPI` objects.
After D16848, we execute some queries through `PhabricatorDatabaseRef`. However, the refs we use weren't getting passed the `-u` / `-p` flags correctly, for specifying alternate administrative credentials.
Test Plan:
- Created a second account (`trunk`).
- Ran `bin/storage -u trunk`.
- Made libphutil throw when not connecting as that user.
- Before patch: some connections incorrectly used the default user, "root".
- After patch: all connections correctly used the configured user, "trunk".
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11893
Differential Revision: https://secure.phabricator.com/D16901
Summary:
Ref T8345. See T8345#201048 for discussion.
This rule (don't show mentions of or from restricted objects) is more consistent with how we render mentions in the timeline and I think generally a better behavior.
Test Plan:
- Mentioned a task on a public task and a private task.
- Privileged user (foreground) sees both.
- Public user (background) sees only the public mention.
{F1929485}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T8345
Differential Revision: https://secure.phabricator.com/D16900
Summary: Basic work in progress, but should show timeline comments for files when in lightbox mode. Looks reasonable.
Test Plan: click on images, see comments from timeline.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Maniphest Tasks: T3612
Differential Revision: https://secure.phabricator.com/D16896
Summary: Ref T11816. We currently color circles green if you're invited, but should color them green if you're attending, too.
Test Plan: Viewed calendar mobile month view, saw attending events in green.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16899
Summary:
Ref T11816. In some cases, Calendar would only export a subset of events because the "export" flag was ignored or the "display" parameter applied an improper date range to the query.
- Make sure the `export` flag gets processed, even though it isn't a "real" field on the search engine.
- Clear the "display" parameter to avoid date range windowing coming from the day/month logic.
Test Plan: Exported a "display=month" view, verified future events came with it.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16898
Summary:
See downstream issue here: <https://phabricator.wikimedia.org/T150992>
In at least one case (project milestones) we have a locked, non-lockable field. This means "this is locked, and you can't change the fact that it is locked".
At least for now, preserve this behavior.
Test Plan: Created a new milestone of an existing project. This worked correctly with the patch.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D16895
Summary: Fixes T11882. Document using `~/.ssh/config` to mitigate the inconvenience of port 2222.
Test Plan: Read document.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11882
Differential Revision: https://secure.phabricator.com/D16894
Summary: This supports doing a bunch of sales funnel tracking on Phacility.
Test Plan: See next diff.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D16890
Summary: Ref T11034. Try to produce a roughly-one-sentence summary instead of a roughly-one-paragraph summary for the browse dialog.
Test Plan:
- Added unit tests, ran unit tests.
- Wrote a longer summary for a project, browsed to it, saw a shorter summary.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11034
Differential Revision: https://secure.phabricator.com/D16892
Summary: This code should go inside the field-locking loop. Otherwise, it only applies to the last field, and fatals if there are no fields.
Test Plan: Carefuller inspection.
Reviewers: chad
Reviewed By: chad
Subscribers: 20after4
Differential Revision: https://secure.phabricator.com/D16889
Summary:
Certain unusual queries, like `[-]`, could tokenize into a list which included the empty string.
This would then convert into a query for `... LIKE "%"` which just joins the entire table.
Instead: tokenize smarter; never return the empty token; add some test cases.
Test Plan: Ran unit tests. Queried for `[[blah blah]]`, saw a reasonable query come out the other end.
Reviewers: chad
Reviewed By: chad
Subscribers: 20after4
Differential Revision: https://secure.phabricator.com/D16888
Summary:
Ref T11887. This isn't a great fix but makes the method behave properly until I get around to a real fix.
In the longer term, I want to convert all of this pluggable Harbormaster/Drydock stuff (blueprints, artifacts, build plans) to use EditEngine + EditField instead of the weird mishmash of older/custom stuff it currently uses. However, this is a more involved project to execute and I'd like to be in that area of the codebase first so it gets adequate testing.
Until that happens, just put a reasonble-ish mechanism in place to let artifacts correct inbound types. This is the only artifact type and only parameter which needs casting.
Test Plan:
- Made a `curl` call to `harbormaster.createartifact` to create a URI artifact with `?...&ui.external=1`.
- Before patch: type error on `ui.external` not being a boolean.
- After patch: artifact created successfully.
Reviewers: chad
Reviewed By: chad
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T11887
Differential Revision: https://secure.phabricator.com/D16887
Summary:
Ref T8510. Use "\n" as a delimiter between name sections. Specifically, project "AAA" with tag "zzz" should be a better match for query "AAA" than project "AAA BBB" is.
Make use of this delimiter slighlty more obvious in the UI.
Test Plan:
- Created projects "Phacility" and "Phacility Core Access".
- Typed "Phacility".
- Before patch: first hit is "Phacility Core Access".
- After patch: first hit is "Phacility".
- Viewed debugging output table, saw visual explanation of behavior.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T8510
Differential Revision: https://secure.phabricator.com/D16886
Summary:
Fixes T11886. D16882 prevented workboard cards from being dragged between columns because it reduced the effective document height to almost nothing (~78px, just the menu bars).
Instead, use the larger of the document height and viewport height as "infinity".
Test Plan: Dragged columns between workboards successfully.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11886
Differential Revision: https://secure.phabricator.com/D16885
Summary:
Fixes T11785. Lightbox calls `JX.Stratcom.pass()` to let other handlers react, but should not. At least today, we never put, e.g., links inside a lightbox.
This code appears in the original commit so it was probably just copy/pasted from somewhere and I missed it in review.
(Or there's some edge case I'm not thinking of and we'll figure it out soon enough.)
Additionally, blacklist `/file/data/` from Quicksand naviagtion: Quicksand should never fetch these URIs.
Test Plan:
- Disabled `security.alternate-file-domain`.
- Enabled Quicksand ("Persistent Chat").
- Clicked an image thumbnail on a task.
- Repeated that until things flipped out a bit.
- After the patch: no issues.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11785
Differential Revision: https://secure.phabricator.com/D16884
Summary: Visually, I think these are much cleaner (with colors), and provide more workspace. I also don't really use the sidenav here and if I did, it would be to go back to the project homepage. I think this is overall better. If navigation page to project home is difficult or hard to find, we can maybe make a better header / crumbs bar to reduce that.
Test Plan: New project -> basic new board. Existing project -> color board. Desktop, Mobile, Fullscreen
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16882
Summary: Ref T11816. This could be a little cleaner, but we currently have two copies of the logic. Get them using the same code. Once that's actually working I can go make the code a little prettier.
Test Plan: Viewed Calendar month view tooltips, saw the same values as subheaders.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16880
Summary:
Ref T11816.
- Host gets weird behavior around defaulting to the viewer.
- Invitees get weird behavior around defaulting to the viewer.
- "All Day" is just sort of weird since start / end date aren't customizable.
- Recurring/Frequency are weird here and don't make much sense.
I can't immediately come up with reasons that any of these are particularly useful/valuable to default. More of them can be made editable after T10222 gets sorted out.
Test Plan: Edited edit engine custom forms for Calendar events, saw a more sensible list of customizable fields (e.g., policy stuff).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16878
Summary: Uh, non-all-day-events said 1:30 - 2:29 PM, which is real silly.
Test Plan: Looked at a non-all-day-event.
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D16874
Summary: Adds a headerimage and lets you set it on posts for added reverence. Is that a word?
Test Plan:
Add an image, see an image.
{F1923010}
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16873
Summary:
Fixes T11809. Ref
- Explicitly document the summary icon hints -- I don't think these are too hard to figure out (and maybe this stuff should just go in the tooltips) but we can start here.
- Use color + shape to distinguish between "cancelled" and "declined", not just color (for users with vision accessibility issues).
- Translate a "minute(s)" string into sensible English.
- Use RSVP status on the month view green circle thing.
Test Plan:
- Read docs.
- Looked at month view.
- Read reminder mail.
- Viewed month view mobile view.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11809
Differential Revision: https://secure.phabricator.com/D16872
Summary:
Ref T11816. Since the dashboard got updated, Differential now interprets "Responsible Users: epriestley" to mean "epriestley, or any project or package epriestley is part of". You can query for just "epriestley" with "exact(epriestley)".
Give Calendar invites the same behavior: "epriestley" means "any event epriestley is invited to, or a project they are a member of is invited to". Individual invites can be queried with "exact(epriestley)".
This is a little bit copy-pastey but I want to wait for a third use case to clean it up since I think I'm going to have to do a bunch of generalization around "how does an individual PHID get turned into a bunch of PHIDs".
Test Plan: Queried for "Invited: dog", "invited: viewer", "invited; exact(dog)", etc.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16870
Summary: No view engine yet (adding header image next), but adds subtitle to display like PhameBlog
Test Plan: Add a subtitle, remove a subtitle.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16871
Summary: Fixes some outlying issues with new comment box and Phame Posts.
Test Plan: Review CSS in sandbox, post new reply, see proper spacing.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16869
Summary:
Ref T11816. Projects can be invited to an event, but the UI is currently fairly agnostic about them.
Instead, introduce the idea of "RSVPs", which are basically invites for you as an individual or for any group you're a part of. When we go to check if you're invited, we check for you individually first, then check for any groups you belong to if you haven't already accepted/declined.
On the calendar detail page:
- Show the quick "Join" / "Decline" buttons if any project you're a member of is invited.
- If you're invited, highlight any projects which you're a member of to make that more clear.
On other calendar views:
- If you're invited as part of a project, show the "multiple users" icon.
- If it's just you, continue showing the "add one user" icon.
Test Plan: Viewed month view, day view, detail view. Invited groups and individuals. Invited "Dog Project", accepted invite as user "Dog".
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16868
Summary:
Ref T11816. This logic was correct, we just did all the work and then mostly threw away the results. This worked correctly anyway in some timezones.
Instead, actually use `$min_date` and `$max_date`.
Test Plan: In "America/Toronto" with server in a more-western timezone, viewed a "Nov 11 - Nov 12" all-day event, saw those dates.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16867
Summary:
Ref T11816. We currently try to expand the picker control from the little calendar icon. This works alright on desktop, but not great on tablet/mobile.
On tablet/mobile, center the control on screen instead. Also, mask the background since it can get a bit busy because we can't really control what's under the control anymore.
Finally, move the little popup thing around if the user resizes the window.
Test Plan: {F1922773}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16866
Summary: Fixes T11867. This should really be on the `CredentialType` itself, but just punt that for now until the API endpoint gets updated. We'll need the actual code here anyway in some form.
Test Plan: {F1922728}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11867
Differential Revision: https://secure.phabricator.com/D16864
Summary: Fixes T11868. This is silly and does not make sense.
Test Plan: Edited a Phurl URL, verified mail only went to me, not to the object itself.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11868
Differential Revision: https://secure.phabricator.com/D16863
Summary:
Ref T9304. This adds a "GuidanceEngine" which can generate "Guidance".
In practice, this lets third-party code (rSERVICES) remove and replace instructions in the UI, which is basically only usefulf or us to tell users to go read the documentation in the Phacility cluster.
The next diff tailors the help on the "Auth Providers" and "Create New User" pages to say "PHACILITY PHACILITY PHACILITY PHACILITY".
Test Plan: Browed to "Auth Providers" and "Create New User" on instanced and non-instanced installs, saw appropriate guidance.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9304
Differential Revision: https://secure.phabricator.com/D16861
Summary: Fixes T11866. This got converted wrong when doing the `/source/` stuff.
Test Plan: Browsed the root directory of a Subversion repository in Diffusion.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11866
Differential Revision: https://secure.phabricator.com/D16860
Summary: See D16851 - there's now a difference in their meaning, so don't unite them in the UI.
Test Plan: Load manage page of repos
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D16858
Summary:
Fixes T11818. We don't discard output, so once we read more than 2GB of output we'll exceed the maximum size of a string in an internal buffer.
Instead, configure the future so output is discarded.
Test Plan: Added logging to `libphutil/`, saw internal buffer grow steadily before this change and stay constant at 0 after this change.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11818
Differential Revision: https://secure.phabricator.com/D16855
Summary:
Ref T11818. See that task for a description.
This is like a tiny pinch of hackiness but not really unreasonable. Basically, `aphlict` is already an "uber-server" and one copy can handle a ton of instances.
Test Plan: Started `bin/aphlict` without a valid database connection.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11818
Differential Revision: https://secure.phabricator.com/D16854
Summary: Fixes T4245. When a repository has a short name, use `/source/shortname/` as its primary URI.
Test Plan:
- Cloned Git repositories from shortnames via HTTP and SSH.
- Cloned Mercurial repositories from shortnames via HTTP and SSH.
- Cloned Subversion repositories from shortnames via SSH.
- Browsed Git, Mercurial and Subversion repositories.
- Added and removed short names to various repositories.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4245
Differential Revision: https://secure.phabricator.com/D16851
Summary: Ref T11816. This got dropped somewhere along the way, so the mobile month view no longer showed a green-colored hint if a day has events you're invited to.
Test Plan: Viewed Calendar month view on mobile, saw green circles for days with invited events.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11816
Differential Revision: https://secure.phabricator.com/D16852
Summary: Depends on D16847. Ref T11044. This updates the remaining storage-related workflows from the CLI to accommodate multiple masters.
Test Plan:
- Configured multiple masters.
- Ran all `bin/storage` workflows.
- Ran `arc unit --everything`.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11044
Differential Revision: https://secure.phabricator.com/D16848
Summary:
Depends on D16115. Ref T11044. In the brave new world of multiple masters, we need to check the schemata on each master when looking for missing storage patches, keys, schema changes, etc.
This realigns all the "check out what's up with that schema" calls to work for multiple hosts, and updates the web UI to include a "Server" column and allow you to browse per-server.
This doesn't update `bin/storage`, so it breaks things on its own (and unit tests probably won't pass). I'll update that in the next change.
Test Plan: Configured local environment in cluster mode with multiple masters, saw both hosts' status reported in web UI.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11044
Differential Revision: https://secure.phabricator.com/D16847
Summary:
Ref T11044. This moves toward partitioned application databases:
- You can define multiple masters.
- Convert all the easily-convertible code to become multi-master aware.
This doesn't convert most of `bin/storage` or "Config > Database (Stuff)" yet, as both are quite involved. They still work for now, but only operate on the first master instead of all masters.
Test Plan: Configured multiple masters, browsed around, ran `bin/storage` commands, ran `bin/storage --host ...`.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11044
Differential Revision: https://secure.phabricator.com/D16115
Summary: I moved and then un-moved this incorrectly in D16846.
Test Plan: Looked at the old code, which worked better.
Reviewers: jacksongabbard, chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D16849
Summary:
Ref T11085. To recreate the issue:
- From the web UI, click "Edit Revision".
- Write something like this as your "Summary" (i.e., put another field marker, like "Test Plan:", into the summary):
> This is a test of the
> Test Plan: field to see
> if it works.
- Save changes.
Later, when the summary is amended into a commit message, the parser will see two "Test Plan:" fields and fail to parse the message.
Instead, prevent users from making this edit.
Test Plan: {F1917640}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T11085
Differential Revision: https://secure.phabricator.com/D16846
