Summary:
Some time ago, we added `ORDER BY id ASC` to the worker `UPDATE ...` query, because someone reported that their MySQL read slaves were complaining about the query (I can't find the exact error message, but something to the effect of the rows the query affected not being deterministic). This seemed harmless since it should be the same as the query's implicit order (I guess?), but actually made the query dramatically slower for large numbers of rows.
On my local machine, this query takes about 2 seconds with ~1M rows. If I run `SELECT`, or run `UPDATE` without ORDER BY, the query takes < 0.01s. I don't understand exactly what's happening -- my guess is something to do with the ORDER BY implying that a lot of rows need to be locked?
In T2372, a user is seeing 20-60s rumtimes on this query.
I solved this by doing a SELECT, followed by an UPDATE. Each query runs quickly. This introduces the possibility of a race (two processes SELECT the same rows, then try to UPDATE), which we currently recover from by having the second UPDATE fail and then having that daemon try again 1 second later. This seems generally reasonable. Some alternatives I considered:
- We could SELECT ... LOCK FOR UPDATE, but failing and retrying a little later seems at least as good as blocking.
- We could select more rows than we need, and then try to lock some of them randomly. I think this would work well, but it's a bit more complex than what we're doing now so I left it until we have a clearer need.
Test Plan:
Inserted ~1M tasks into the queue. Ran `phd debug taskmaster`, saw ~2s task updates. Applied patch. Ran `phd debug taskmaster`, saw <1ms updates. Ran `phd launch 8 taskmaster`, saw rapid completion of tasks.
This stuff also has fairly thorough unit test coverage.
Reviewers: vrana, btrahan
Reviewed By: vrana
CC: aran
Maniphest Tasks: T2372
Differential Revision: https://secure.phabricator.com/D4576
Summary:
In D4359 I fixed an error with 'lint' in SVN repositories, but created an error with the 'lint' column in Javascript. Specifically, when we load the column information over Ajax, we now always include a 'lint' key, even if there is no lint column.
Instead, access the 'lint' property conditionally (so SVN works) but don't include the key if there's no data (so Javascript works).
Test Plan: Loaded SVN, non-SVN non-lint, non-SVN+lint repositories. Everything appeared to work correctly.
Reviewers: asherkin, codeblock
Reviewed By: codeblock
CC: aran
Differential Revision: https://secure.phabricator.com/D4578
Summary: These have been marked as deprecated since May 2012. Clean them up.
Test Plan: Grepped for `repository-launch`, `phd_load_tracked_repositories`: no hits.
Reviewers: btrahan, vrana
Reviewed By: vrana
CC: aran
Maniphest Tasks: T2372
Differential Revision: https://secure.phabricator.com/D4575
Summary:
Makes various fixes to the Daemon console UI:
- Removes timeline, timeline cursors, and timeline-related controllers. This abstraction is all but dead and just waiting on an eventual cleanup effort with Facebook (see T2003). There's no need to inspect or debug it anymore.
- Instead of showing the 15 most recently launched non-exited daemons, show all the running daemons. With the old rule, "dead" daemons tended to build up at the bottom of the list -- e.g., secure.phabricator.com shows the 7 active daemons, then 8 dead daemons from as far back as Aug 2012. Showing running daemons is far more useful.
- Simplify the two "Running Daemons" and "All Daemons" subviews into one "All Daemons" subview. The main console now has "running daemons", effectively.
- Create a "Recently completed tasks" view, which shows how many tasks of each task class have completed in the last 15 minutes and how long they took on average. Understanding how quickly tasks are completing is one of the most common uses of the daemon console, and it's currently almost useless for that. Now that we archive tasks, we can show this information in an easily digestable form.
- Partially modernize all of the remaining views.
Test Plan: Looked at daemon console.
Reviewers: btrahan, chad, vrana
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2372, T2003
Differential Revision: https://secure.phabricator.com/D4573
Summary: ignore - array - Array of nicks to ignore all mesages from
Test Plan: run phabot with ignore set
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4582
Summary: The easiest approach here is proably to provide a more specific rule in the sheet CSS. This saves us from having to write any JS, notably.
Test Plan: Hovered over "+" on homepage.
Reviewers: chad, btrahan
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D4579
Summary: D4453 and D4427 sailed past one another, like ships in the night.
Test Plan: Verified Differential hover and selected states.
Reviewers: asherkin, chad
Reviewed By: chad
CC: aran
Differential Revision: https://secure.phabricator.com/D4577
Summary: Might not be the cleanest way to do this, but seems to work.
Test Plan:
- Saved an option which used the new enum type.
- Changed it.
- Saw it show up on the list view.
Reviewers: epriestley, chad
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2255
Differential Revision: https://secure.phabricator.com/D4572
Summary: Simple dialog for creating memes. We can add more features (typeahead, selection thumbs, preview) later.
Test Plan: {F29815}
Reviewers: DeedyDas, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T2353
Differential Revision: https://secure.phabricator.com/D4557
Summary:
See title.
Also some minor styling/consistency fixes.
Test Plan:
- Clicked subscribe
- Canceled to make sure it went away
- Clicked it again
- Clicked subscribe
- Saw my name in the cc field.
Reviewers: epriestley, chad, btrahan
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4571
Summary:
As far as I know, we never actually need `php` to be available from the web UI. I think the history here is:
- Long ago, we checked for 'pcntl' as an extension during setup.
- Someone had an install where 'pcntl' was available from the CLI, but not the web UI. So we switched the check to use the CLI.
- Someone had an install where the CLI binary was php-fpm, which caused the 'pcntl' check to loop endlessly, so we added more checks.
But we don't actually need to do any of this -- when the user tries to run the daemons, they get an explicit message that they need to install pcntl already, and we never (as far as I know) try to run PHP scripts from the web UI other than the pcntl_available.php check (we only run `git`, `svn`, `hg`, `ssh-agent`, `diff`, `xhpast` and `pygmentize`, I think).
Test Plan: Thought carefully about places we might execute PHP scripts from the web UI. Looked through /scripts/ to try to identfiy anything we might execute.
Reviewers: btrahan, vrana
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D4568
Summary: Removes unneeded filters. Also - how do I get it to highlight the firs nav item? I assumed '' would match.
Test Plan: Review calendar app
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4547
Summary: Fixes T2369. If you create a task with a description like `= Header =`, the Excel writer interprets it as a formula. Explicitly set the cell types to strings to avoid this.
Test Plan: Exported a task with the description `=1,`; no exception after this patch.
Reviewers: btrahan, chad, vrana
Reviewed By: chad
CC: aran
Maniphest Tasks: T2369
Differential Revision: https://secure.phabricator.com/D4567
Summary: Fixes T2329. Adds crumbs and gets rid of the double-depth menus.
Test Plan: Looked at transcript list, test controller and transcript detail.
Reviewers: chad, btrahan
Reviewed By: chad
CC: aran
Maniphest Tasks: T2329
Differential Revision: https://secure.phabricator.com/D4564
Summary: Fixes T1330. This has been broken with a TODO since we open sourced. :/
Test Plan: Used test console to verify this produced the correct field value for several commits.
Reviewers: btrahan, chad, vrana
Reviewed By: chad
CC: aran
Maniphest Tasks: T1330
Differential Revision: https://secure.phabricator.com/D4563
Summary: Fixes T2243. We recently added the FileTreeView to Diffusion commits. However, if the page doesn't have any changesets (e.g., it has an error message instead, like "this commit hasn't imported yet"), we fail to build a file tree. In this case, don't try to build one.
Test Plan: Looked at not-imported and imported commits in Diffusion, saw proper rendering/crumbs and no exceptions.
Reviewers: btrahan, chad, vrana
Reviewed By: chad
CC: aran
Maniphest Tasks: T2243
Differential Revision: https://secure.phabricator.com/D4562
Summary:
Pholio tile to show up on home page
First image is shown at mock page.
Test Plan: Pholio app shows up on home page. First image is shown at mock page.
Reviewers: epriestley
CC: aran, Korvin
Maniphest Tasks: T2356
Differential Revision: https://secure.phabricator.com/D4553
Summary: Fix spacing when there are no tasks, remove a panel background.
Test Plan: reload page, check other maniphest pages
Reviewers: codeblock
Reviewed By: codeblock
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D4558
Summary: Fixes T2330. Deletes the last traces of the footer.
Test Plan: Looked at several pages, saw no footer. Grepped for footer function and CSS class.
Reviewers: DeedyDas, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T2330
Differential Revision: https://secure.phabricator.com/D4554
Summary: Fixes T2323. Prior to search refactoring (I think), this stuff was loaded implicitly. Now load it explicitly.
Test Plan: Added a comment to a Ponder answer without an exception.
Reviewers: btrahan, vrana
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2323
Differential Revision: https://secure.phabricator.com/D4536
Summary: Just removed the link and created a new field under preferences. Now the setting is under Display Preferences.
Test Plan: Enablied/Disabled dark console to see if it works.
Reviewers: epriestley
Reviewed By: epriestley
CC: irinav, aran, Korvin
Maniphest Tasks: T2344
Differential Revision: https://secure.phabricator.com/D4549
Conflicts:
src/view/page/PhabricatorStandardPageView.php
Summary: Fixed T2349
Test Plan: Temporarily added a VERSION file with some text and it rendered it correctly.
Reviewers: epriestley
CC: aran, Korvin
Maniphest Tasks: T2349
Differential Revision: https://secure.phabricator.com/D4544
Summary: Unit test for T2345
Test Plan: Ran unit tests, checked that it passed.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4552
Summary:
T2154
Adding the Conduit query method implementation, and metadata to the phutil register library.
Test Plan:
Choose conduit.query on the web UI to see information about the method.
Then, click the "Call Method" button and observe the method result.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4550
Summary: Removes the panel-view on login and adds additonal responsive styles for mobile forms.
Test Plan: View in mobile browser, resize page.
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4530
Summary: Created "Applications" application which lists all the installed applications in phabricator.
Test Plan: Navigated to localphabricatorinstall.com/applications and check whether it actually shows the list of all installed applications.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2352
Differential Revision: https://secure.phabricator.com/D4548
Summary: Fixes T2357. In D4529, I recommended we just hide tiles for beta applications on installs without beta apps enabled. However, this creates at least a couple of issues, notably the stuff in T2357. The main app I was concerned wih Config, which we ended up un-beta'ing anyway, so I think this will probably solve more problems than it creates.
Test Plan: Verified routes for beta applications dropped out when beta was disabled. Verified "Customize Applications" no longer shows these applications.
Reviewers: ljalonen, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T2357
Differential Revision: https://secure.phabricator.com/D4546
Summary: Prior to re-doing responsive form CSS, update the Herald app for no panel backgrounds.
Test Plan: Went through Herald application, see no panels.
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4545
Summary:
T2345
getConfig throws an Exception when the key does not exist.
Also removes dead code that throws an Exception.
Test Plan:
Reloaded the Phabricator home page. In the process, found
2 Exceptions thrown due to nonexistent keys. After addressing these problems,
the home page loads without Exceptions.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4541
Summary: Fixed T2354
Test Plan: Testing in progress until this task is resolved.
Reviewers: epriestley
CC: aran, Korvin
Maniphest Tasks: T2354
Differential Revision: https://secure.phabricator.com/D4542
Summary: Pholio to show up on home page
Test Plan: Pholio app shows up on home page
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2355
Differential Revision: https://secure.phabricator.com/D4543
Summary: Fixed T2349
Test Plan:
Could not visibly see version at footer any more. Appeared in the top of /config.
Does not appear as a config option in /config.
Reviewers: epriestley
CC: aran, Korvin
Maniphest Tasks: T2349
Differential Revision: https://secure.phabricator.com/D4539
Summary: See rP883829e6676fc3412b83b6ab16f7bf5b56b174b8
Test Plan: Verified no XSS with a title like `<b>!</b>`.
Reviewers: vrana, btrahan
Reviewed By: vrana
CC: aran
Differential Revision: https://secure.phabricator.com/D4534
Summary: I've stored `PhutilSafeHTML` instance to cache on devbox and then wasn't able to read it in production.
Test Plan: Displayed revision with unreadable cache, saw error in error log but not fatal.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Differential Revision: https://secure.phabricator.com/D4520
Summary: Fixes T2316
Test Plan:
When the config file allows reopening,
navigate to a closed revision and reopen it in the user interface,
and verify that the revision now "needs review."
Also checks that the reopen option is unavailable when disallowed
by the config file.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2316
Differential Revision: https://secure.phabricator.com/D4526
Summary: Technically we should have these for all the OAuth providers but I don't think anyone really has trouble with them and it can probably be done generically after T1536. Preserve the functionality, at least.
Test Plan: Broke my config, verified warnings appeared.
Reviewers: btrahan, vrana
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2228
Differential Revision: https://secure.phabricator.com/D4506
Summary: These are nonblocking warnings and can move to post-install.
Test Plan: Broke my environment and observed the warnings.
Reviewers: btrahan, vrana
Reviewed By: vrana
CC: aran, asherkin
Maniphest Tasks: T2228
Differential Revision: https://secure.phabricator.com/D4505
Summary:
Migrate to the new hotness. Also:
- Remove a string test, which is now impossible since the config will repair itself and raise a type error.
- Restore the header even in /config/ -- this check is kind of hacky and it feels a bit more natural now that it's above the menu.
Test Plan: Set my local disk path to something invalid, verified I got a setup error.
Reviewers: btrahan, vrana
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T2228
Differential Revision: https://secure.phabricator.com/D4497
