Summary:
Splits out the SQL changes. These are most of the changes, but primarily mechanical:
- Moved "initialize.sql" to "0000.legacy.sql" and partially reverted to an older version, such that patches 0000 + 000 + 001 + ... + 137 put us in the right state when applied sequentially.
- Removed "create database" commands from all SQL. These are handled by separate DB patches now, so we have the data to do operations like "storage databases" (list databases) and "storage destroy" (drop databases).
- Removed "phabricator_" namespace from all SQL, and replaced with "{$NAMESPACE}_" token so we can namespace databases.
- Shortened some column lengths so patches apply correctly if originally created as InnoDB; also a few similar tweaks elsewhere.
Test Plan: See D2323 for discussion and test plan.
Reviewers: edward, vrana, btrahan, jungejason
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T140, T345
Differential Revision: https://secure.phabricator.com/D2329
Summary:
D477 added functionality to the project list view but had a couple of
performance issues that I missed in review, because it took the query count for
the page from around 3 to as many as 300, including up to 100 heavyweight search
index queries.
This fixes the two simple N+1 query problems. This general pattern of data
access often occurs:
COUNTEREXAMPLE
$cats = load_cats();
foreach ($cats as $cat) {
$cats_hats = load_hats_for_cat($cat);
// ...
}
But this issues "N+1" queries, i.e. if you load 100 cats you issue 101 queries.
It is faster to group the queries instead:
$cats = load_cats();
$hats = load_all_hats_for_these_cats($cats);
foreach ($cats as $cat) {
$cats_hats = $hats[$cat->getID()];
}
MySQL can execute one query which returns all the results much faster than 100
queries which return one result, especially if the database is not local (i.e.,
over the network).
However, this doesn't save a ton of time. The bigger issue is that I didn't have
the right keys on the relationship tables in the search engine. This adds them,
and reduces the search engine lookup cost from 25-80ms (for
secure.phabricator.com) down to 1-3ms.
I still probably want to get this out of the loop at some point but it's okay
for now and the page loads in a few ms rather than taking more than a second.
Test Plan:
Used "services" tab, "xhprof" and "EXPLAIN" to analyze page performance. I
measured these changes:
- Query count: 1 + (3 * N projects) -> 3 + (N projects) (e.g., 301 -> 103)
- Total time spent querying, ignoring search indexes: 40ms (local.aprhont.com)
-> 20ms (local.aphront.com)
- Cost for search index query: 25-80ms (secure.phabricator.com) -> 1-3ms
Reviewed By: cadamo
Reviewers: cadamo, aran, jungejason, tuomaspelkonen
CC: aran, cadamo, epriestley
Differential Revision: 485
