Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
<?php
|
|
|
|
|
|
|
|
final class PhabricatorUserCache extends PhabricatorUserDAO {
|
|
|
|
|
|
|
|
protected $userPHID;
|
|
|
|
protected $cacheIndex;
|
|
|
|
protected $cacheKey;
|
|
|
|
protected $cacheData;
|
|
|
|
protected $cacheType;
|
|
|
|
|
|
|
|
protected function getConfiguration() {
|
|
|
|
return array(
|
|
|
|
self::CONFIG_TIMESTAMPS => false,
|
|
|
|
self::CONFIG_COLUMN_SCHEMA => array(
|
|
|
|
'cacheIndex' => 'bytes12',
|
|
|
|
'cacheKey' => 'text255',
|
|
|
|
'cacheData' => 'text',
|
|
|
|
'cacheType' => 'text32',
|
|
|
|
),
|
|
|
|
self::CONFIG_KEY_SCHEMA => array(
|
|
|
|
'key_usercache' => array(
|
|
|
|
'columns' => array('userPHID', 'cacheIndex'),
|
|
|
|
'unique' => true,
|
|
|
|
),
|
|
|
|
'key_cachekey' => array(
|
|
|
|
'columns' => array('cacheIndex'),
|
|
|
|
),
|
|
|
|
'key_cachetype' => array(
|
|
|
|
'columns' => array('cacheType'),
|
|
|
|
),
|
|
|
|
),
|
|
|
|
) + parent::getConfiguration();
|
|
|
|
}
|
|
|
|
|
|
|
|
public function save() {
|
|
|
|
$this->cacheIndex = Filesystem::digestForIndex($this->getCacheKey());
|
|
|
|
return parent::save();
|
|
|
|
}
|
|
|
|
|
|
|
|
public static function writeCache(
|
|
|
|
PhabricatorUserCacheType $type,
|
|
|
|
$key,
|
|
|
|
$user_phid,
|
|
|
|
$raw_value) {
|
2016-06-03 16:24:55 +02:00
|
|
|
self::writeCaches(
|
|
|
|
array(
|
|
|
|
array(
|
|
|
|
'type' => $type,
|
|
|
|
'key' => $key,
|
|
|
|
'userPHID' => $user_phid,
|
|
|
|
'value' => $raw_value,
|
|
|
|
),
|
|
|
|
));
|
|
|
|
}
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
|
2016-06-03 16:24:55 +02:00
|
|
|
public static function writeCaches(array $values) {
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
if (PhabricatorEnv::isReadOnly()) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2016-06-03 16:24:55 +02:00
|
|
|
if (!$values) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
$table = new self();
|
|
|
|
$conn_w = $table->establishConnection('w');
|
|
|
|
|
2016-06-03 16:24:55 +02:00
|
|
|
$sql = array();
|
|
|
|
foreach ($values as $value) {
|
|
|
|
$key = $value['key'];
|
|
|
|
|
|
|
|
$sql[] = qsprintf(
|
|
|
|
$conn_w,
|
|
|
|
'(%s, %s, %s, %s, %s)',
|
|
|
|
$value['userPHID'],
|
|
|
|
PhabricatorHash::digestForIndex($key),
|
|
|
|
$key,
|
|
|
|
$value['value'],
|
|
|
|
$value['type']->getUserCacheType());
|
|
|
|
}
|
|
|
|
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
$unguarded = AphrontWriteGuard::beginScopedUnguardedWrites();
|
|
|
|
|
2016-06-03 16:24:55 +02:00
|
|
|
foreach (PhabricatorLiskDAO::chunkSQL($sql) as $chunk) {
|
|
|
|
queryfx(
|
|
|
|
$conn_w,
|
|
|
|
'INSERT INTO %T (userPHID, cacheIndex, cacheKey, cacheData, cacheType)
|
|
|
|
VALUES %Q
|
2016-06-05 02:39:33 +02:00
|
|
|
ON DUPLICATE KEY UPDATE
|
|
|
|
cacheData = VALUES(cacheData),
|
|
|
|
cacheType = VALUES(cacheType)',
|
2016-06-03 16:24:55 +02:00
|
|
|
$table->getTableName(),
|
|
|
|
$chunk);
|
|
|
|
}
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
|
|
|
|
unset($unguarded);
|
|
|
|
}
|
|
|
|
|
2016-12-06 14:02:12 +01:00
|
|
|
public static function readCaches(
|
|
|
|
PhabricatorUserCacheType $type,
|
|
|
|
$key,
|
|
|
|
array $user_phids) {
|
|
|
|
|
|
|
|
$table = new self();
|
|
|
|
$conn = $table->establishConnection('r');
|
|
|
|
|
|
|
|
$rows = queryfx_all(
|
|
|
|
$conn,
|
|
|
|
'SELECT userPHID, cacheData FROM %T WHERE userPHID IN (%Ls)
|
|
|
|
AND cacheType = %s AND cacheIndex = %s',
|
|
|
|
$table->getTableName(),
|
|
|
|
$user_phids,
|
|
|
|
$type->getUserCacheType(),
|
|
|
|
PhabricatorHash::digestForIndex($key));
|
|
|
|
|
|
|
|
return ipull($rows, 'cacheData', 'userPHID');
|
|
|
|
}
|
|
|
|
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
public static function clearCache($key, $user_phid) {
|
2016-06-05 03:19:16 +02:00
|
|
|
return self::clearCaches($key, array($user_phid));
|
|
|
|
}
|
|
|
|
|
|
|
|
public static function clearCaches($key, array $user_phids) {
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
if (PhabricatorEnv::isReadOnly()) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2016-06-05 03:19:16 +02:00
|
|
|
if (!$user_phids) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
$table = new self();
|
|
|
|
$conn_w = $table->establishConnection('w');
|
|
|
|
|
|
|
|
$unguarded = AphrontWriteGuard::beginScopedUnguardedWrites();
|
|
|
|
|
|
|
|
queryfx(
|
|
|
|
$conn_w,
|
2016-06-05 03:19:16 +02:00
|
|
|
'DELETE FROM %T WHERE cacheIndex = %s AND userPHID IN (%Ls)',
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
$table->getTableName(),
|
|
|
|
PhabricatorHash::digestForIndex($key),
|
2016-06-05 03:19:16 +02:00
|
|
|
$user_phids);
|
Provide a general-purpose, modular user cache for settings and other similar data
Summary:
Ref T4103. Currently, we issue a `SELECT * FROM user_preferences ... WHERE userPHID = ...` on every page to load the viewer's settings.
There are several other questionable data accesses on every page too, most of which could benefit from improved caching strategies (see T4103#178122).
This query will soon get more expensive, since it may need to load several objects (e.g., the user's settings and their "role profile" settings). Although we could put that data on the User and do both in one query, it's nicer to put it on the Preferences object ("This inherits from profile X") which means we need to do several queries.
Rather than paying a greater price, we can cheat this stuff into the existing query where we load the user's session by providing a user cache table and doing some JOIN magic. This lets us issue one query and try to get cache hits on a bunch of caches cheaply (well, we'll be in trouble at the MySQL JOIN limit of 61 tables, but have some headroom).
For now, just get it working:
- Add the table.
- Try to get user settings "for free" when we load the session.
- If we miss, fill user settings into the cache on-demand.
- We only use this in one place (DarkConsole) for now. I'll use it more widely in the next diff.
Test Plan:
- Loaded page as logged-in user.
- Loaded page as logged-out user.
- Examined session query to see cache joins.
- Changed settings, saw database cache fill.
- Toggled DarkConsole on and off.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T4103
Differential Revision: https://secure.phabricator.com/D16001
2016-06-01 21:14:39 +02:00
|
|
|
|
|
|
|
unset($unguarded);
|
|
|
|
}
|
|
|
|
|
|
|
|
public static function clearCacheForAllUsers($key) {
|
|
|
|
if (PhabricatorEnv::isReadOnly()) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
$table = new self();
|
|
|
|
$conn_w = $table->establishConnection('w');
|
|
|
|
|
|
|
|
$unguarded = AphrontWriteGuard::beginScopedUnguardedWrites();
|
|
|
|
|
|
|
|
queryfx(
|
|
|
|
$conn_w,
|
|
|
|
'DELETE FROM %T WHERE cacheIndex = %s',
|
|
|
|
$table->getTableName(),
|
|
|
|
PhabricatorHash::digestForIndex($key));
|
|
|
|
|
|
|
|
unset($unguarded);
|
|
|
|
}
|
|
|
|
|
|
|
|
}
|