1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-26 16:52:41 +01:00
phorge-phorge/src/applications/almanac
epriestley e6ddd6d0e9 Cache Almanac URIs for repositories
Summary:
Ref T11954. This is kind of complex and I'm not sure I want to actually land it, but it gives us a fairly good improvement for clustered repositories so I'm leaning toward moving forward.

When we make (or receive) clustered repository requests, we must first load a bunch of stuff out of Almanac to figure out where to send the request (or if we can handle the request ourselves).

This involves several round trip queries into Almanac (service, device, interfaces, bindings, properties) and generally is fairly slow/expensive. The actual data we get out of it is just a list of URIs.

Caching this would be very easy, except that invalidating the cache is difficult, since editing any binding, property, interface, or device may invalidate the cache for indirectly connected services and repositories.

To address this, introduce `PhabricatorCacheEngine`, which is an extensible engine like `PhabricatorDestructionEngine` for propagating cache updates. It has two modes:

  - Discover linked objects (that is: find related objects which may need to have caches invalidated).
  - Invalidate caches (that is: nuke any caches which need to be nuked).

Both modes are extensible, so third-party code can build repository-dependent caches or whatever. This may be overkill but even if Almanac is the only thing we use it for it feels like a fairly clean solution to the problem.

With `CacheEngine`, make any edit to Almanac stuff propagate up to the Service, and then from the Service to any linked Repositories.

Once we hit repositories, invalidate their caches when Almanac changes.

Test Plan:
  - Observed a 20-30ms performance improvement with `ab -n 100`.
  - (The main page making Conduit calls also gets a performance improvement, although that's a little trickier to measure directly.)
  - Added debugging code to the cache engine stuff to observe the linking and invalidation phases.
  - Made invalidation throw; verified that editing properties, bindings, etc, properly invalidates the cache of any indirectly linked repositories.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T11954

Differential Revision: https://secure.phabricator.com/D17000
2016-12-06 09:14:45 -08:00
..
application Clean up some old cluster-ish documentation 2016-04-12 19:14:08 -07:00
capability Simplify locking of Almanac cluster services 2016-02-25 03:38:39 -08:00
conduit Store Almanac "service types" instead of "service classes" 2016-02-26 06:21:50 -08:00
controller Fix getIcon calls in PHUIObjectListItem 2016-08-19 09:35:09 -07:00
editor Update EditEngine for two column 2016-03-28 09:18:55 -07:00
engineextension Cache Almanac URIs for repositories 2016-12-06 09:14:45 -08:00
management Deactivate SSH keys instead of destroying them completely 2016-05-18 14:54:28 -07:00
phid Rough-in Almanac namespaces 2016-02-22 04:58:35 -08:00
property Allow Almanac properties to be deleted, use EditEngine instead of CustomField 2016-02-22 11:28:26 -08:00
query Modernize Nuance queries and search engines 2016-03-07 15:50:47 -08:00
servicetype Store Almanac "service types" instead of "service classes" 2016-02-26 06:21:50 -08:00
storage Remove empty implementations of describeAutomaticCapabilities() 2016-11-09 15:24:22 -08:00
typeahead Store Almanac "service types" instead of "service classes" 2016-02-26 06:21:50 -08:00
util Make repository daemon locks more granular and forgiving 2016-05-13 05:17:27 -07:00
view Allow Almanac bindings to be disabled and unused interfaces to be removed 2016-02-26 06:21:23 -08:00