1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-14 10:52:41 +01:00
Commit graph

11 commits

Author SHA1 Message Date
epriestley
42c0f060d5 Push feed publishing deeper into the task queue
Summary:
Ref T2852. I want to model Asana integration as a response to feed events. Currently, we queue one feed event for each HTTP hook.

Instead, always queue one feed event and then have it queue any necessary followup events (now, http hooks; soon, asana).

Add a script to make it easy to reproducibly fire feed event publishing.

Test Plan:
Republished a feed event and verified it hit configured HTTP hooks correctly.

  $ ./bin/feed republish 5765774156541908292 --trace
  >>> [2] <connect> phabricator2_feed
  <<< [2] <connect> 1,660 us
  >>> [3] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [3] <query> 595 us
  >>> [4] <connect> phabricator2_differential
  <<< [4] <connect> 760 us
  >>> [5] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [5] <query> 478 us
  >>> [6] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [6] <query> 449 us
  >>> [7] <connect> phabricator2_user
  <<< [7] <connect> 1,062 us
  >>> [8] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [8] <query> 540 us
  >>> [9] <connect> phabricator2_file
  <<< [9] <connect> 951 us
  >>> [10] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [10] <query> 498 us
  >>> [11] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [11] <query> 507 us
  Republishing story...
  >>> [12] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [12] <query> 685 us
  >>> [13] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [13] <query> 489 us
  >>> [14] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [14] <query> 512 us
  >>> [15] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [15] <query> 601 us
  >>> [16] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [16] <query> 405 us
  >>> [17] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [17] <query> 551 us
  >>> [18] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [18] <query> 507 us
  >>> [19] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [19] <query> 428 us
  >>> [20] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [20] <query> 419 us
  >>> [21] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [21] <query> 591 us
  >>> [22] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [22] <query> 406 us
  >>> [23] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [23] <query> 593 us
  >>> [24] <http> http://127.0.0.1/derp/
  <<< [24] <http> 746,157 us
  [2013-06-24 20:23:26] EXCEPTION: (HTTPFutureResponseStatusHTTP) [HTTP/500] Internal Server Error

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6291
2013-06-25 16:29:47 -07:00
vrana
b3a63a62a2 Introduce PhabricatorEmptyQueryException
Summary: It's dumb to execute a query which we know will return an empty result.

Test Plan: Looked at comment preview with "11", didn't see "1 = 0" in DarkConsole.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5177
2013-03-06 19:22:00 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
epriestley
696a1b22ba Make feed stories properly respect object policies
Summary:
  - When a feed story's primary object is a Policy object, use its visibility policy to control story visibility. Leave an exception for
  - Augment PhabricatorPolicyAwareQuery so queries may do pre-policy filtering without the need to handle their own buffering/cursor code. (We could slightly improve this: if a query returns less than a page of pre-filtered results we could keep getting pre-filtered results until we had at least a page's worth and then filter them all at once.)
  - Load and attach "required objects" to feed stories. We need this for policies anyway, and it will let us simplify story implementations by sourcing data directly from the object when we don't have some need to denormalize it (e.g., "title was changed from X to Y" needs to save the values of X and Y from when we published the story, but "user asked question X" can reflect the current version of the question).

Test Plan: Loaded main feed, project feed, notification menu / dropdown, notificaiton list, paginated things.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3783
2012-10-23 12:01:59 -07:00
epriestley
51c4b199d0 Allow policy-aware queries to prefilter results
Summary:
Provides a simple way for policy-aware queries to pre-filter results without needing to maintain separate cursors, and fixes a bunch of filter-related edge cases.

  - For reverse-paged cursor queries, we previously reversed each individual set of results. If the final result set is built out of multiple pages, it's in the wrong order overall, with each page in the correct order in sequence. Instead, reverse everything at the end. This also simplifies construction of queries.
  - `AphrontCursorPagerView` would always render a "<< First" link when paging backward, even if we were on the first page of results.
  - Add a filtering hook to let queries perform in-application pre-policy filtering as simply as possible (i.e., without maintaing their own cursors over the result sets).

Test Plan: Made feed randomly prefilter half the results, and paged forward and backward. Observed correct result ordering, pagination, and next/previous links.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3787
2012-10-23 12:01:11 -07:00
vrana
511a8bae34 Optimize feed query
Summary:
This was killing us. `EXPLAIN` after:

<table>
<tr><th>id</th><th>select_type</th><th>table</th><th>type</th><th>possible_keys</th><th>key</th><th>key_len</th><th>ref</th><th>rows</th><th>Extra</th></tr>
<tr><td>1</td><td>SIMPLE</td><td>story</td><td>index</td><td>chronologicalKey</td><td>chronologicalKey</td><td>8</td><td></td><td>201</td><td> </td></tr>
<tr><td>1</td><td>SIMPLE</td><td>ref</td><td>ref</td><td>chronologicalKey</td><td>chronologicalKey</td><td>8</td><td>phabricator_feed.story.chronologicalKey</td><td>1</td><td>Using index</td></tr>
</table>

Test Plan: /feed/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3628
2012-10-04 16:07:39 -07:00
epriestley
a1df1f2b70 Allow projects to be set as policies
Summary:
  - Renames `PhabricatorPolicyQuery` to `PhabricatorPolicyAwareQuery` (a query which respects policy settings).
  - Introduces `PhabricatorPolicyQuery`, which loads available policies (e.g., "member of project X").
  - Introduces `PhabricatorPolicy`, which describes a policy.
  - Allows projects to be set as policies.
  - Allows Paste policies to be edited.
  - Covers crazy cases where you make projects depend on themselves or each other because you are a dastardly villan.

Test Plan: Set paste and project policies, including crazy policies like A -> B -> A, A -> A, etc.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3476
2012-09-13 10:15:08 -07:00
epriestley
ed4a155c91 Rename "IDPaged" to "CursorPaged", "executeWithPager" to "executeWith[Cursor|Offset]Pager"
Summary:
I'm trying to make progress on the policy/visibility stuff since it's a blocker for Wikimedia.

First, I want to improve Projects so they can serve as policy groups (e.g., an object can have a visibility policy like "Visible to: members of project 'security'"). However, doing this without breaking anything or snowballing into a bigger change is a bit awkward because Projects are name-ordered and we have a Conduit API which does offset paging. Rather than breaking or rewriting this stuff, I want to just continue offset paging them for now.

So I'm going to make PhabricatorPolicyQuery extend PhabricatorOffsetPagedQuery, but can't currently since the `executeWithPager` methods would clash. These methods do different things anyway and are probably better with different names.

This also generally improves the names of these classes, since cursors are not necessarily IDs (in the feed case, they're "chronlogicalKeys", for example). I did leave some of the interals as "ID" since calling them "Cursor"s (e.g., `setAfterCursor()`) seemed a little wrong -- it should maybe be `setAfterCursorPosition()`. These APIs have very limited use and can easily be made more consistent later.

Test Plan: Browsed around various affected tools; any issues here should throw/fail in a loud/obvious way.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3177
2012-08-07 11:54:06 -07:00
epriestley
310cf00fc3 Consolidate feed query code
Summary: Simplify FeedQuery by making it extend from PhabricatorIDPagedPolicyQuery

Test Plan: Looked at feed on home, projects, user profile, and called `feed.query`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2905
2012-07-02 15:41:19 -07:00
epriestley
d49b06c322 Consolidate PhabricatorFeedStory construction code
Summary: This code is duplicated in two places; share it.

Test Plan: Looked at feed and notifications.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2901
2012-07-02 10:37:22 -07:00
vrana
6cc196a2e5 Move files in Phabricator one level up
Summary:
- `kill_init.php` said "Moving 1000 files" - I hope that this is not some limit in `FileFinder`.
- [src/infrastructure/celerity] `git mv utils.php map.php; git mv api/utils.php api.php`
- Comment `phutil_libraries` in `.arcconfig` and run `arc liberate`.

NOTE: `arc diff` timed out so I'm pushing it without review.

Test Plan:
/D1234
Browsed around, especially in `applications/repository/worker/commitchangeparser` and `applications/` in general.

Auditors: epriestley

Maniphest Tasks: T1103
2012-06-01 12:32:44 -07:00
Renamed from src/applications/feed/query/PhabricatorFeedQuery.php (Browse further)