1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-26 07:20:57 +01:00
Commit graph

550 commits

Author SHA1 Message Date
epriestley
f9fcaa1f84 Migrate project membership to edges
Summary:
  - Store project members in edges.
  - Migrate existing members to edge storage.
  - Delete PhabricatorProjectAffiliation.
  - I left the actual underlying data around just in case something goes wrong; we can delete it evenutally.

Test Plan:
  - Ran migration.
  - Created a new project.
  - Joined and left a project.
  - Added and removed project members.
  - Manually called PhabricatorOwnersOwner::loadAffiliatedUserPHIDs() to verify its behavior.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3186
2012-08-07 18:02:05 -07:00
epriestley
d5a0352fd7 Make project membership edits use transactions and PHIDs and not be awful
Summary:
  - Split project profile editing apart from project membership editing.
  - Make project membership editing simpler and easier to use.
  - Drop role / owner stuff from the UI.

Test Plan: Added and removed project members. Edited project profile information.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3184
2012-08-07 11:57:38 -07:00
epriestley
f01c89f8de Remove subprojects from the Projects UI and API
Summary:
These are currently useless and confusing (they have no application impact), and should be migrated to edges if we want to restore them in some form.

I left the actual storage so this doesn't destroy any data, it just removes all traces of this feature from the UI.

Test Plan: Looked at and edited projects.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3183
2012-08-07 11:57:27 -07:00
epriestley
d74b84a729 Add a "Project" application
Summary: Move routes into a formal application class.

Test Plan: Checked /applications/, browsed various routes.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran, floatinglomas

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3181
2012-08-07 11:54:49 -07:00
epriestley
ed4a155c91 Rename "IDPaged" to "CursorPaged", "executeWithPager" to "executeWith[Cursor|Offset]Pager"
Summary:
I'm trying to make progress on the policy/visibility stuff since it's a blocker for Wikimedia.

First, I want to improve Projects so they can serve as policy groups (e.g., an object can have a visibility policy like "Visible to: members of project 'security'"). However, doing this without breaking anything or snowballing into a bigger change is a bit awkward because Projects are name-ordered and we have a Conduit API which does offset paging. Rather than breaking or rewriting this stuff, I want to just continue offset paging them for now.

So I'm going to make PhabricatorPolicyQuery extend PhabricatorOffsetPagedQuery, but can't currently since the `executeWithPager` methods would clash. These methods do different things anyway and are probably better with different names.

This also generally improves the names of these classes, since cursors are not necessarily IDs (in the feed case, they're "chronlogicalKeys", for example). I did leave some of the interals as "ID" since calling them "Cursor"s (e.g., `setAfterCursor()`) seemed a little wrong -- it should maybe be `setAfterCursorPosition()`. These APIs have very limited use and can easily be made more consistent later.

Test Plan: Browsed around various affected tools; any issues here should throw/fail in a loud/obvious way.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D3177
2012-08-07 11:54:06 -07:00
epriestley
314c25365d Drive menu icons from Applications
Summary:
This makes no changes, it just moves the menu icons to the applications instead of hard-coded on the page.

I'm going to try to address some of the angst in T1593 next...

Test Plan: Loaded logged-in / logged out pages. Clicked menu items. Looked at /applications/.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1593, T1569

Differential Revision: https://secure.phabricator.com/D3146
2012-08-05 14:12:43 -07:00
Alan Huang
358baf1b3a Improve the flag.* Conduit methods
Summary:
 - Put the code to generate informational dicts about flags into the
   base class.
 - Update flag.delete to accept an object PHID in order to delete the
   flag on that object, since currently the model is that each object
   may have at most one flag, and each flag has exactly one object,
   although the former is not enforced.
 - Add flag.edit, which creates or updates a flag, optionally with the
   given color and note.

Test Plan:
Spend endless hours repeatedly running arc call-conduit and
arc flag.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3141
2012-08-03 11:41:33 -07:00
epriestley
194dc40672 Add a meta-application
Summary:
  - Adds a new "Applications" application.
  - Builds an application list via application config instead of via hard-coding, so we can move toward better concepts of installing/uninstalling applications, etc.
  - Applications indicate that they need attention with notice counts and brief status messages rathern than 50 giant tables of all sorts of app data.

I want to try replacing the home screen with this screen, pretty much. Not sure if this is totally crazy or not. What does everyone else think?

Test Plan: Will add screenshots.

Reviewers: btrahan, chad, vrana, alanh

Reviewed By: vrana

CC: aran, davidreuss, champo

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3129
2012-08-02 14:07:21 -07:00
Alan Huang
d31accf076 Add a flag.delete Conduit method
Summary: Fairly straightforward: allow deletion of flags from Conduit.

Test Plan: Try arc call-conduit with different ids.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1556

Differential Revision: https://secure.phabricator.com/D3132
2012-08-02 12:25:01 -07:00
Nick Harper
88caa45854 Save daemon state to database
Summary:
To make it easier to monitor daemons, let's store their current state
(running, died, exited, or unknown) to the db. The purpose of this is to
provide more information on the daemon console about the status of daemons,
especially when they are running on multiple machines. This is mostly backend
work, with only a few frontend changes. (It is also dependent on a change
to libphutil.)

These changes will make dead or stuck daemons more obvious, and will allow
more work on the frontend to hide daemons (and logs) that have exited cleanly,
i.e. ones we don't care about any more.

Test Plan:
- run db migration, check in db that all daemons were marked as exited
- start up a daemon, check in db that it is marked as running
- open web interface, check that daemon is listed as running
- after daemon has been running for a little bit, check in db that dateModified
  is being updated (indicating daemon is properly sending heartbeat)
- kill -9 daemon (but don't run bin/phd yet), and check that db still shows it
  as running
- edit daemon db entry to show it as being on a different host, and backdate
  dateModified field by 3 minutes, and check the web ui to show that the status
  is unknown.
- change db entry to have proper host, check in web ui that daemon status is
  displayed as dead. Check db to see that the status was saved.
- run bin/phd stop, and see that the formerly dead daemon is now exited.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3126
2012-08-01 17:06:04 -07:00
epriestley
a476e5c08d Include symbols in main typeahead
Summary:
  - Include symbols in main typeahead results.
  - Simplify the symbol query a bit and extend PhabricatorOffsetPagedQuery. There was some stuff around language ranking that I got rid of, I think the theory there was that mapping file extensions to languages might not work in general but I think it works fine in practice, and we have more config stuff now around guessing languages and getting the mappings right.
  - Make it easier to debug the typeahead by showing the results in page format for non-ajax requests.
  - When we have too many results, show only the top few of each type.

Depends on D3116, D3117

Test Plan: Used typeahead, got symbols in results. Hit endpoint with non-ajax, got useful debug view.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3118
2012-08-01 12:36:47 -07:00
epriestley
7be02e659a Make typeahead results more structured
Summary: Get rid of this positional array garbage.

Test Plan: Used typeaheads in menu, herald, maniphest, differential, repository/projects.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3116
2012-08-01 12:36:19 -07:00
vrana
606ef34d61 Load commit branches and tags by AJAX
Summary:
Each query takes over 2 seconds in FBCODE.
I didn't find a way how to speed them up.
There's also no easy way how to parallelize them at least.
So AJAX is the last instance.

Test Plan: Loaded commit with one branch and no tag.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3112
2012-07-31 17:01:41 -07:00
epriestley
3c7944d297 Switch to new menubar
Summary:
  - Looks better (can probably still use some tweaks), especially search.
  - Moves logout from weird footer location to main menu.
  - Reactive: on tablets and phones, the menu adjusts to remain useful.
  - Fixed position on desktops for future side nav changes.
  - Adds an icon header thing that's currently hard-coded but will be application-driven soon.

Test Plan: Used menu on desktop, tablet, phone, logged in / logged out, toggled darkconsole. Will add some screenshots.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3105
2012-07-30 16:09:14 -07:00
epriestley
c8afc741fa Add support for placeholders
Summary:
Support placeholder text for inputs. We currently don't use this because it requires JS and doesn't degrade (no JS means you have zero idea what the input is for if it isn't separately labeled) but there are some cases where intent is obvious from context (for example, the search input in the menu bar, which is fairly obvious on its own and will soon have a magnifying glass icon) and in such cases it's much prettier and saves a bunch of space over an explicit label. Add a behavior so we can add placeholders where they make sense.

This implementation is somewhat sanity-checked agianst the two jQuery placeholder implementations I was able to google:

https://github.com/danielstocks/jQuery-Placeholder/
https://github.com/mathiasbynens/jquery-placeholder

Since we don't currently have any uses cases, I haven't included support for making JS access to the `value` work, for password inputs, or for dynamically altering the placeholder.

Test Plan: Played around with the placeholder in the UI example in various browsers and couldn't break it.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3103
2012-07-30 16:08:10 -07:00
epriestley
fceabd42e8 Allow Fact app to draw charts
Summary: For any count fact, allow a chart to be drawn. INCREDIBLY POWERFUL DATA ANALYSIS PLATFORM.

Test Plan: Drew a chart of object counts. Drew the Maniphest burn chart.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3099
2012-07-30 10:44:08 -07:00
epriestley
f0af273165 Add FactCursors and application fact datasources
Summary:
  - Add PhabricatorApplication. This is a general class that I have grand designs for, but used here to allow applications to provide objects for analysis by the facts appliction.
  - Add FactCursors, to keep track of where iterators are.
  - Make the daemon do something sort of useful.
  - Add `bin/fact cursors` for showing and managing objects and cursors.
  - Add some options to `bin/fact analyze`.

Test Plan:
  - `bin/fact cursors`, `bin/fact cursors --reset DifferentialRevision`, `bin/fact cursors --reset X`
  - `bin/fact analyze`, `bin/fact analyze --all`, `bin/fact analyze --iterator DifferentialRevision --skip-aggregates`
  - `bin/phd debug fact`

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3098
2012-07-30 10:43:49 -07:00
epriestley
f652123c5a Add PhabricatorFactSpec, for naming and formatting facts
Summary: Not totally sure about this but I think it's okay?

Test Plan: Loaded /fact/, got a more readable page.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3090
2012-07-27 17:29:44 -07:00
epriestley
486f7c1e8e Add aggregated facts to the Facts application
Summary:
Some facts are aggregations of other facts. For example, we may compute how many times each macro is used in each object as a "raw fact":

  Dnnn uses macro "psyduck" 6 times.

But we want to present this data in aggregate form, e.g. "order macros by popularity". We can do this at runtime and it probably won't be too awful a query, but we can also aggregate it cheaply:

  Macro "psyduck" is used 3920 times across all objects.

...and then do a query like "select macros ordered by usage".

"Aggregate" facts support facts like this. The aggregate facts I've implemented are:

  - Count of all objects.
  - Count of objects of type X.
  - Last time facts were updated.

These clearly fit the "aggregate" facts template well. I'm not 100% sure macros do. We can use this table to answer a question like "What are the most popular macros, ordered by use?" We can also use it to answer a question like "What are the most popular macros in the last 6 months?", if we build a specific fact for that. But we can't use it to answer a question like "What are the most popular macros between times X and Y?". Maybe that's important; maybe not.

This seems like a good fit for at least some types of facts.

I'll de-magic the keys a bit in the next diff.

Test Plan: Ran the engines and got some aggregated facts about other facts.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3089
2012-07-27 13:46:01 -07:00
epriestley
7c934e4176 Add a basic "fact" application
Summary:
Basic "Fact" application with some storage, part of a daemon, and a control binary.

= Goals =

The general idea is that we have various statistics we'd like to compute, like the frequency of image macros, reviewer responsiveness, task close rates, etc. Computing these on page load is expensive and messy. By building an ETL pipeline and running it in a daemon, we can precompute statistics and just pull them out of "stats" tables.

One way to do this is just to completely hard-code everything, e.g. have a daemon that runs every hour which issues a big-ass query and dumps results into a table per-fact or per fact-group. But this has a bunch of drawbacks: adding new stuff to the pipeline is a pain, various fact aggregators can't share much code, updates are slow and expensive, we can never build generic graphs on top of it, etc.

I'm hoping to build an ETL pipeline which is generic enough that we can use it for most things we're interested in without needing schema changes, and so that installs can use it also without needing schema changes, while still being specific enough that it's fast and we can build useful stuff on top of it. I'm not sure if this will actually work, but it would be cool if it does so I'm starting pretty generally and we'll see how far I get. I haven't built this exact sort of thing before so I might be way off.

I'm basing the whole thing on analyzing entire objects, not analyzing changes to objects. So each part of the pipeline is handed an object and told "analyze this", not handed a change. It pretty much deletes all the old data about that thing and then writes new data. I think this is simpler to implement and understand, and it protects us from all sorts of weird issues where we end up with some kind of garbage in the DB and have to wipe the whole thing.

= Facts =

The general idea is that we extract "facts" out of objects, and then the various view interfaces just report those facts. This change has on type of fact, a "raw fact", which is directly derived from an object. These facts are concerete and relate specifically to the object they are derived from. Some examples of such facts might be:

  D123 has 9 comments.
  D123 uses macro "psyduck" 15 times.
  D123 adds 35 lines.
  D123 has 5 files.
  D123 has 1 object.
  D123 has 1 object of type "DREV".
  D123 was created at epoch timestamp 89812351235.
  D123 was accepted by @alincoln at epoch timestamp 8397981839.

The fact storage looks like this:

  <factType, objectPHID, objectA, valueX, valueY, epoch>

Currently, we supprot one optional secondary key (like a user PHID or macro PHID), two optional integer values, and an optional timestamp. We might add more later. Each fact type can use these fields if it wants. Some facts use them, others don't. For instance, this diff adds a "N:*" fact, which is just the count of total objects in the system. These facts just look like:

  <"N:*", "PHID-xxxx-yyyy", ...>

...where all other fields are ignored. But some of the more complex facts might look like:

  <"DREV:accept", "PHID-DREV-xxxx", "PHID-USER-yyyy", ..., ..., nnnn> # User 'yyyy' accepted at epoch 'nnnn'.
  <"FILE:macro", "PHID-DREV-xxxx", "PHID-MACR-yyyy", 17, ..., ...> # Object 'xxxx' uses macro 'yyyy' 17 times.

Facts have no uniqueness constraints. For @vrana's reviewer responsiveness stuff, we can insert multiple rows for each reviewer, e.g.

  <"DREV:reviewed", "PHID-DREV-xxxx", "PHID-USER-yyyy", nnnn, ..., mmmm> # User 'yyyy' reviewed revision 'xxxx' after 'nnnn' seconds at 'mmmm'.

The second value (valueY) is mostly because we need it if we sample anything (valueX = observed value, valueY = sample rate) but there might be other uses. We might need to add "objectB" at some point too -- currently we can't represent a fact like "User X used macro Y on revision Z", so it would be impossible to compute macro use rates //for a specific user// based on this schema. I think we can start here though and see how far we get.

= Aggregated Facts =

These aren't implemented yet, but the idea is that we can then take the "raw facts" and compute derived/aggregated/rollup facts based on the raw fact table. For example, the "count" fact can be aggregated to arrive at a count of all objects in the system. This stuff will live in a separate table which does have uniqueness constraints, and come in the next diff.

We might need some kind of time series facts too, not sure about that. I think most of our use cases today are covered by raw facts + aggregated facts.

Test Plan: Ran `bin/fact` commands and verified they seemed to do reasonable things.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran, majak

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3078
2012-07-27 13:34:21 -07:00
Evan Priestley
94445d41e7 Merge pull request #168 from ossareh/support_ldap_search_for_users
Support ldap search for users
2012-07-26 14:54:10 -07:00
Michael Ossareh
a9af5d611d Prevent the ability to scrape for valid usernames
- return the same error message when either bind or the username search
   fails to find a user
 - config variables should use hypen and not underscore
2012-07-26 14:32:51 -07:00
epriestley
c614af53bf Add PhutilFactsUpateIterator
Summary: This iterator processes objects that have been updated.

Test Plan:
Ran this test script:

  $cursor = null;
  $table  = new DifferentialRevision();
  while (true) {
    $iterator = new PhabricatorFactsUpdateIterator($table, $cursor);
    foreach ($iterator as $new_cursor => $update) {
      echo "{$new_cursor} => D".$update->getID()."\n";
      $cursor = $new_cursor;
    }
    echo "Zzz...\n";
    sleep(5);
  }

Verified it iterated over every object and then stopped. Made a comment on a differenial revision, verified it iterated over the object after 15 seconds.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3077
2012-07-26 13:09:36 -07:00
epriestley
fc09bcf0a3 Move qsprintf() test cases from libphutil to Phabricator
Summary: Also move the other tests up so they'll trigger when this stuff is touched.

Test Plan: liberate

Reviewers: nh, btrahan, vrana

Reviewed By: nh

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3074
2012-07-26 12:01:47 -07:00
vrana
1970ceefe3 Compute reviewer stats
Summary:
The final goal is to display reviewers response time on homepage.
This is a building block for it.

The algorithm is quite strict - it doesn't count simple comment as response because reviewers would be able to cheat with comments like "I'm overwhelmed right now and will review next week".
We are more liberate in Phabricator where reviewers response with comments without changing the status quite often but I'm not trying to improve response times in Phabricator so this is irrelevant.
Reviewers in Facebook changes status more often (to clean their queue) so I follow this approach.

There is currently no way to track reviewers silently added and removed in Edit Revision but it's not a big deal.
The algorithm doesn't track commandeered revision, there's a TODO for it.

Response times are put in two buckets: `$reviewed` and `$not_reviewed`.
`$reviewed` contains reviewers who took action, `$not_reviewed` contains reviewers who didn't respond on time.
I will probably compute average time from `$reviewed` and raise it for those `$not_reviewed` that are higher than this average.
The idea is to not favor reviewers who were only lucky for being in a group with someone fast.

Test Plan: Passed test.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3062
2012-07-24 23:37:58 -07:00
epriestley
7ffe802671 Remove queryfx() from phabricator/
Summary: Seee D3057.

Test Plan: Loaded site.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran, chad

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3058
2012-07-24 12:34:02 -07:00
epriestley
17e20bc363 Remove AphrontConnection from Phabricator
Summary: See D3055.

Test Plan: Loaded pages and such.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3056
2012-07-24 11:50:19 -07:00
epriestley
5d4a6bcf95 Break AphrontDatabaseConnection dependencies on PhabricatorEnv
Summary: We pull "retries" and a doc link from PhabricatorEnv directly. Break these dependencies so the classes can move to libphutil.

Test Plan: Browsed site, triggered a schema exception and verified I still got the useful footer text.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3053
2012-07-24 10:47:27 -07:00
epriestley
be63cb386b Remove AphrontWriteGuard from phabricator/
Summary: See D3051.

Test Plan: Loaded some pages.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1283

Differential Revision: https://secure.phabricator.com/D3052
2012-07-24 10:46:07 -07:00
Bob Trahan
bc29a3e8a2 Make inline comment preview work in Diffusion
Summary: created a PhabricatorInlineCommentPreviewController so controllers in Diffusion and Differential respectively just have to handle the URI mapping and data loading like good little controllers.

Test Plan:
left inline comments on commits, deleted inline commits, submitted inline comments -- all worked well
did the same on some diffs

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1176

Differential Revision: https://secure.phabricator.com/D3034
2012-07-23 11:01:28 -07:00
epriestley
9be12551a9 Move Task <=> Revision storage to Edges
Summary:
  - Add edges for this relationship.
  - Use edges to store this data.
  - Migrate old data.
  - Fix some warnings with generating feed stories about Aux and Edge transactions.
  - Fix a task-task edge issue with "Create Subtask".

Test Plan:
  - Migrated data, verified reivsions showed up.
  - Attached and detached tasks to revisions and vice versa.
  - Created a new revision with attached tasks.
  - Created a subtask.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3018
2012-07-20 08:59:39 -07:00
Bob Trahan
ae13d33859 Phame - introduce blogs
Summary:
blogs are collections of posts. a blog also has metadata like a name, description and "bloggers" that can edit the metadata of the blog and contribute posts.

changes include the post edit flow where bloggers can now select which blogs to publish to. also made various small tweaks throughout the UI to make things sensical and clean as the concept of blogs is introduced.

there's edges powering this stuff.  bloggers <=> blogs and posts <=> blogs in particular.

Test Plan:
made blogs, deleted blogs, tried to make blogs with no bloggers. all went well.
verified ui to publish only showed up for public posts, published posts to blogs, un-published posts to blogs, re-published posts to blogs, deleted posts and verified they disappeared from blogs.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3003
2012-07-19 09:03:10 -07:00
epriestley
9196a6bd9f Use Edges to store dependencies between tasks in Maniphest
Summary:
  - Use edges to store "X depends on Y" information in Maniphest.
  - Show both "Depends On" and "Dependent Tasks".
  - Migrate all the old edges.

Test Plan:
  - Added some relationships, migrated, verified they were preserved.
  - Added some new valid relationships, verified tasks got updated with sensible transactions and sent reasonable emails.
  - Tried to add a cycle, got an ugly but effective error.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3006
2012-07-18 20:41:42 -07:00
epriestley
a7bcc532da Add an iterator to make it easier to perform database migrations
Summary: See D3006, D3007. Make it easier to do migrations like that without holding all results in memory.

Test Plan:
Ran this code with an artificially small page size (2):

  foreach (new LiskMigrationIterator(new DifferentialRevision()) as $rev) {
    echo "Revision ".$rev->getID()."\n";
  }

Verified each revision as loaded and processed.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D3008
2012-07-18 20:01:23 -07:00
epriestley
378feb3ffb Centralize rendering of application mail bodies
Summary: This is a minor quality-of-life improvement to prevent D2968 from being as nasty as it is.

Test Plan: Ran unit tests; generated Differential, Maniphest and Diffusion emails and verified the bodies looked sensible.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T931

Differential Revision: https://secure.phabricator.com/D2986
2012-07-16 19:01:43 -07:00
Alan Huang
8065d72899 Add a differential.createinline method to Conduit
Summary: a silly thing because I was bored

Test Plan: and I said "arc call-conduit", and there were comments

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1500

Differential Revision: https://secure.phabricator.com/D2971
2012-07-16 11:49:06 -07:00
Evan Priestley
62e039b748 Merge pull request #150 from linead/ldap_dir
Ldap dir
2012-07-11 18:37:36 -07:00
epriestley
2b0b9a1573 Add a generic multistep Markup cache
Summary:
The immediate issue this addresses is T1366, adding a rendering cache to Phriction. For wiki pages with code blocks especially, rerendering them each time is expensive.

The broader issue is that out markup caches aren't very good right now. They have three major problems:

**Problem 1: the data is stored in the wrong place.** We currently store remarkup caches on objects. This means we're always loading it and passing it around even when we don't need it, can't genericize cache management code (e.g., have one simple script to drop/GC caches), need to update authoritative rows to clear caches, and can't genericize rendering code since each object is different.

To solve this, I created a dedicated cache database that I plan to move all markup caches to use.

**Problem 2: time-variant rules break when cached.** Some rules like `**bold**` are time-invariant and always produce the same output, but some rules like `{Tnnn}` and `@username` are variant and may render differently (because a task was closed or a user is on vacation). Currently, we cache the raw output, so these time-variant rules get locked at whatever values they had when they were first rendered. This is the main reason Phriction doesn't have a cache right now -- I wanted `{Tnnn}` rules to reflect open/closed tasks.

To solve this, I split markup into a "preprocessing" phase (which does all the parsing and evaluates all time-invariant rules) and a "postprocessing" phase (which evaluates time-variant rules only). The preprocessing phase is most of the expense (and, notably, includes syntax highlighting) so this is nearly as good as caching the final output. I did most of the work here in D737 / D738, but we never moved to use it in Phabricator -- we currently just do the two operations serially in all cases.

This diff splits them apart and caches the output of preprocessing only, so we benefit from caching but also get accurate time-variant rendering.

**Problem 3: cache access isn't batched/pipelined optimally.** When we're rendering a list of markup blocks, we should be able to batch datafetching better than we do. D738 helped with this (fetching is batched within a single hunk of markup) and this improves batching on cache access. We could still do better here, but this is at least a step forward.

Also fixes a bug with generating a link in the Phriction history interface ($uri gets clobbered).

I'm using PHP serialization instead of JSON serialization because Remarkup does some stuff with non-ascii characters that might not survive JSON.

Test Plan:
  - Created a Phriction document and verified that previews don't go to cache (no rows appear in the cache table).
  - Verified that published documents come out of cache.
  - Verified that caches generate/regenerate correctly, time-variant rules render properly and old documents hit the right caches.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1366

Differential Revision: https://secure.phabricator.com/D2945
2012-07-09 15:20:56 -07:00
epriestley
1089a48d4a Allow edges to be configured to prevent cycles
Summary:
Certain types of things we should be storing in edges (notably, Task X depends on Task Y) should always be acyclic. Allow `PhabricatorEdgeEditor` to enforce this, since we can't correctly enforce it outside of the editor without being vulnerable to races.

Each edge type can be marked acyclic. If an edge type is acyclic, we perform additional steps when writing new edges of that type:

  - We acquire a global lock on the edge type before performing any reads or writes. This ensures we can't produce a cycle as a result of a race where two edits add edges which independently do not produce a cycle, but do produce a cycle when combined.
  - After performing writes but before committing transactions, we load the edge graph for each acyclic type and verify that it is, in fact, acyclic. If we detect cycles, we abort the edit.
  - When we're done, we release the edge type locks.

This is a relatively high-complexity change, but gives us a simple way to flag an edge type as acyclic in a robust way.

Test Plan: Ran unit tests.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2940
2012-07-09 10:39:38 -07:00
epriestley
7cf6313be9 Add a generic object for unit tests
Summary:
A later diff adds unit tests against edges, but we need real objects to connect with edges. Add some trivial objects to the Harbormaster database to compliment the similar HarbormasterScratchTable.
On its own, this does nothing interesting.

Test Plan: Built unit tests on this in a followup.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1162

Differential Revision: https://secure.phabricator.com/D2937
2012-07-09 10:39:14 -07:00
Bob Trahan
67691c196e Phame - add an "Everyone but Me" view and make published posts truly accessible to the whole world.
Summary:
accessibility covers not only a given post but also the various "published" views.

to keep the code relative clean, this diff also splits up the post list controller logic quite a bit. this also feels like good preparation for some other work around introducing "blogs" which are collections of published posts from bloggers with some fancy features around that.

Test Plan: clicked around various parts of the Phame application as a logged in user, a logged in user with no personal posts, and without any user logged in at all. various views all seemed reasonable.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D2898
2012-07-05 15:43:14 -07:00
linead
51d2d06e37 Added ldap import controller 2012-07-04 12:10:38 +10:00
epriestley
ddf67fce58 Add an example event listener, improve documentation, and add a commit discovery event
Summary: Improve documentation around Phabricator events.

Test Plan: Generated and read documentation. Ran test script.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran

Maniphest Tasks: T1092

Differential Revision: https://secure.phabricator.com/D2917
2012-07-03 16:46:27 -07:00
epriestley
9f4cfd40bc Insert Maniphest transactions when edges are edited
Summary:
  - See D2741.
  - When EdgeEditor performs edits, emit events.
  - Listen for Maniphest edge events and save the changes as transactions.
  - Do all this in a reasonably generic way that won't take too much rewriting as we use edges more generally.

Test Plan: Attached and detached commits from tasks, saw reasonable-looking transactions spring into existence.

Reviewers: btrahan, davidreuss

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2906
2012-07-02 15:42:06 -07:00
epriestley
534c0aa326 Minor, move all storage/query/db code to src/infrastructure/storage 2012-07-02 10:49:00 -07:00
epriestley
9ba6ebb97b Minor, move all remarkup code to src/infrastructure/markup/ 2012-07-02 10:44:37 -07:00
epriestley
fc45398ba9 Support basic notification aggregation
Summary: This is both only partially complete (supports Maniphest only) and somewhat overcomplicated (includes support for applying similar algorithms to Feed), but provides runtime aggregation of notifications.

Test Plan: {F13502}

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2884
2012-07-01 11:08:59 -07:00
vrana
99df72b81c Unhighlight lines modified in rebase
Summary:
Diff of diffs display changes between new versions of file.
This is bad after rebase because there can be many unrelated changes so it is hard to spot the real change.

This diff unhighlights the lines that were added or removed in rebase.
The changes are still visible (they can be sometimes relevant) but very subtle.

Test Plan:
# Add, change and delete line. Display diff.
# Add and change some lines in parent. Rebase. Display diff. Display diff of diff.
# Change and add some lines. Display diff. Display diff to first diff. Display diff to second diff.

Reviewers: epriestley, jungejason

Reviewed By: jungejason

CC: jungejason, aran, Korvin

Differential Revision: https://secure.phabricator.com/D2761
2012-06-29 17:21:23 -07:00
epriestley
2bb8150506 Add specific close status queries to maniphest.find (and rename it to maniphest.query)
Summary:
  - Allow clients to query for specific closed statuses (invalid, resolved, wontfix, etc), not just "closed" tasks.
  - Rename this method to maniphest.query and deprecate maniphest.find as an alias to maniphest.query, for API consistency.

Test Plan: Ran queries for all tasks, "wontfix" tasks, closed tasks.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2887
2012-06-29 09:17:19 -07:00
Andrew Gallagher
3223defe96 Add support for postponed linters
Summary:
This adds:
1) A new "arc:lint-postponed" diff property which stores a list of
lint names that are postponed and a finishpostponedlinters conduit
method which removes linters from this list.  Postponed linters are
shown in the lint details.
2) A updatelintmessages conduit message, which adds additional lint
messages to the "arc:lint" diff property.

In combination, this provides very basic support for running
asynchronous static analysis tools.  When the diff is being created,
a list of asynchronous static analysis runs can be added to the
diff's postponed linters list.  As these postponed linters finish
up, then can report new lint messages back to the diff then mark
themselves as complete.

The client is currently responsible for filtering the lint messages
by things like affected lines and files.

Test Plan:
Used conduit call API to add lint messages and remove postponed
linters from a test diff.

Reviewers: epriestley, vrana, nh, jungejason

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1332

Differential Revision: https://secure.phabricator.com/D2792
2012-06-28 18:09:38 -07:00