1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-22 13:30:55 +01:00
Commit graph

656 commits

Author SHA1 Message Date
jungejason
c80d1480d5 Add Basic Auditing Functionalities
Summary:
add basic auditing functionalities. For the related commits for a
package, we detect the following conditions which might be suspicious to the
owners of the package:

* no revision specified
* revision not found
* author not match
* reviewedby not match
* owners not involved
* commit author not recognized

The owners of the package can change the status of the audit entries by
accepting it or specify concern.

The owner can turn on/off the auditing for a package.

Test Plan:
*  verified that non-owner cannot see the details of the audit and cannot modify
it
*  verified that all the audit reasons can be detected
*  tested dropdown filtering and package search
*  verified really normal change not detected
*  verified accept/concern a commit
*  tested enable/disable a package for auditing
*  verified one audit applies to all <commit, packages> to the packages the
auditor owns
*  verified that re-parsing a commit won't have effect if there exists a
 relationship for <commit, package> already

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, benmathews, btrahan, mpodobnik, prithvi, TomL, epriestley

Differential Revision: 1242
2011-12-20 13:36:53 -08:00
epriestley
afc2f8526b Allow Phriction documents to be deleted
Summary:
  - Add a "delete" operation. Delete is just a special edit which removes the
page from indexes and shows a notice that the document has been deleted.
  - When a user deletes all the content on a page, treat it as a delete.
  - When a conduit call deletes all the content on a page, treat it as a delete.
  - Add page status to Conduit.
  - Add change type field to history.
  - Added a couple of constants to support a future 'move' change, which would
move content from one document to another.

Test Plan:
  - Verified deleted pages vanish from the document index (and restoring them
puts them back).
  - Verified deleted pages show "This page has been deleted...".
  - Created, edited and deleted a document via Conduit.
  - Deleted pages via "delete" button.
  - Deleted pages via editing content to nothing.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: skrul, aran, btrahan, epriestley

Maniphest Tasks: T680

Differential Revision: 1230
2011-12-17 11:45:25 -08:00
epriestley
93d5d29541 Remove "Former" project members
Summary:
This is a needlessly confusing/complex feature that I originally wrote sort of
speculativley. I think we can better serve what little need may exist here with
project feeds.

I'm probably going to get rid of or deemphasize "role" too and just add "Join
Project" and "Leave Project" buttons.

Test Plan: Viewed project list, project profile. Edited project profile and
affiliation.

Reviewers: btrahan, jungejason, zeeg

Reviewed By: btrahan

CC: aran, btrahan

Maniphest Tasks: T681

Differential Revision: 1228
2011-12-16 17:46:02 -08:00
jungejason
c13b7da290 Add Related Commits for Owners
Summary:
For each commit, find the affected packages, and provide a way to
search by package.

Test Plan:
create commits that touch and don't touch two packages, and verify
that they display correctly in all the UI pages.

Reviewers: epriestley, blair, nh, tuomaspelkonen

Reviewed By: epriestley

CC: benmathews, aran, epriestley, btrahan, jungejason, mpodobnik, prithvi

Maniphest Tasks: T83

Differential Revision: 1208
2011-12-14 22:48:57 -08:00
Bob Trahan
d739d8ccd5 Paste - upgrade scheme to support queries by authorPHID
Test Plan: ran upgrade_schema.php on my local install.   re-examined SQL query
"explain select * from phabricator_pastebin.pastebin_paste where authorPHID =
"PHID-USER-xee4ju2teq7mflitwfcs" ORDER BY id DESC;" on my local install and
noted fewer rows scanned, index used, etc

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1218
2011-12-14 19:48:47 -08:00
epriestley
4fd81150be Remove "Updated" view from Differential
Summary:
This landed during my review drama embargo and is a generally good idea but had
some implementation issues.

@elynde reports it has been broken for some time, although it still works on
secure.phabricator.com so I'm guessing it's just taking a zillion years to run
at Facebook. It's up to more than a second for me on secure.phabricator.com:

https://secure.phabricator.com/file/view/PHID-FILE-v4ql4c66u3xnkarmrpm4/

The basic problem is that some of the data architecture around this
implementation is hard to scale. I want to pursue a similar feature eventually,
but drive it off notifications that we'll ship through real-time infrastructure
too.

I'm also trying to get rid of DifferentialRevisionListData and this simplifies
that somewhat.

Test Plan:
  - Grepped for table name, table constant, query constant, and class name; no
hits.
  - Applied SQL patch.
  - Verified that Differential no longer shows "Updated".

Reviewers: elynde, btrahan, jungejason

Reviewed By: elynde

CC: aran, elynde

Differential Revision: 1178
2011-12-07 06:55:03 -08:00
epriestley
720af267a7 Add a missing key to the ManiphestTransaction table
Summary:
Caught a bad (8ms) query that didn't have a good key available. Add an
appropriate key.

The query in question is:

   SELECT * FROM `maniphest_transaction` WHERE taskID = 262 ORDER BY id ASC;

...which is used in the task detail view.

Test Plan: Ran EXPLAIN on this query before/after upgrading, it now uses the
newly available key.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran

Differential Revision: 1037
2011-10-23 14:43:03 -07:00
epriestley
0669abc5f0 Use a proper entropy source to generate file keys
Summary:
See T549. Under configurations where files are served from an alternate domain
which does not have cookie credentials, we use random keys to prevent browsing,
similar to how Facebook relies on pseudorandom information in image URIs (we
could some day go farther than this and generate file sessions on the alternate
domain or something, I guess).

Currently, we generate these random keys in a roundabout manner. Instead, use a
real entropy source and store the key on the object. This reduces the number of
sha1() calls in the codebase as per T547.

Test Plan: Ran upgrade scripts, verified database was populated correctly.
Configured alternate file domain, uploaded file, verified secret generated and
worked properly. Changed secret, was given 404.

Reviewers: jungejason, benmathews, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley

Differential Revision: 1036
2011-10-23 14:42:23 -07:00
epriestley
ddce177d81 Add a name token table so on-demand typeaheads can match last names
Summary: See T585. We currently don't match middle/last/nth names in on-demand
tokenizers. Build a table so we can match them.

Test Plan:
Ran upgrade script, verified table looks sensible. Searched for "priestley" in a
tokenizer, got a bunch of test account hits.

  mysql> select * from user_nametoken;
  +-------------------+--------+
  | token             | userID |
  +-------------------+--------+
  | evan              |      1 |
  | priestley         |      1 |
  | epriestley        |      1 |
  | epriestley2       |      2 |
  | ducks             |      4 |
  | epriestley3       |      4 |
  | asdf              |      6 |
  | epriestley99      |      6 |
  ...

Reviewers: bh, nh, jungejason, tuomaspelkonen, aran

Reviewed By: aran

CC: aran

Differential Revision: 1034
2011-10-23 14:25:26 -07:00
epriestley
9a4bb3901e Allow bugs@ addresses to blanket-accept tasks
Summary: Allow configuration of a default author for bugs@ emails which don't
correspond to a known system user.

Test Plan: Configured a default author, sent some mails from nonsense addresses,
tasks were created.

Reviewers: davidreuss, jungejason, nh, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley, ide

Differential Revision: 1013
2011-10-20 14:26:19 -07:00
Marek Sapota
cb0700a99c Allow databases to be created outside upgrade_schema scrpt.
Summary:
Current scripts make it hard to administer Phabricator instance while not having
direct (priviledged) access to the database.  This change allows scenario where
DB administrator creates the databases for you before you run update_schema
script.

Test Plan:
Create the databases before running the update_schema script - it shouldn't
complain that the databases already exist.

Reviewers: aran, epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1014
2011-10-17 13:59:56 -07:00
epriestley
254f606e89 Tie all the pieces for symbol cross-references together
Summary:
This makes symbol cross-references work in Differential. You need to do a little
legwork but I'll document that once the change has baked for a little while.

Basically:

  - Projects are annotated with indexed languages, and "shared library" projects
(for example, symbols in Phabricator should be searched for in Arcanist and
libphutil).
  - When we render a changeset, we check if its language is an indexed one. If
it is, we invoke the decorator Javascript.
  - The Javascript takes you to a lookup page, which either gives you a list of
matching symbols (if several match) or redirects you instantly to the
definition.

Test Plan: Clicked class and function symbols in a diff, got jumped into
sensible sorts of places in Diffusion.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 980
2011-10-09 17:58:17 -07:00
epriestley
016b060aea Add a relation table for Revisions to local commit hashes
Summary:
This allows us to performantly query for diffs related to a given local hash.
Immediate applications are:

  - Commit detection in Mercurial and Git-Immutable workflows.
  - Some async unit test stuff @mgummelt was doing.

Test Plan:
Diffed locally under SVN/Git/hg, checked the table, got sensible output.

  mysql> select * from differential_revisionhash;
  +------------+------+------------------------------------------+
  | revisionID | type | hash                                     |
  +------------+------+------------------------------------------+
  |         40 | gtcm | 8c6fb2f95598a50f7aac64a5f4cc6c12b5db42f5 |
  |         40 | gttr | 54710e361a465f4ff39565a93b2a221b6e7dd07c |
  |         41 | hgcm | c29cb69aec14                             |
  |         41 | hgcm | e7309be4eabb                             |
  |         41 | hgcm | 4e885caeff60                             |
  |         41 | hgcm | 213ee1cd30ea                             |
  |         41 | hgcm | b4050fb3490f                             |
  |         41 | hgcm | 72a76bd7ffa2                             |
  |         41 | hgcm | 06c2687e63fb                             |
  |         41 | hgcm | 2b464bde6b48                             |
  +------------+------+------------------------------------------+
  10 rows in set (0.00 sec)

NOTE: Mercurial hashes are short-form but I'll shoot out a separate Arcanist
diff to fix this.

Reviewers: Makinde, fratrik, mgummelt, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 961
2011-09-26 15:02:37 -07:00
epriestley
43a3f4d234 Build an "affected path" index when attaching diffs to revisions
Summary: See T262. This creates the index on the Differential side which we need in order to execute this query efficiently on the Diffusion side.

Also renames "DiffusionGitPathIDQuery" to "DiffusionPathIDQuery", this query object has nothing to do with git.

Test Plan: Attached top-level and sub-level diffs to revisions and verified they populated the table with sensible data.

Reviewers: bmaurer, aravindn, fmoo, jungejason, nh, tuomaspelkonen, aran

CC:

Differential Revision: 931
2011-09-15 07:45:14 -07:00
epriestley
cd05c960ff Add storage for repository symbol tracking
Summary: See T315 for an extensive description of this feature. Adds the
descibed storage table.

Test Plan: Used phpsh to read/write symbol objects.

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: tuomaspelkonen

CC: aran, epriestley, tuomaspelkonen

Differential Revision: 897
2011-09-13 08:49:44 -07:00
epriestley
e875c81f6d Remove blameRevision and revertPlan from the DifferentialRevision schema
Summary:
These fields use auxiliary storage now. Migrate the data and get rid of the
columns in the main table.

  - This might take a little while to run, although there are <500k rows so
probably not too long.
  - Maybe grab a backup of the table first, if I screwed something up this will
delete the data in these fields.

Test Plan:
  - Ran migration locally.
  - Browsed Differential.
  - Grepped for "revertPlan" and "blameRevision".

Reviewers: jungejason, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason, epriestley

Differential Revision: 832
2011-09-04 16:19:12 -07:00
epriestley
69445222f7 Track content sources (email, web, conduit, mobile) for replies
Summary:
When an object is updated, record the content source for the update. This mostly
isn't terribly useful but one concrete thing I want to do with it is let admins
audit via-email replies more easily since there are a bunch of options which let
you do hyjinx if you intentionally configure them insecurely. I think having a
little more auditability around this feature is generally good. At some point
I'm going to turn this into a link admins can click to see details.

It also allows us to see how frequently different mechanisms are used, and lets
you see if someone is at their desk or on a mobile or whatever, at least
indirectly.

The "tablet" and "mobile" sources are currently unused but I figured I'd throw
them in anyway. SMS support should definitely happen at some point.

Not 100% sure about the design for this, I might change it to plain text at some
point.

Test Plan: Updated objects and saw update sources rendered.

Reviewers: jungejason, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, epriestley, jungejason

Differential Revision: 844
2011-08-30 11:08:27 -07:00
epriestley
dd74903cae Add basic auxiliary field storage for Differential
Summary:
Precursor to building this out to solve T343. This is similar to the Maniphest
fields we landed recently, although I think they're dissimilar enough that it
isn't worth going crazy trying to make them share code, at least for now.

This doesn't really do anything yet, just adds a storage object and a couple of
selector/field indirection classes.

Test Plan: Ran SQL upgrade script, created an aux field.
Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, jungejason
Differential Revision: 798
2011-08-14 10:04:21 -07:00
epriestley
879431fb50 Improve GC performance for Herald Transcripts
Summary: This has to table scan a ginormous table right now, give it a fighting
chance with a more usable key.
Test Plan:
  - Launched GC daemon, no errors.
  - Used test console to create a new transcript.
  - Viewed some old transcripts.
  - Ran EXPLAIN on the SELECT and verified it was utilizing the garbageCollected
key.

Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, epriestley, jungejason
Differential Revision: 735
2011-07-28 18:50:54 -07:00
hunterbridges
aeae33b7d6 Key Value Store for ManiphestTask
Test Plan: Look at a task detail. Some dummy attributes are automatically added in ManiphestTaskDetailController.
Reviewed By: epriestley

Differential Revision: 730
2011-07-25 19:11:55 -07:00
epriestley
6e08a9215d Move "Preferences" to "Settings"
Summary:
It makes more sense to just make this a settings panel rather than a standalone
app, particularly since setting panels are relatively well separated now.

Also default-disabled the SSH Keys interface since it won't currently be useful
for most installs.

Test Plan: Edited preferences.
Reviewed By: jungejason
Reviewers: tuomaspelkonen, jungejason, aran
CC: aran, jungejason
Differential Revision: 716
2011-07-24 12:25:43 -07:00
Hua Wang
2a894438ae Add a 'description' field to Phriction
Summary:
Add a new column to PhrictionContent called 'comment' or 'description' or
something
        Add an optional field to the Phriction document editing interface that
allows you to add a comment

Test Plan:
Run the sql patch to modify phriction_content table
           Create a new wiki page in Phriction
           Type in words in description field and save the page
           Visit this page and click "Edit Page" button
           The content in the desciption field is saved

Reviewed By: epriestley
Reviewers: epriestley, hsb, codeblock
Commenters: codeblock
CC: aran, codeblock, hwang, epriestley
Differential Revision: 709
2011-07-23 21:11:42 -07:00
epriestley
8df62d5352 Allow users to associate SSH Public Keys with their accounts
Summary:
With the sshd-vcs thing I hacked together, this will enable Phabricator to host
repositories without requiring users to have SSH accounts.

I also fixed "subporjects" and added an explicit ENGINE to it.

Test Plan: Created, edited and deleted public keys. Attempted to add the same
public key twice. Attempted to add invalid and unnamed public keys.
Reviewed By: aran
Reviewers: jungejason, tuomaspelkonen, aran, cadamo, codeblock
CC: aran, epriestley
Differential Revision: 711
2011-07-23 09:15:20 -07:00
Cristian Adamo
8cca3079fc Add subprojects to a main project.
Summary:
Quite basic subproject attachment.

Test Plan:
Go to some proj
ect and add another as a subproject.
Play around the ui.

Reviewers:	epr
iestley
CC:
2011-07-19 15:50:15 -03:00
Ricky Elrod
95d6e0a3c4 Store parents of forked pastes, and list child pastes if there are any.
Summary:
Added a 'parent' field which stores a PHID of another paste. If it is not NULL show a list of children pastes on view.
Also did some misc. refactoring to clean up the code a bit, specifically in the Create controller.

Test Plan:
- Checked old pastes, they were not affected.
- Added a paste, successfully.
- Forked it, successfully.
- Went to the original paste, saw the child paste listed.
- Forked it again, saw the new one added to the list.

Reviewers:
epriestley

CC:

Differential Revision: 672
2011-07-15 18:42:08 -04:00
epriestley
4c44c44964 Add Phriction to the main nav menu
Summary:
A few minor things:

  - Add Phriction to the homepage.
  - Link to remarkup reference on the edit screen.
  - Add a help tab with a super basic help document.

Test Plan: Cursory inspection.
Reviewed By: codeblock
Reviewers: hsb, codeblock, jungejason, tuomaspelkonen, aran
CC: aran, epriestley, codeblock
Differential Revision: 655
2011-07-12 09:26:51 -07:00
epriestley
5704b2bc70 Basic edit/create workflow for Phriction
Summary:
This is another chunk of D636, I just simplified it a bit and added slugs.

When you go to a page like /w/pokemon/, it allows you to create or edit the
page.

Title vs slug stuff is a little funky but I think mostly-reasonable.

Test Plan: Created and edited /w/, /w/pokemon/, etc.
Reviewed By: hsb
Reviewers: hsb, codeblock, jungejason, aran, tuomaspelkonen
CC: aran, hsb
Differential Revision: 643
2011-07-11 14:47:33 -07:00
epriestley
f95913ec47 Phriction basics
Summary:
Basically a copy/paste of parts of D636, but with two changes:

  - Fully separate the index table ("document") from the content table
("content"). I think this will be a cleaner solution in the long run.
  - Build slugs into the document structure.

This doesn't do anything useful, it just normalizes slugs and lays some
groundwork.

Test Plan:
  - Visited various /w/ pages and saw them normalize correctly.
  - Verified the DAO works by inserting dummy rows.

Reviewed By: codeblock
Reviewers: hsb, codeblock, jungejason, aran, tuomaspelkonen
CC: aran, codeblock, epriestley
Differential Revision: 638
2011-07-11 11:42:50 -07:00
epriestley
467308dd12 Script to selectively convert MyISAM tables to InnoDB
Summary:
A bunch of the .sql patch files don't explicitly specify the table engine, but
we should always use InnoDB with the exception of one table which needs MyISAM
for FULLTEXT.

MySQL doesn't no-op an ALTER TABLE statment that changes the engine back to
itself and converting large tables can be time consuming, so convert only the
required tables.

Test Plan: Ran on secure.phabricator.com and my local box, it fixed all the
issues in about 3 seconds on secure.phabricator.com and <<1 second on my local.
Reviewed By: codeblock
Reviewers: codeblock, tuomaspelkonen, jungejason, aran
CC: aran, epriestley, codeblock
Differential Revision: 641
2011-07-11 11:42:28 -07:00
epriestley
233a2055f1 Add missing keys to some tables
Summary: I wrote a little query plan plugin for "Services", it's not ready yet
but it found a few cases where we're doing table scans.
Test Plan:
https://secure.phabricator.com/file/view/PHID-FILE-374fd7e846a0ce27d5fb/
https://secure.phabricator.com/file/view/PHID-FILE-2f680de9b8c3c3b7089e/
https://secure.phabricator.com/file/view/PHID-FILE-742f645fcf011252ff25/

Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, jungejason
Differential Revision: 627
2011-07-09 10:55:15 -07:00
epriestley
51c2726a34 Add Differential parse cache to the GC daemon
Summary:
Add the differential parse cache to the GC. This is the largest object in the
system by a wide margin, I think.

This table is potentially gigantic which is why the script truncates it before
doing a schema change.

Test Plan: Ran the GC daemon, it cleaned up some parse caches.
Reviewed By: jungejason
Reviewers: tuomaspelkonen, jungejason, aran
Commenters: tuomaspelkonen
CC: aran, jungejason, tuomaspelkonen, epriestley
Differential Revision: 620
2011-07-08 17:31:25 -07:00
epriestley
57097c2874 Port the "Slowvote" application
Summary: Port slowvote. This has some style/layout roughness but gets us most of
the way there. I'll followup to fix some of the markup issues.
Test Plan: Created and voted in several different kinds of poll.
Reviewed By: codeblock
Reviewers: codeblock, tomo, jungejason, aran, tuomaspelkonen
Commenters: aran, jungejason
CC: aran, codeblock, jungejason, epriestley
Differential Revision: 613
2011-07-08 12:38:53 -07:00
Ricky Elrod
420235f9c4 Drag-drop file upload.
Summary:
- have files be uploaded by drag+drop instead of browse.
- Files are named by their uploaded filename, the user isn't given a chance to enter a file name. Is this bad?
- Store author PHID now with files
- Allow an ?author=<username> to limit the /files/ list by author.
- If one file is uploaded, the user is taken to its info page.
- If several are uploaded, they are taken to a list of their files.

Test Plan:
- Quickly tested everything and it still worked, I'd recommend some people try this out before it gets committed though. It's a rather huge revision.

Reviewers:
epriestley, Ttech

CC:

Differential Revision: 612
2011-07-08 15:20:57 -04:00
tuomaspelkonen
49310391e0 Added subscriber view to Maniphest.
Summary:
People want to see all the tasks they have subscribed to in one
view. A new table was added for this to make queries faster.

Test Plan:
Tested that the view was initially empty. After running the
reindex_maniphest.php script, I saw the correct tasks there. Added
myself as a subscriber to one task and made sure the view was
updated. Removed myself as a subscriber from one task and made sure
the view was updated again.

Reviewed By: epriestley
Reviewers: epriestley, jungejason, codeblock
CC: aran, rm, epriestley
Differential Revision: 603
2011-07-07 14:08:52 -07:00
epriestley
ece9d792b2 Build basic infrastructure for an activity feed
Summary: This defines an extremely basic version of an activity feed, like
Facebook's news feed. It doesn't do much of interest yet.
Test Plan: Published some feed stories:
https://secure.phabricator.com/file/view/PHID-FILE-5061aa72105bbdc05b21/
Reviewed By: tuomaspelkonen
Reviewers: jungejason, tuomaspelkonen, aran
Commenters: codeblock, jungejason
CC: aran, epriestley, codeblock, tuomaspelkonen, jungejason
Differential Revision: 593
2011-07-06 16:19:29 -07:00
Ricky Elrod
9454060c29 Add a syntax highlight dropdown, if pygments is enabled.
Summary:
- Add a default list of supported languages to default.conf.php
  and make the initial/default value customizable.
- Store a '' in the database to infer the language from the filename/title.

Test Plan:
Tested in my sandbox with pygments enabled and disabled and various
combinations of filename/extension/dropdown selection.

Reviewers:
epriestley

CC:

Differential Revision: 587
2011-07-04 12:23:43 -04:00
epriestley
de0c89261e Allow Maniphest tasks to be filtered by Project
Summary:
Major things taking place here:

  - A new table for storing <task, project> relationships.
  - Moved all task query logic into a dedicated class.
  - Added a "projects" filter to the UI.

I was originally going to try to drive this off the main search index but the
perf benefits of a custom schema make an overwhelming argument in favor of doing
it this way.

Test Plan:
Filtered tasks by author and owner and zero, one, and more than one project.
Exercised all the group/sort options. Ran the index script over my 100k task
corpus. Edited task-project membership and verified the index updated.

Reviewed By: cadamo
Reviewers: gc3, jungejason, cadamo, tuomaspelkonen, aran
CC: aran, cadamo, epriestley
Differential Revision: 556
2011-06-29 21:56:47 -07:00
epriestley
4469ef8f30 Allow Maniphest to scale to a massive size
Summary:
Maniphest is missing some keys and some query strategy which will make it
cumbersome to manage more than a few tens of thousands of tasks.

Test Plan:
Handily manipulated 100k-scale task groups. Maniphest takes about 250ms to
select and render pages of 1,000 tasks and has no problem paging and filtering
them, etc. We should be good to scale to multiple millions of tasks with these
changes.

Reviewed By: gc3
Reviewers: fratrik, jungejason, aran, tuomaspelkonen, gc3
Commenters: jungejason
CC: anjali, aran, epriestley, gc3, jungejason
Differential Revision: 534
2011-06-28 06:41:05 -07:00
epriestley
e0e6ec9117 Allow affiliations to carry project ownership information; transform profile
images correctly

Summary:
This is sort of doing two things at once:

  - Add an "isOwner" flag to Project Affiliation to lay the groundwork for T237.
  - Rename the "QuickCreate" workflow to "Create" and funnel all creation
through it.
  - Reorganize the image transformation stuff and use it to correctly
crop/resize uploaded images.

Test Plan:
Created and edited projects and affailiations. Uploaded project, user, and
profile photos. Verified existing thumbnailing in Maniphest still works
properly.

Reviewed By: cadamo
Reviewers: cadamo, aran, jungejason, tuomaspelkonen
CC: aran, epriestley, cadamo
Differential Revision: 529
2011-06-28 06:40:41 -07:00
epriestley
46a7676a6a Properly scope some SQL. 2011-06-21 14:46:59 -07:00
epriestley
431c57688e Improve performance of project list view
Summary:
D477 added functionality to the project list view but had a couple of
performance issues that I missed in review, because it took the query count for
the page from around 3 to as many as 300, including up to 100 heavyweight search
index queries.

This fixes the two simple N+1 query problems. This general pattern of data
access often occurs:

  COUNTEREXAMPLE
  $cats = load_cats();
  foreach ($cats as $cat) {
    $cats_hats = load_hats_for_cat($cat);
    // ...
  }

But this issues "N+1" queries, i.e. if you load 100 cats you issue 101 queries.
It is faster to group the queries instead:

  $cats = load_cats();
  $hats = load_all_hats_for_these_cats($cats);
  foreach ($cats as $cat) {
    $cats_hats = $hats[$cat->getID()];
  }

MySQL can execute one query which returns all the results much faster than 100
queries which return one result, especially if the database is not local (i.e.,
over the network).

However, this doesn't save a ton of time. The bigger issue is that I didn't have
the right keys on the relationship tables in the search engine. This adds them,
and reduces the search engine lookup cost from 25-80ms (for
secure.phabricator.com) down to 1-3ms.

I still probably want to get this out of the loop at some point but it's okay
for now and the page loads in a few ms rather than taking more than a second.

Test Plan:
Used "services" tab, "xhprof" and "EXPLAIN" to analyze page performance. I
measured these changes:

  - Query count: 1 + (3 * N projects) -> 3 + (N projects) (e.g., 301 -> 103)
  - Total time spent querying, ignoring search indexes: 40ms (local.aprhont.com)
-> 20ms (local.aphront.com)
  - Cost for search index query: 25-80ms (secure.phabricator.com) -> 1-3ms

Reviewed By: cadamo
Reviewers: cadamo, aran, jungejason, tuomaspelkonen
CC: aran, cadamo, epriestley
Differential Revision: 485
2011-06-21 14:43:15 -07:00
epriestley
4acf457431 Merge remote branch 'cadamo/devel'
Conflicts:
	src/__celerity_resource_map__.php
2011-06-20 16:06:03 -07:00
epriestley
25bbe741f4 Provide a default non-NULL timezone in the PhabricatorUser class
Summary:
Without this, user creation throws an exception when trying to insert NULL into
a non-NULL field.

Test Plan:
Created a new user.

Reviewed By: fratrik
Reviewers: fratrik, toulouse, aran, jungejason, tuomaspelkonen
CC: aran, fratrik
Differential Revision: 480
2011-06-20 13:13:51 -07:00
Cristian Adamo
7851b6573f Project list and profile view modifications
Summary: Added some change on the project's list view, to show information about
active tasks, population, etc. Also modified the "profile view", and added a class "PhabricatorProfileView" to render the profile, both on projects and users.

Test Plan: play around the project directory :)

Reviewers: epriestley ericfrenkiel

CC:

Differential Revision: 477
2011-06-20 16:13:44 -03:00
epriestley
5915d1dbde Fix syntax clowning in patch 046. 2011-06-20 05:59:42 -07:00
epriestley
513178c7ec Deconflict SQL patches. 2011-06-20 05:58:32 -07:00
epriestley
1cd1e1ed18 "arc install-certificate", server-side components
Summary:
Provides a new workflow for making it non-horrible to install certificates.
Basically you run "arc install-certificate" and then copy/paste a short token
off a webpage and it does the ~/.arcrc edits for you.

Test Plan:
Installed certificates, used bad tokens, hit rate limiting.

Reviewed By: aran
Reviewers: aran, jungejason, tuomaspelkonen
CC: aran
Differential Revision: 460
2011-06-20 05:18:02 -07:00
Andrew Toulouse
9b522982fa Add timezone support
Summary:
Allows user-configurable timezones. Adds a preference panel, and migrates to the
new date rendering in easily-modified areas of the code. ***In progress***.

Test Plan:
Check database to make sure the field is being changed when the settings are
changed; check affected views to see how they render times.

Reviewed By: epriestley
Reviewers: epriestley
CC: aran, epriestley, toulouse
Differential Revision: 475
2011-06-18 13:07:43 -07:00
epriestley
aa86cf0ed8 Countdown tweaks
Summary:
A few tweaks to hsb's Countdown implementation:

  - Allow the page to be rendered "chromeless", suitable for display on one of
the dozens of monitors everyone has laying around.
  - Show title of countdown in deletion dialog.
  - When creating a new countdown default to time(), not Dec 31, 1969.
  - Add extra "/" after editing to avoid needless redirect.
  - Tweak some page titles.
  - Show countdown author in list view.
  - Highlight tab in list view.
  - Tweak menu copy.
  - Link countdown title in list view, separate buttons into different columns
so they pick up padding.

Test Plan:
Created, edited and deleted a timer. Viewed a timer and toggled chrome mode.
Viewed timer list.

Reviewed By: hsb
Reviewers: hsb, aran, jungejason, tuomaspelkonen
CC: aran, hsb, epriestley
Differential Revision: 454
2011-06-13 18:53:43 -07:00
Hafsteinn Baldvinsson
a11b5e8bbc Countdown
Summary:
Addon that allows you to create a live countdown page to
some event.

Here is the ticket that this code is based on
https://secure.phabricator.com/T36

Test Plan:
Tested by manually setting dates in the timer.js file and
checking if they made sense.
I'm not sure if it works across different timezones though.

Reviewers: epriestley

CC:

Differential Revision: 436
2011-06-13 02:23:46 +00:00
Ricky Elrod
b9c9f90164 Pastebin
Summary:
This is Paste. It needs some work, but epriestley recommended that I just commit something that works, and expand on that later.

Specifically, it lacks the ability to view a raw paste right now, and to turn off line numbers, making it hard to copy/paste from for now. It works for showing other people code, however.

Test Plan:
Pasted stuff, and was able to view it, and see it in the list on /paste/. Put a file extension in the title, and saw that syntax highlighting worked as expected.

Reviewers:
epriestley

CC:

Differential Revision: 424
2011-06-10 13:41:18 -04:00
epriestley
2e8990e9e0 Store metadata with Differential and Maniphest comments, and store added
reviewers as metadata

Summary:
The "Add reviewer" implementation is super lazy right now since I didn't want to
do a schema change. Man up and add a column. I also plan to store "via"
information here (e.g., via email or via mobile).

NOTE: This schema change may take a while since the comment table is pretty big
in Facebook's install.

This needs a little CSS work but I think it's reasonable for now.

Test Plan:
Made comments on revisions and tasks. Added reviewers to a revision, got linked
names instead of a blob of text.

Reviewed By: aran
Reviewers: tomo, jungejason, tuomaspelkonen, aran
Commenters: tomo
CC: aran, tomo, epriestley
Differential Revision: 394
2011-06-09 10:43:25 -07:00
gc3
22c1b38655 herald: add the ability to execute a rule the first time only
Summary:
- added a new config class for representing the kind of repetition a rule has
(once, every time, first time only)

- added an email action to herald rules for differential to allow someone to get
an email but only the first one

- changed the herald rule ui to allow a user to pick the amount of repetition

Test Plan:
created a test rule and ran it over and over

Reviewed By: epriestley
Reviewers: epriestley, tuomaspelkonen
CC: aran, epriestley, gc3
Revert Plan:
Tags:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Differential Revision: 357
2011-06-09 10:35:37 -07:00
epriestley
8af5bb117d Basic image thumbnailing
Summary:
This is still very rough but provides basic support for generating image
thumbnails. I need to separate stuff out a bit but I'm going to integrate into
Maniphest before I hit the profile stuff so this seems like a reasonable
starting point.

Test Plan:
Generated some image thumbnails in various sizes.

Reviewed By: aran
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran
Differential Revision: 333
2011-05-27 09:33:33 -07:00
epriestley
deb80b7652 Provide an activity log for login and administrative actions
Summary: This isn't complete, but I figured I'd ship it for review while it's still smallish.

Provide an activity log for high-level system actions (logins, admin actions). This basically allows two things to happen:

  - The log itself is useful if there are shenanigans.
  - Password login can check it and start CAPTCHA'ing users after a few failed attempts.

I'm going to change how the admin stuff works a little bit too, since right now you can make someone an agent, grab their certificate, revert them back to a normal user, and then act on their behalf over Conduit. This is a little silly, I'm going to move "agent" to the create workflow instead. I'll also add a confirm/email step to the administrative password reset flow.

Test Plan: Took various administrative and non-administrative actions, they appeared in the logs. Filtered the logs in a bunch of different ways.

Reviewers: jungejason, tuomaspelkonen, aran

CC:

Differential Revision: 302
2011-05-20 19:08:26 -07:00
epriestley
f9f8ef0e6e Admin and disabled flags for users
Summary:
Provide an "isAdmin" flag for users, to designate administrative users.

Restore the account editing interface and allow it to set role flags and reset
passwords.

Provide an "isDisabled" flag for users and shut down all system access for them.

Test Plan:
Created "admin" and "disabled" users. Did administrative things with the admin
user. Tried to do stuff with the disabled user and was rebuffed. Tried to access
administrative interfaces with a normal non-admin user and was denied.

Reviewed By: aran
Reviewers: tuomaspelkonen, jungejason, aran
CC: ccheever, aran
Differential Revision: 278
2011-05-12 11:17:50 -07:00
epriestley
f7e2b03077 Add a "setup" mode which guides new users through application configuration
Summary:
Alters the installation instructions to guide installers into a "setup" mode
which does config file sanity checking.

Test Plan:
Put myself in setup mode, simulated all the failures it detects, took myself out
of setup mode, Phabricator works OK.

Reviewed By: tuomaspelkonen
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, tuomaspelkonen, epriestley
Differential Revision: 230
2011-05-10 15:12:30 -07:00
epriestley
25dee6ecd2 Support email replies in Phabricator
Summary:
Provides support for per-user x per-object unique reply-to email addresses, plus
SMTP integration.

This does not actually make Phabricator use these in outbound email.

Test Plan:
Used test console to validate in-Phabricator routing and handling.

Piped emails into the "mail_handler.php" script to validate mail parsing.

Configured sendmail and sent mail to Phabricator.

Technically I haven't conducted all parts of this test on the same machine since
I lost the will to configure more SMTP servers after configuring phabricator.com

Reviewed By: jungejason
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, epriestley, jungejason
Differential Revision: 226
2011-05-05 14:58:57 -07:00
epriestley
03ebbccbc9 Restore image proxying to Remarkup
Summary:
Previously, Remarkup allowed you to paste in an image URI and get an inline
image. However, it did this by hotlinking the image which isn't so hot in an
open source product.

Restore this feature, but use image proxying instead. The existing image macro
code does most of the work.

There is a mild security risk depending on the network setup so I've left this
default-disabled and made a note about it. It should be safe to enable for
Facebook.

Test Plan:
Pasted in image and non-image links, got reasonable behavior. Verified proxying
appears to work. Verified that file:// shenanigans produce 400.

Reviewed By: tuomaspelkonen
Reviewers: aran, jungejason, tuomaspelkonen
Commenters: cpiro
CC: aran, cpiro, tuomaspelkonen
Differential Revision: 214
2011-05-03 18:49:06 -07:00
epriestley
8370f93048 Make X-Herald-Rules header sticky
Summary:
This isn't terribly elegant but it solves the problem without loss of
generality. We can pursue a more finessed solution later if it seems prudent.

Test Plan:
Created a revision matched by a blanket herald rule, and then commented on it.
Comment email had X-Herald-Rules header in it.

Reviewed By: aran
Reviewers: aran, tuomaspelkonen, jungejason
CC: aran
Differential Revision: 218
2011-05-03 06:06:57 -07:00
epriestley
7ca4835438 Simplify database initialization, fix default data
Summary:
Some users have had problems with the database initialization process, simplify
it by creating a new "initialize.sql" dump at v34.

I also populated this dump with the right landing screen (so all the tools
actually have links) and a default avatar.

Test Plan:
Dropped all databases, initialized according to documentation, ended up in a
good state with sensible defaults.

Reviewed By: jungejason
Reviewers: aran, jungejason, tuomaspelkonen
CC: aran, jungejason
Differential Revision: 210
2011-05-02 13:38:46 -07:00
epriestley
94df249775 Improve schema upgrade workflow for unprivileged users
Summary:
In a basically reasonable configuration where you connect
with a non-privileged user from the web workflow, upgrade_schema.php
won't have enough privileges. Allow the user to override the normal
auth with -u and -p.

Test Plan:
Tried to do a schema upgrade with an underprivileged user,
got a useful error message instead of garbage.

Reviewed By: Girish
Reviewers: Girish, davidrecordon, jungejason, tuomaspelkonen, aran
CC: aran, epriestley, Girish
Differential Revision: 191
2011-04-30 00:50:48 -07:00
elynde
bd0a4c0d04 Differential Updates View
Summary:
This adds a new view to differential called Updates.

The high-level goal of Updates is to enabled differential to be
effectively used without email notifications. I've tried doing things
like automatically deleting differential emails where I'm in the 'to'
line since they show up on the main diffential page but then there's
always the chance an important diff flies by without me seeing it. Also,
sometimes someone comments on a diff post-commit but differential
doesn't surface those diffs.

I re-created a test db on my devserver using mysqldump to get data on
revs > 230000 so I would have some test data. We need to add a simple
viewtime table but I didn't want to do that in production. Here's the
table:

  CREATE TABLE differential_viewtime (
    viewerPHID varchar(64) not null,
    objectPHID varchar(64) not null,
    viewTime int unsigned not null,
    PRIMARY KEY (viewerPHID, objectPHID)
  );

Issues:
  -Once we turn this on, all diffs will be 'unviewed'. What do you think
about a 'Clear All' button or something?
  -Maybe we should add a pager

This feature would be insanely useful, let me know what you think.

Test Plan:
Loaded Updates in my sandbox

  http://phabricator.dev1577.snc6.facebook.com/differential/filter/updates/

Clicked a diff, then went back, made sure diff disappeared from Updates
list

Reviewed By: tuomaspelkonen
Reviewers: epriestley, jungejason, tuomaspelkonen
Commenters: epriestley
CC: epriestley, elynde, tuomaspelkonen
Differential Revision: 169
2011-04-28 14:40:41 -07:00
epriestley
ca7a0de1cf Prevent a race in Phabricator workers
Summary: See D133. Workers can also be subject to the same race, invert the
row relationship in the same way.

Test Plan: Launched repository master daemons and some taskmasters and used
the Daemon console to veify that they were able to process tasks. Manually
checked the database to make sure data got linked correctly and that new data
was inserted correctly.

Reviewers: jungejason

CC: tuomaspelkonen

Differential Revision: 135
2011-04-14 12:09:56 -07:00
epriestley
ee1e2da8fb Avoid Timeline race condition
Summary: While I should fix the transactional stuff, that patch is going to be
tricky and transactions have some performance implications. This is a simple
fix which prevents the race.

Instead of having the data point at the event ID, have the event point at a
data ID. Insert the data first, then insert the event with the right data
pointer. This is super simple and prevents the race issue.

Test Plan:
  - Ran the schema upgrade script, verified that the database was
    correctly upgraded. Was also prompted to stop daemons.
  - Ran 'repository-launch-master', verified that the discovery daemons were
    able to discover new commits and insert events for them. Verified the
    committask daemon was consuming events and converting them into tasks.
  - Verified new tasks looked correct in the database.
  - Browsed web interface.

Reviewers: jungejason

CC: tuomaspelkonen

Differential Revision: 133
2011-04-14 10:12:10 -07:00
tuomaspelkonen
f7fe75f756 Image macros for Phabricator!
Summary:
Added long waited image macro support for differential and others.

Test Plan:
Tried a couple of different macros and made sure they appear nicely
in the comment preview. Made sure that the normal comments are shown
correctly.

Reviewed By: epriestley
Reviewers: epriestley
CC: jungejason, tuomaspelkonen, epriestley
Differential Revision: 129
2011-04-13 20:08:13 -07:00
epriestley
43449e49c0 Very basic system agent support. 2011-04-12 18:19:25 -07:00
epriestley
158a561dff Add some metamta keys. 2011-04-12 18:19:24 -07:00
epriestley
720bf02570 Missing key for large datasizes. 2011-04-10 17:25:24 -07:00
epriestley
75b11d6d7d Lint and unit star support.
Test plan: quack

Differential Revision: 35
2011-04-10 17:19:01 -07:00
epriestley
e6c5d6c8ae Properly support mailing lists, with actual testing! 2011-04-10 10:16:14 -07:00
epriestley
8218ca15b9 Turns out MySQL tables need keys. Who knew?! 2011-04-09 22:19:10 -07:00
epriestley
22297b71a0 Close the loop on Diffusion commits posting back to Differential. 2011-04-07 21:59:42 -07:00
epriestley
aa0ff43999 xhpastview 2011-04-06 22:43:56 -07:00
epriestley
5dea10e27c Fix field capitalization. 2011-04-05 22:30:10 -07:00
epriestley
11ea93260a Sync up UUIDs and create project configs. 2011-04-05 21:55:04 -07:00
epriestley
23f882a0ee Some owners write workflows. 2011-04-03 22:03:27 -07:00
epriestley
65e1386753 owners 2011-04-03 15:50:51 -07:00
epriestley
7140107ed9 Optimize session query for nontrivial number of user accounts. 2011-04-02 16:39:40 -07:00
epriestley
82fffe466a Add basic detail-parser functionality. 2011-04-01 17:11:55 -07:00
tuomaspelkonen
6b8c2d110a Merge branch 'preferences' 2011-03-31 14:30:32 -07:00
tuomaspelkonen
123fdabb75 Removed "PHID Types" storage object and interface components
Summary:
Removed because code wasn't used or really needed.

Test Plan:
* Tested that "PHID List" and "PHID Lookup" pages work correctly.
* Tested that new PHIDs can be allocated with the predefined set of types

Reviewed By: epriestley
Reviewers: epriestley
CC: epriestley
Differential Revision: 88
2011-03-31 14:01:13 -07:00
tuomaspelkonen
28fe9f4eca User preferences ported from tools
Summary:

Internal tools, e.g., differential and diffusion  have user defined
preferences for monospaced font and the option for showing either the
name of the tool or the glyph of the tool in the title.

These preferences were ported to phabricator. These preferences can be
modified in /preferences/ and they both affect diffusion and differential
at the moment.

Test Plan:

* Created an empty database
* Loaded /preferences/ and modified the monospaced font and clicked save
* Confirmed that the same page was loaded with the message that preferences
  have been saved and that the example text used the user defined font

* in /preferences/ changed the option to show tool names as plain text and
  clicked save
* Confirmed that the same page was loaded with '[Preferences]' in the title
  instead of a glyph

* These same tests were also executed for differential and diffusion

Reviewers: epriestley
CC: jungejason

Differential Revision: 91
2011-03-31 13:44:20 -07:00
epriestley
34e8d902c7 Restore "Shortcuts" feature to Diffusion. 2011-03-31 00:33:44 -07:00
epriestley
e6cf7a9cb0 More Diffusion junk. 2011-03-30 22:08:41 -07:00
epriestley
919421ead7 Derp derp, apparentl neglected to save this file. 2011-03-26 23:59:29 -07:00
epriestley
fa8f168bd0 Allow commits to be marked as 'bad' so they won't be parsed. Useful if you work
at a company where someone deleted the entire repository *cough cough*.
2011-03-26 23:52:09 -07:00
epriestley
75fcf56b67 Rough cut of Herald home controller. 2011-03-22 13:50:08 -07:00
epriestley
e1d68e852a Sorta need this file. 2011-03-20 17:55:31 -07:00
epriestley
01a20c0480 Fix various parsing bugs in Differential. 2011-03-19 14:42:17 -07:00
EC2 Default User
31d2790075 durf durf sql 2011-03-15 20:51:53 +00:00
epriestley
bb144542d3 Diffusion/phd/console improvements. 2011-03-15 13:38:14 -07:00
epriestley
4196bfb3ef Minor fixes. 2011-03-14 07:43:13 -07:00
epriestley
383b3d740c Improve some diffusioney things.
Summary:

Test Plan:

Reviewers:

CC:
2011-03-12 22:51:40 -08:00
epriestley
485b5e5ded Make the Diffusion UI vaguely usable in some cases. 2011-03-12 16:17:34 -08:00
epriestley
48b5d6d025 Diffusion change parser for SVN.
Summary: there is a small possibility this works correctly

Test Plan:

Reviewers:

CC:
2011-03-11 14:13:23 -08:00
epriestley
8bcbeface1 Increase parseriness of some parsery things.
Summary: None of these are parsnips.

Test Plan:

Reviewers:

CC:
2011-03-11 09:34:22 -08:00
epriestley
ec084ca419 Rough cut of Workers
Summary: workers do work, provided I have written them correctly. perhaps it
is so.

Test Plan:

Reviewers:

CC:
2011-03-10 13:48:29 -08:00
epriestley
57495c4287 Rough cut of repository tracking
Summary: Basic scaffolding for repository tracking, plus daemon infrastructure
(Timelines, Cursors) and some fixes (memory usage, mysql_connect() junk).

Test Plan: parsed Javelin git commit history via daemon

Reviewers:

CC:
2011-03-06 22:29:22 -08:00
epriestley
063269a00a Store OAuth tokens and more OAuth account info.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-22 10:27:27 -08:00
epriestley
c3c16d0ac0 Github OAuth
Summary:

Test Plan:

Reviewers:

CC:
2011-02-21 00:23:24 -08:00
epriestley
c244d4bff2 Fully-qualify this SQL patchfile.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-20 20:10:07 -08:00
epriestley
7fb9a48690 Task -> Project assocation, file uploads
Summary:

Test Plan:

Reviewers:

CC:
2011-02-20 20:08:53 -08:00
epriestley
fdd510ba17 Rough cut of projects.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-20 18:41:23 -08:00
epriestley
bec2a1230c Up-to-date version of SQL.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-20 13:27:57 -08:00
epriestley
2e4022aca8 Improve docs to a semi-defensible level.
Summary:

Test Plan:

Reviewers:

CC:
2011-02-11 14:58:45 -08:00