1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-23 07:12:41 +01:00
Commit graph

503 commits

Author SHA1 Message Date
Chad Little
c9610721b1 Status Icons
Summary: Status icons for next to people names

Test Plan: photoshop

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2064

Differential Revision: https://secure.phabricator.com/D6479
2013-07-17 08:44:11 -07:00
Levi Jackson
d27e7c52b2 Add explicit mysql.port configuration
See: https://github.com/facebook/phabricator/pull/356

Reviewed by: epriestley
2013-07-14 16:06:23 -07:00
epriestley
544a84ebb9 Move outbound mail lists to CLI and enhance details
Summary: Finish off moving all this stuff to the CLI. Ref T3306.

Test Plan:
  PROPERTIES
  ID: 6483
  Status: void
  Retry Count: 0
  Next Retry: 1373494457
  Related PHID: PHID-DREV-5bnb33yeuhuaulyc3exg
  Message: Message has no valid recipients: all To/Cc are disabled, invalid, or configured not to receive this mail.

  PARAMETERS
  from: PHID-USER-lqiz3yd7wmk64ejugvov
  is-html:
  parent-message-id: null
  thread-id: differential-rev-PHID-DREV-5bnb33yeuhuaulyc3exg-req
  is-first-message: null
  is-bulk: 1
  mailtags: ["differential-comment"]
  cc: ["PHID-USER-cluwcdowc35gmperlkbi"]
  subject: D22: quack quack
  subject-prefix: [Differential]
  vary-subject-prefix: [Commented On]
  worker-task: 936546

  HEADERS
  Thread-Topic: D22: quack quack
  X-Herald-Rules: none
  X-Differential-Author: <PHID-USER-lqiz3yd7wmk64ejugvov>
  X-Differential-CC: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Differential-CC: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Differential-CC: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Differential-CC: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Differential-CC: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Differential-CC: <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-CCs: <PHID-USER-ly3pvrtdkw7lbgs72jvr>, <PHID-USER-cluwcdowc35gmperlkbi>, <PHID-MLST-wkxaantg3q6pgdkty5pt>, <PHID-USER-aeabc4ipqbifny3rw4ok>, <PHID-USER-zqxtb3oi4pouwxnxlv3f>, <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-Explicit-CC: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Differential-Explicit-CC: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Differential-Explicit-CC: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Differential-Explicit-CC: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Differential-Explicit-CC: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Differential-Explicit-CC: <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Differential-Explicit-CCs: <PHID-USER-ly3pvrtdkw7lbgs72jvr>, <PHID-USER-cluwcdowc35gmperlkbi>, <PHID-MLST-wkxaantg3q6pgdkty5pt>, <PHID-USER-aeabc4ipqbifny3rw4ok>, <PHID-USER-zqxtb3oi4pouwxnxlv3f>, <PHID-USER-cknqtm2dzw7twnwyiaye>
  X-Phabricator-To: <PHID-USER-lqiz3yd7wmk64ejugvov>
  X-Phabricator-Cc: <PHID-USER-ly3pvrtdkw7lbgs72jvr>
  X-Phabricator-Cc: <PHID-USER-cluwcdowc35gmperlkbi>
  X-Phabricator-Cc: <PHID-MLST-wkxaantg3q6pgdkty5pt>
  X-Phabricator-Cc: <PHID-USER-aeabc4ipqbifny3rw4ok>
  X-Phabricator-Cc: <PHID-USER-zqxtb3oi4pouwxnxlv3f>
  X-Phabricator-Cc: <PHID-USER-cknqtm2dzw7twnwyiaye>

  RECIPIENTS
  ! dog (dog)
      - This user is disabled; disabled users do not receive mail.

  BODY
  epriestley has commented on the revision "quack quack".

    zxcbzxcb

  REVISION DETAIL
    http://local.aphront.com:8080/D22

  To: epriestley
  Cc: Unknown User, dog, list, duck, epriestley992, asana

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3306

Differential Revision: https://secure.phabricator.com/D6423
2013-07-10 18:52:22 -07:00
epriestley
e8f679fa88 Move the inbound mail list to the CLI
Summary: Same as D6419. See D6413.

Test Plan:
      13  epriestley        D6                    Re: [local/Differential] [Request, 83 lines] D6: aabd
      14  epriestley        D6                    Re: [local/Differential] [Request, 83 lines] D6: aabd
      15  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      16  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      17  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      18  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      19  -                 -                     Re: [local/Diffusion] [Commit] rGTEST8fc313c77729: derp
      20  -                 -                     (No subject.)
      21  epriestley        D22                   (No subject.)
      22  epriestley        D12                   (No subject.)

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6420
2013-07-10 15:18:37 -07:00
epriestley
6bac7e390e Move "send test" to bin/mail
Summary: Like D6417. See D6413.

Test Plan:
  >>> orbital ~/devtools/phabricator $ echo "hello" | ./bin/mail send-test --to dog --cc duck --from epriestley --subject 'hey there' --attach README --tag blip --tag bloop
  Reading message body from stdin...
  Mail sent! You can view details by running this command:

      phabricator/ $ ./bin/mail show-outbound --id 6480

  >>> orbital ~/devtools/phabricator $ ./bin/mail show-outbound --id 6480
  PROPERTIES
  ID: 6480
  Status: sent
  Retry Count: 0
  Next Retry: 1373493443
  Related PHID:
  Message:

  PARAMETERS
  to: ["PHID-USER-cluwcdowc35gmperlkbi"]
  cc: ["PHID-USER-aeabc4ipqbifny3rw4ok"]
  subject: hey there
  is-html:
  is-bulk:
  mailtags: ["blip","bloop"]
  from: PHID-USER-lqiz3yd7wmk64ejugvov
  worker-task: 936543

  BODY
  hello

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6419
2013-07-10 15:18:24 -07:00
epriestley
7fa2343822 Move mail "Receive Test" from web UI to CLI
Summary:
Ref T3306. Moves this from the web to the CLI, which is a tiny bit clunkier but way better as far as policies go and more repeatable for development.

See discussion in D6413.

Test Plan: Ran `bin/mail receive-test`, verified mail was received. Used and abused various options.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3306

Differential Revision: https://secure.phabricator.com/D6417
2013-07-10 15:13:24 -07:00
epriestley
fcb56c6371 Move "scripts/sql/probe.php" to "bin/storage probe"
Summary: This makes it namespace/database/connection aware and a little easier to find. Also use pht() / PhutilConsole.

Test Plan: Ran `bin/storage probe`.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D6341
2013-07-02 16:34:17 -07:00
epriestley
90123dd739 Add DifferentialDiffQuery and change most callsites
Summary:
Ref T603. This introduces a policy-aware DifferentialDiffQuery and converts most callsites.

I've left unusual callsites (mostly: hard to get the viewer, unusual query, queries related to active diffs) alone for now, so this isn't exhaustive but hits 60-80% of sites.

Test Plan: Created diff; created revision; viewed diffs and revisions; made additional conduit calls.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603

Differential Revision: https://secure.phabricator.com/D6338
2013-07-01 12:38:42 -07:00
epriestley
42c0f060d5 Push feed publishing deeper into the task queue
Summary:
Ref T2852. I want to model Asana integration as a response to feed events. Currently, we queue one feed event for each HTTP hook.

Instead, always queue one feed event and then have it queue any necessary followup events (now, http hooks; soon, asana).

Add a script to make it easy to reproducibly fire feed event publishing.

Test Plan:
Republished a feed event and verified it hit configured HTTP hooks correctly.

  $ ./bin/feed republish 5765774156541908292 --trace
  >>> [2] <connect> phabricator2_feed
  <<< [2] <connect> 1,660 us
  >>> [3] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [3] <query> 595 us
  >>> [4] <connect> phabricator2_differential
  <<< [4] <connect> 760 us
  >>> [5] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [5] <query> 478 us
  >>> [6] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [6] <query> 449 us
  >>> [7] <connect> phabricator2_user
  <<< [7] <connect> 1,062 us
  >>> [8] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [8] <query> 540 us
  >>> [9] <connect> phabricator2_file
  <<< [9] <connect> 951 us
  >>> [10] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [10] <query> 498 us
  >>> [11] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [11] <query> 507 us
  Republishing story...
  >>> [12] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [12] <query> 685 us
  >>> [13] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [13] <query> 489 us
  >>> [14] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [14] <query> 512 us
  >>> [15] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [15] <query> 601 us
  >>> [16] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [16] <query> 405 us
  >>> [17] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [17] <query> 551 us
  >>> [18] <query> SELECT story.* FROM `feed_storydata` story JOIN `feed_storyreference` ref ON ref.chronologicalKey = story.chronologicalKey WHERE (ref.chronologicalKey IN (5765774156541908292)) GROUP BY story.chronologicalKey ORDER BY story.chronologicalKey DESC
  <<< [18] <query> 507 us
  >>> [19] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [19] <query> 428 us
  >>> [20] <query> SELECT * FROM `differential_revision` WHERE phid IN ('PHID-DREV-ywqmrj5zgkdloqh5p3c5')
  <<< [20] <query> 419 us
  >>> [21] <query> SELECT * FROM `user` WHERE phid in ('PHID-USER-lqiz3yd7wmk64ejugvov')
  <<< [21] <query> 591 us
  >>> [22] <query> SELECT * FROM `file` WHERE phid IN ('PHID-FILE-gq6dlsysvxbn3dgwvky7')
  <<< [22] <query> 406 us
  >>> [23] <query> SELECT * FROM `user_status` WHERE userPHID IN ('PHID-USER-lqiz3yd7wmk64ejugvov') AND UNIX_TIMESTAMP() BETWEEN dateFrom AND dateTo
  <<< [23] <query> 593 us
  >>> [24] <http> http://127.0.0.1/derp/
  <<< [24] <http> 746,157 us
  [2013-06-24 20:23:26] EXCEPTION: (HTTPFutureResponseStatusHTTP) [HTTP/500] Internal Server Error

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6291
2013-06-25 16:29:47 -07:00
epriestley
c94ef134e4 Add bin/auth refresh for debugging OAuth token refresh issues
Summary: Ref T2852. Provide a script for inspecting/debugging OAuth token refresh.

Test Plan: Ran `bin/auth refresh` with various arguments, saw token refreshes.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2852

Differential Revision: https://secure.phabricator.com/D6276
2013-06-24 15:55:41 -07:00
epriestley
6a2e27ba8d Put all DifferentialComment loading behind DifferentialCommentQuery
Summary:
Ref T2222.

I'm thinking about how I want to approach the Asana sync, and I want to try to do T2222 first so that we can build it cleanly on top of ApplicationTransactions. I think we can at least walk down this road a little bit and if it turns out to be scary we can take another approach.

I was generally very happy with how the auth migration turned out (seemingly, it was almost completely clean), and want to pursue a similar strategy here. Basically:

  - Wrap the new objects in the old objects for reads/writes.
  - Migrate all the existing data to the new table.
  - Everything hard is done; move things over a piece at a time at a leisurely pace in lots of smallish, relatively-easy-to-understand changes.

This deletes or abstracts all reads of the DifferentialComment table. In particular, these things are **deleted**:

  - The script `undo_commits.php`, which I haven't pointed anyone at in a very long time.
  - The `differential.getrevisionfeedback` Conduit method, which has been marked deprecated for a year or more.
  - The `/stats/` interface in Differential, which should be rebuilt on Fact and has never been exposed in the UI. It does a ton of joins and such which are prohibitively difficult to migrate.

This leaves a small number of reading interfaces, which I replaced with a new `DifferentialCommentQuery`. Some future change will make this actually load transactions and wrap them with DifferentialComment interfaces.

Test Plan: Viewed a revision; made revision comments

Reviewers: btrahan

Reviewed By: btrahan

CC: edward, chad, aran

Maniphest Tasks: T2222

Differential Revision: https://secure.phabricator.com/D6260
2013-06-21 12:51:18 -07:00
epriestley
c0cc7bbfdf Provide bin/auth ldap for LDAP diagnostics
Summary: Ref T1536.

Test Plan: Ran `bin/auth ldap`.

Reviewers: mbishopim3, chad

Reviewed By: mbishopim3

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6218
2013-06-17 13:26:25 -07:00
epriestley
278905543e Add very basic bin/auth tool
Summary: Ref T1536. This script basically exists to restore access if/when users shoot themselves in the foot by disabling all auth providers and can no longer log in.

Test Plan: {F46411}

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1536

Differential Revision: https://secure.phabricator.com/D6205
2013-06-17 10:55:05 -07:00
Chad Little
02b59e685f PHUIButtonView class
Summary: Rough pass at a PHUIButtonView Class. Keeps phutil_tag intact and adds some image features if you use the class.

Test Plan: UIExamples

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6192
2013-06-12 18:23:35 -07:00
Chad Little
8e6bda51f2 Login icons
Summary: Took a stab at some login icons for buttons.

Test Plan: photoshop

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D6174
2013-06-11 10:22:09 -07:00
epriestley
5d1f94ac8a Fix some Phabricator lint warnings
Summary: Lint.

Test Plan: Lint.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D6127
2013-06-04 15:28:24 -07:00
epriestley
87bc6eb28c Use ObjectItemListView in Diviner
Summary: Ref T988. Nuke the old directory CSS.

Test Plan: {F44796}

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D6093
2013-05-31 10:51:53 -07:00
epriestley
0f070236bd Add a bin/files purge workflow
Summary:
We can lose file data through various means; one reasonable way is if files get deleted from disk with 'local-disk' storage. If data goes missing,

Ref T3265. Also, reduce some code duplication.

Test Plan:
Ran `bin/files purge`, `bin/files migrate`, `bin/files rebuild` with various args.

Deleted a file with "local-disk" storage, ran `bin/files purge`, made sure it got picked up.

Reviewers: dctrwatson, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T3265

Differential Revision: https://secure.phabricator.com/D6068
2013-05-29 06:28:57 -07:00
epriestley
e01ceaa07f Provide 'bin/cache', for managing caches
Summary:
See <https://github.com/facebook/phabricator/issues/323>. We have a very old cache management script which doesn't purge all the modern caches (and does purge some caches which are no longer in use). Update it so it purges all the modern caches (remarkup, general, changeset), no longer purges outdated caches, and is easier to use.

Also delete a lot of "this script has moved" scripts from the last few rounds of similar cleanup, I believe all of these have been in master for at least several months, which should be enough time for users to get used to the new stuff.

Test Plan: Ran `bin/cache` with various arguments. Verified caches were purged.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5978
2013-05-20 10:16:35 -07:00
epriestley
e4525f9de1 Provide some diagnostic tools for examining inbound and outbound mail
Summary: We can't show this stuff on the web UI because it has password reset links and private reply-to addresses, but we can provide easier CLI tools than "root around in the database". Land a rough version of `bin/mail show-inbound` and `bin/mail show-outbound`.

Test Plan: Used both commands to examine mail from the CLI.

Reviewers: btrahan

Reviewed By: btrahan

CC: tido, euresti, aran

Differential Revision: https://secure.phabricator.com/D5963
2013-05-20 10:13:42 -07:00
epriestley
a1a46656cb Minor, package more resources and rebuild Celerity map. 2013-05-18 17:16:24 -07:00
epriestley
4fc9b8e785 Apply sprite sheet changes to Phabricator
Summary: Basically a one line change, plus a regen. This should be the last full-rewrite these things get since they'll stop fully rehashing all the time now.

Test Plan: See D5964.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D5965
2013-05-18 10:34:10 -07:00
epriestley
9e5410e6a2 Rename bin/files metadata to bin/files rebuild and let it rebuild MIME information
Summary:
See <https://github.com/facebook/phabricator/issues/320>. Files can end up with a bad MIME type, and we don't update it when uploading another copy of the file since obviously the new copy has the same data and thus the same MIME type.

  - Rename `bin/files metadata` to `bin/files rebuild` to make it a more consistent verb.
  - Let it rebuild MIME types so users who hit issues like this can run `bin/files rebuild --all --rebuild-mime` to straighten things out.

Test Plan: Ran `bin/files` in various modes, examined output.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5951
2013-05-17 10:00:31 -07:00
epriestley
eabe3a4d33 Begin improving the soundness of received mail
Summary:
We/I broke a couple of things here recently (see D5911) and are doing some work here in general (see D5912, etc.).

Generally, this code is pretty oldschool and not especially well architected for modern application-oriented Phabricator. It hardcodes a lot of stuff which should be applications' responsibilites.

Take the first steps toward making it more solid to reduce the risk here. In particular:

  - Factor out the "self mail" and "duplicate mail" checks and add unit tests.
  - Make Message-ID hash handling automatic.

Test Plan: Ran unit tests.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5915
2013-05-13 16:32:19 -07:00
epriestley
cbd3c0b7ac Add action icons to object list views
Summary:
We have a few interfaces where add "Edit", "Delete" or some other action to a list. Currently, this happens via icons, but these are cumbersome and weird, are inconsistent, can't be workflow'd, are hard to hit on desktops and virtually impossible to hit on mobile, and generally just feel iffy to me. Prominent examples are Projects and Flags. I'd like to try adding an "edit" action to Maniphest (to provide quick edit from list views, basically). It looks like some of Releeph would benefit here, as well.

Instead, provide first-class actions:

{F42978}

They produce targets which my meaty ham-fists can plausibly hit on mobile, too:

{F42979}

(We could do some kind of swipe-to-expose thing eventually, but I think putting them by default is OK?)

Test Plan: Added UIExamples. Checked desktop/mobile.

Reviewers: chad, btrahan, edward

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D5890
2013-05-10 12:57:01 -07:00
Jakub Vrana
3bfad75764 Put JX.phtize in package 2013-04-23 17:58:56 -07:00
Jakub Vrana
5c0333cef9 Make Celerity merger Windows compatible 2013-04-23 11:11:21 -07:00
Chad Little
5c100c0d69 Payment sprite icons.
Summary: Adds some icons for taking money.

Test Plan: UIExamples

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5753
2013-04-22 16:41:00 -07:00
deedydas
4fb1a9e00f First Diff of Test Data Generator
Summary: Progress to fix T2903

Test Plan: Ran './bin/lipsum help' and 'generate' workflows.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2903

Differential Revision: https://secure.phabricator.com/D5672
2013-04-12 14:07:16 -07:00
Jakub Vrana
d98401833b Check presence of required functions in daemon launcher
Summary: I had quite some trouble starting daemons on a machine with installed **pcntl** but with [[ http://www.php.net/ini.core#ini.disable-functions | disabled ]] all functions from it.

Test Plan:
  $ bin/phd start

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5568
2013-04-07 08:42:59 -07:00
Angelos Evripiotis
875455ad64 Document reparse --min-date more, add validation
Summary:
Sometimes it seems necessary to force a reparse of recent commits in
production, it took me longer than expected to get this right.

To make this easier, document the usage of --min-date
further with usage examples and print a usage exception with the input
if the supplied value isn't accepted by MySQL.
(otherwise all commits will be affected in the case of user error)

Test Plan:
.. create TEST repo with commits dated 2013-04-03 ..

$ ./reparse.php --all TEST --owners --min-date "2013a-04-03 10:30:19"
.. see usage exception - invalid timestamp ..

$ ./reparse.php --all TEST --owners --min-date "2013-04-03 10:30:19"
.. reparse commits ok ..

$ ./reparse.php --all TEST --owners --min-date "2013-04-04 10:30:19"
.. see 'No commits have been discovered' ..

$ ./reparse.php --all TEST --owners
.. reparse commits ok ..

$ ./reparse.php --help
.. looks ok to me ..

$ ./reparse.php --all TEST --owners --min-date 2013-04-03 10:30:19
.. see error - interprets 10:30:19 as commit and refuses ..

$ ./reparse.php --all TEST --owners --min-date <<first commit time>>
.. parse this commit and following ..

$ ./reparse.php <<revision_id>> --owners --min-date <<first commit time>>
.. see error - insist on --all if --min-date  ..

$ ./reparse.php <<revision_id>> --owners
.. ok  ..

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5579
2013-04-05 07:35:35 -07:00
Chad Little
74609d9366 Action Icons
Summary: First pass at 'action icons' for headers and other items. Ties into future diff.

Test Plan: photoshop, read the css.

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin, AnhNhan

Differential Revision: https://secure.phabricator.com/D5565
2013-04-04 10:38:50 -07:00
epriestley
c73e888ad4 Consolidate file pasting code
Summary: Fixes T2861. This used to work but I think I broke it when I made the notification popup thing work. Merge it into the drag-and-drop since 90% of the code is the same anyway.

Test Plan: Pasted files into Chrome and got uploads. This doesn't work in Safari and Firefox; as far as I know it isn't supported in those browers.

Reviewers: blc, btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2861

Differential Revision: https://secure.phabricator.com/D5530
2013-04-02 09:52:31 -07:00
epriestley
e80c59cbc6 Introduce basic bin/mail with a resend workflow
Summary:
Fixes T2458. Ref T2843. @tido's email from T2843 has exhausted its retries and failed, but we want to try it again with the patch from D5464 to capture the actual error. This sort of thing has come up a few times in debugging, too.

Also fixed some stuff that came up while debugging this.

Test Plan:
  - Ran command with no args.
  - Ran resend with no args.
  - Ran resend with bad IDs.
  - Ran resend with already-queued messages, got "already queued" error.
  - Ran resend with already-sent message, got requeue.

Reviewers: btrahan, tido

Reviewed By: tido

CC: aran

Maniphest Tasks: T2458, T2843

Differential Revision: https://secure.phabricator.com/D5493
2013-03-30 15:53:49 -07:00
Afaque Hussain
69d087a1d7 Replaced AphrontCrumbsView with PhabricatorCrumbsView in PhrictionDiffController
Summary: Same as title

Test Plan: By checking in Phriction UI in Phabricator

Reviewers: epriestley, AnhNhan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5327
2013-03-13 15:18:18 -07:00
Bob Trahan
9d806b69cb Conpherence - some cleanup type stuff
Summary:
this diff does a few, not so exciting things

- changes "conpher" to "conpherence" where it snuck into CSS, spritemap, etc -- I believe we now consistently call it conpherence. Feel free to change it, just change it everywhere. :D
- puts the widget icons in the right order per M14
- makes the "mobile-only" widgets show the toggles only in the mobile view
 - also made it so clicking them does nothing for now
- removes the tasks widget since we don't want it

...my time is getting chopped up funny (yay puppy) so this is just an attempt at something that can go into the codebase and not make it worse. Next up is making the widgets that show on desktop look right / not say "TODO"

Test Plan: played around in Conpherence

Reviewers: epriestley, chad

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2530

Differential Revision: https://secure.phabricator.com/D5334
2013-03-13 13:03:51 -07:00
Chad Little
fbb032c710 Document icons sprite sheet.
Summary: Adds 32 and 64 px images to represent basic documents.

Test Plan: Differential

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5317
2013-03-10 14:16:16 -07:00
epriestley
6b5204dca9 Add support for device swipe events
Summary:
Ref T2700. Allow JS to listen for swipes on devices.

There are a bunch of tricky cases here and I probably didn't get them all totally right, but this interaction broadly looks like this:

  - We implement gesture recognition for the mouse in device modes (narrow browser), and for touch events from an actual device.
  - The sigil `touchable` indicates that a node wants to react to touch events.
  - When the user touches a `touchable` node, we start listening for moves. They might be tapping/clicking (in which case we don't care), but they might also be gesturing.
  - Once the user moves their finger/pointer far enough away from the tap origin, we recognize it as a gesture. I hardcoded this at 20px; I wasn't able to find any "official" Apple value, but 20px seems like a common default.
  - At this point, we look at where their finger has moved.
    - If they moved it mostly up/down, we interpret the gesture as "scroll" and just stop listening. The device does its own thing.
    - However, if they moved it mostly left/right, we interpret it as a "swipe". We start killing the moves so the device doesn't scroll.
  - Once we've recognized that a gesture is underway, we send a "gesture.swipe.start" event and then "gesture.swipe.move" events for every move.
  - When the user ends the gesture, we send "gesture.swipe.end".
  - If the user cancels the gesture (currently, only by tapping with a second finger), we send "gesture.swipe.cancel".
  - Gesture events have raw position data and some convenience fields.

Test Plan:
Wrote UI example and used it from the Desktop, iPhone simulator, and a real iphone.

  - The code always seems to get "scroll" vs "swipe" correct (i.e., consistent with my intentions).
  - The threshold feels pretty good to me.
  - Tapping with a second finger cancels the action.

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T2700

Differential Revision: https://secure.phabricator.com/D5308
2013-03-09 13:53:15 -08:00
vrana
1091dc7aa1 Save blame info to lint messages
Test Plan:
Applied the patch.
Looked at blame and plain blame of SVN and Git file.
Ran the lint saver.
Looked at lint messages list.
/diffusion/lint/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5218
2013-03-06 16:19:01 -08:00
vrana
e57f209056 Load blame in Diffusion by AJAX
Summary:
I have blame enabled by default and displaying files with long history takes easily over 10 seconds.
Load the blame data by AJAX instead.
This is actually doing more work and the total response time is longer but it's worth it for me as I am interested just in the file contents quite often.

I know you were talking about building blame cache but until we have it...

I'm not sure if the AJAX loading indicator in bottom right corner is enough to inform the user that we are loading it on background.

Test Plan:
?view=highlighted
?view=plainblame
?view=blame

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5244
2013-03-06 07:44:45 -08:00
vrana
c9838d81e2 Don't reload page after clicking on Diffusion line link
Summary:
For a single line, I can use the right click.
But for highlighting multiple lines, I have to wait for page reload.

It would be nice to track this in history but that's more involved.
This is actually maybe better behavior.

Test Plan: Clicked on the line link, dragged and dropped on it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D5172
2013-03-04 13:46:34 -08:00
epriestley
32d23254c9 Use --user and --password from bin/storage in PHP migrations
Summary:
Fixes T2059. Ref T2517.

Currently, you can run `bin/storage upgrade` with `--user` and `--password` arguments. However, these clownishly apply only to `.sql` patches -- the `.php` migrations still use the default user and password.

This is dumb. Stop doing it. Respect `--user` and `--password` for PHP patches.

(I implemented "override", which is very similar to "repair", but kept them separate since I think they're semantically distinct enough to differentiate.)

Test Plan: Ran `./bin/storage upgrade --user x --pass y --apply phabricator:20130219.commitsummarymig.php`. Verified the correct user and password were used both for the initial connect and patch application.

Reviewers: chad, vrana

Reviewed By: chad

CC: aran

Maniphest Tasks: T2059, T2517

Differential Revision: https://secure.phabricator.com/D5115
2013-02-25 22:20:23 -08:00
epriestley
2231e5200a Implement basic transaction detail blocks
Summary:
Some transactions (like editing configuration values, task descriptions, or Conpherence images) can't be simply explained and need an additional larger element to show them fully (like a text diff).

Support change details like this in ApplicationTransactions. Implements the element in Config, so you can see changes.

Test Plan: {F32974}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2213

Differential Revision: https://secure.phabricator.com/D4984
2013-02-17 06:37:02 -08:00
epriestley
0e7382b102 Fix conpherence sprites
Summary:
The map had "conph" but everything else refers to "conpher". The "conph" sprite thing won when I regenerated sprites for tokens.

I should just fix this so it can't happen, but unbreak for now. Renamed "conph" -> "conpher", regenerated sprites, nuked all the "conph" stuff.

Test Plan: Looked at Conpherence, saw icons.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4982
2013-02-15 19:04:25 -08:00
epriestley
49c40d209d Tokens v1
Summary:
Features!

  - Giving tokens.
  - Taking tokens back.
  - Not giving tokens.

Test Plan: See screenshots.

Reviewers: chad, vrana

Reviewed By: chad

CC: aran, btrahan

Maniphest Tasks: T2541

Differential Revision: https://secure.phabricator.com/D4964
2013-02-15 07:47:14 -08:00
epriestley
7a2826392f Simplify the "update_phabricator.sh" script
Summary: Pare this down to the bare bones. Diviner is going away soon anyway.

Test Plan: Read instructions.

Reviewers: edward

Reviewed By: edward

CC: aran

Differential Revision: https://secure.phabricator.com/D4951
2013-02-14 07:22:43 -08:00
epriestley
f705c978b5 render_tag -> tag: phabricator_render_form -> phabricator_form
Summary: Convert most phabricator_render_form callsites. In the case of the "headsup view", it converts it by deleting the element entirely (this is the very old Maniphest/Differential header which we no longer use).

Test Plan: Poked around a bit.

Reviewers: vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4726
2013-01-30 11:30:38 -08:00
epriestley
39221b1d3f Merge branch 'master' into phutil_tag
(Synchronizing.)
2013-01-29 11:05:02 -08:00
epriestley
114ed6c7fe DarkConsole: fix rendering, move request log, load over ajax
Summary:
This accomplishes three major goals:

  # Fixes phutil_render_tag -> phutil_tag callsites in DarkConsole.
  # Moves the Ajax request log to a new panel on the left. This panel (and the tabs panel) get scrollbars when they get large, instead of making the page constantly scroll down.
  # Loads the panel content over ajax, instead of dumping it into the page body / ajax response body. I've been planning to do this for about 3 years, which is why the plugins are architected the way they are. This should make debugging easier by making response bodies not be 50%+ darkconsole stuff.

Additionally, load the plugins dynamically (the old method predates library maps and PhutilSymbolLoader).

Test Plan:
{F30675}

  - Switched between requests and tabs, reloaded page, saw same tab.
  - Used "analyze queries", "profile page", triggered errors.
  - Verified page does not load anything by default if dark console is closed with Charles.
  - Generally banged on it a bit.

Reviewers: vrana, btrahan, chad

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2432

Differential Revision: https://secure.phabricator.com/D4692
2013-01-28 18:45:32 -08:00
Chad Little
93eac1f9d3 Add Conpherence sprites.
Summary: Let's see if I did this right. This adds on and off state icons (1 and 2x) for conpherence. I think I need to tweak and add more CSS to have the off hover state be the on icon. Will check.

Test Plan: spritegen

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2400

Differential Revision: https://secure.phabricator.com/D4709
2013-01-28 15:56:29 -08:00
epriestley
07767fda00 Use direct inclusion, not submodules, to bring Javelin into Phabricator
Summary:
Submoduling is slightly convenient for developers but hellishly difficult for many users. Since we make about a dozen updates to Javelin per year, just include the source directly.

Even if we run `git submodule status` more often, this creates additional problems for users with PATH misconfigured.

Fixes T2062 by nuking it from orbit.

Test Plan: Loaded site, browsed around. Grepped for references to submodules.

Reviewers: btrahan, vrana

CC: aran

Maniphest Tasks: T2062

Differential Revision: https://secure.phabricator.com/D4581
2013-01-22 12:43:55 -08:00
epriestley
35d73414f8 Remove legacy support for 'phd repository-launch' and 'phd repository-launch-readonly'
Summary: These have been marked as deprecated since May 2012. Clean them up.

Test Plan: Grepped for `repository-launch`, `phd_load_tracked_repositories`: no hits.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2372

Differential Revision: https://secure.phabricator.com/D4575
2013-01-22 12:26:08 -08:00
epriestley
baa9d96082 Increase the power of bin/config
Summary:
Fixes T2254. Make the CLI for config more powerful:

  - Add validation for `set`.
  - Add `get`.
  - Add `list`.
  - Add `delete`.

The `get` command produces fairly verbose JSON to support flags like `--all`, or `--source database` later. The other commands are straightforward.

Test Plan:
Tested `config set`:

  $ ./bin/config set
  Usage Exception: Specify a configuration key and a value to set it to.
  $ ./bin/config set x
  Usage Exception: Specify a value to set the key 'x' to.
  $ ./bin/config set phabricator.base-uri
  Usage Exception: Specify a value to set the key 'phabricator.base-uri' to.
  $ ./bin/config set phabricator.base-uri x
  Usage Exception: Config option 'phabricator.base-uri' is invalid. The URI must start with 'http://' or 'https://'.
  $ ./bin/config set phabricator.base-uri http://x
  Usage Exception: Config option 'phabricator.base-uri' is invalid. The URI must contain a dot ('.'), like 'http://example.com/', not just a bare name like 'http://example/'. Some web browsers will not set cookies on domains with no TLD.
  $ ./bin/config set phabricator.base-uri http://x.com
  Set 'phabricator.base-uri' in local configuration.
  $

Tested `config get`:

  $ ./bin/config get pygments.enabled
  {
    "config" : []
  }
  $ ./bin/config set pygments.enabled true
  Set 'pygments.enabled' in local configuration.
  $ ./bin/config get pygments.enabled
  {
    "config" : [
      {
        "key"    : "pygments.enabled",
        "source" : "local",
        "value"  : true
      }
    ]
  }
  $

Tested `config delete`:

  $ ./bin/config delete
  Usage Exception: Specify a configuration key to delete.
  $ ./bin/config delete x x
  Usage Exception: Too many arguments: expected one key.
  $ ./bin/config delete x
  Usage Exception: No such configuration key 'x'! Use `config list` to list all keys.
  $ ./bin/config delete pygments.enabled
  Deleted 'pygments.enabled' from local configuration.
  $ ./bin/config delete pygments.enabled
  Usage Exception: Configuration key 'pygments.enabled' is not set in local configuration!
  $

Tested `config list`:

  $ ./bin/config list
  account.editable
  account.minimum-password-length
  amazon-ec2.access-key
  amazon-ec2.secret-key
  amazon-s3.access-key
  amazon-s3.endpoint
  amazon-s3.secret-key
  amazon-ses.access-key
  amazon-ses.secret-key
  aphront.default-application-configuration-class
  audit.can-author-close-audit
  auth.email-domains
  auth.login-message
  auth.password-auth-enabled
  auth.require-email-verification
  auth.sessions.conduit
  auth.sessions.web
  auth.sshkeys.enabled
  cache.enable-deflate
  celerity.force-disk-reads
  celerity.minify
  celerity.resource-hash
  celerity.resource-path
  config.hide
  config.lock
  config.mask
  controller.oauth-registration
  darkconsole.always-on
  darkconsole.enabled
  debug.profile-rate
  debug.stop-on-redirect
  differential.allow-reopen
  differential.allow-self-accept
  differential.always-allow-close
  differential.anonymous-access
  differential.custom-remarkup-block-rules
  differential.custom-remarkup-rules
  differential.days-fresh
  differential.days-stale
  differential.enable-email-accept
  differential.expose-emails-prudently
  differential.field-selector
  differential.generated-paths
  differential.require-test-plan-field
  differential.revision-custom-detail-renderer
  differential.show-host-field
  differential.show-test-plan-field
  differential.whitespace-matters
  disqus.application-id
  disqus.application-secret
  disqus.auth-enabled
  disqus.auth-permanent
  disqus.registration-enabled
  disqus.shortname
  environment.append-paths
  events.listeners
  facebook.application-id
  facebook.application-secret
  facebook.auth-enabled
  facebook.auth-permanent
  facebook.registration-enabled
  facebook.require-https-auth
  feed.http-hooks
  feed.public
  files.image-mime-types
  files.viewable-mime-types
  gcdaemon.ttl.daemon-logs
  gcdaemon.ttl.differential-parse-cache
  gcdaemon.ttl.general-cache
  gcdaemon.ttl.herald-transcripts
  gcdaemon.ttl.markup-cache
  gcdaemon.ttl.task-archive
  github.application-id
  github.application-secret
  github.auth-enabled
  github.auth-permanent
  github.registration-enabled
  google.application-id
  google.application-secret
  google.auth-enabled
  google.auth-permanent
  google.registration-enabled
  ldap.activedirectory_domain
  ldap.anonymous-user-name
  ldap.anonymous-user-password
  ldap.auth-enabled
  ldap.base_dn
  ldap.hostname
  ldap.port
  ldap.real_name_attributes
  ldap.referrals
  ldap.search-first
  ldap.search_attribute
  ldap.start-tls
  ldap.username-attribute
  ldap.version
  load-libraries
  log.access.format
  log.access.path
  maniphest.custom-fields
  maniphest.custom-task-extensions-class
  maniphest.default-priority
  maniphest.enabled
  metamta.can-send-as-user
  metamta.default-address
  metamta.differential.attach-patches
  metamta.differential.inline-patches
  metamta.differential.patch-format
  metamta.differential.reply-handler
  metamta.differential.reply-handler-domain
  metamta.differential.subject-prefix
  metamta.differential.unified-comment-context
  metamta.diffusion.attach-patches
  metamta.diffusion.byte-limit
  metamta.diffusion.inline-patches
  metamta.diffusion.reply-handler
  metamta.diffusion.reply-handler-domain
  metamta.diffusion.subject-prefix
  metamta.diffusion.time-limit
  metamta.domain
  metamta.herald.show-hints
  metamta.insecure-auth-with-reply-to
  metamta.macro.reply-handler-domain
  metamta.macro.subject-prefix
  metamta.mail-adapter
  metamta.maniphest.default-public-author
  metamta.maniphest.public-create-email
  metamta.maniphest.reply-handler
  metamta.maniphest.reply-handler-domain
  metamta.maniphest.subject-prefix
  metamta.one-mail-per-recipient
  metamta.package.reply-handler
  metamta.package.subject-prefix
  metamta.pholio.reply-handler-domain
  metamta.pholio.subject-prefix
  metamta.placeholder-to-recipient
  metamta.precedence-bulk
  metamta.public-replies
  metamta.re-prefix
  metamta.recipients.show-hints
  metamta.reply.show-hints
  metamta.send-immediately
  metamta.single-reply-handler-prefix
  metamta.user-address-format
  metamta.vary-subjects
  mysql.configuration-provider
  mysql.host
  mysql.implementation
  mysql.pass
  mysql.user
  notification.client-uri
  notification.debug
  notification.enabled
  notification.log
  notification.pidfile
  notification.server-uri
  notification.user
  phabricator.application-id
  phabricator.application-secret
  phabricator.auth-enabled
  phabricator.auth-permanent
  phabricator.base-uri
  phabricator.csrf-key
  phabricator.env
  phabricator.mail-key
  phabricator.oauth-uri
  phabricator.production-uri
  phabricator.registration-enabled
  phabricator.serious-business
  phabricator.setup
  phabricator.show-beta-applications
  phabricator.show-error-callout
  phabricator.show-stack-traces
  phabricator.timezone
  phame.skins
  phd.log-directory
  phd.pid-directory
  phd.start-taskmasters
  phd.trace
  phd.verbose
  phid.external-loaders
  phpmailer.mailer
  phpmailer.smtp-host
  phpmailer.smtp-password
  phpmailer.smtp-port
  phpmailer.smtp-protocol
  phpmailer.smtp-user
  phriction.enabled
  policy.allow-public
  pygments.dropdown-choices
  pygments.enabled
  recaptcha.enabled
  recaptcha.private-key
  recaptcha.public-key
  remarkup.enable-embedded-youtube
  repository.default-local-path
  search.elastic.host
  search.engine-selector
  security.alternate-file-domain
  security.hmac-key
  security.require-https
  sendgrid.api-key
  sendgrid.api-user
  storage.default-namespace
  storage.engine-selector
  storage.local-disk.path
  storage.mysql-engine.max-size
  storage.s3.bucket
  storage.upload-size-limit
  style.monospace
  syntax-highlighter.engine
  syntax.filemap
  test.value
  tokenizer.ondemand
  translation.override
  translation.provider
  uri.allowed-protocols
  $

Reviewers: btrahan, codeblock

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2254

Differential Revision: https://secure.phabricator.com/D4570
2013-01-21 15:27:42 -08:00
epriestley
d5b006b2cc Remove pcntl/php CLI setup checks
Summary:
As far as I know, we never actually need `php` to be available from the web UI. I think the history here is:

  - Long ago, we checked for 'pcntl' as an extension during setup.
  - Someone had an install where 'pcntl' was available from the CLI, but not the web UI. So we switched the check to use the CLI.
  - Someone had an install where the CLI binary was php-fpm, which caused the 'pcntl' check to loop endlessly, so we added more checks.

But we don't actually need to do any of this -- when the user tries to run the daemons, they get an explicit message that they need to install pcntl already, and we never (as far as I know) try to run PHP scripts from the web UI other than the pcntl_available.php check (we only run `git`, `svn`, `hg`, `ssh-agent`, `diff`, `xhpast` and `pygmentize`, I think).

Test Plan: Thought carefully about places we might execute PHP scripts from the web UI. Looked through /scripts/ to try to identfiy anything we might execute.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D4568
2013-01-21 11:57:24 -08:00
vrana
aa3e95cbeb Cache connection in bin/storage
Summary: Connection takes .3s from dev server to master.

Test Plan:
  $ bin/storage --trace upgrade --namespace x
  $ bin/storage --trace destroy --namespace x

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4480
2013-01-17 11:00:36 -08:00
epriestley
fffa7ffb6c Allow users to customize applicaiton tile sizes
Summary:
See discussion in D4438. Allows users to customize application tiles, and implements generally reasonable defaults so they hopefully won't.

Sizes are "invisible" (internal only, used to hide admin apps from non-admins), "hidden" (hide by default, show after clicking "Show More Applications"), "show" (show a small square tile) and "full" (show a full-width tile with subtitle).

Test Plan:
Default view for a non-admin:

{F29375}

Adjusted settings, hidden:

{F29373}

Adjusted settings, shown:

{F29374}

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4439
2013-01-16 09:00:11 -08:00
epriestley
c506cfe8d9 Implement very basic uberhome
Summary:
No fancy-pants smarty stuff yet, but merges /applications/ and the awful application buttons into the dark navigation.

Hover state is maybe a little weird.

Test Plan: {F29324}

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran, btrahan, codeblock

Differential Revision: https://secure.phabricator.com/D4431
2013-01-15 15:41:22 -08:00
epriestley
7ad3147b3b Remove all older-style navs
Summary: Move all navs to use the newer-style, darker, textured look. I'm //pretty// sure this doesn't break anything.

Test Plan: Looked at a bunch of apps.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4376
2013-01-09 12:03:58 -08:00
epriestley
4adf55919c Port Diviner Core to Phabricator
Summary:
This implements most/all of the difficult parts of Diviner on top of Phabricator instead of as standalone components. See T988. In particular, here are the things I want to fix:

**Performance** The Diviner parser works in two stages. The first stage breaks source files into "Atoms". The second stage renders atoms into a display format (e.g., HTML). Diviner currently has a good caching story on the first step of the pipeline, but zero caching in the second step. This means it's very slow, even for a fairly small project like Phabricator. We must re-render every piece of documentation every time, instead of only changed documentation. Most of this diff concerns itself with addressing this problem. There's a fairly large explanatory comment about it, but the trickiest part is that when an atom changes, other atoms (defined in other places) may also change -- for example, if `class B extends A`, editing A should dirty B, even if B is in an entirely different file. We perform analysis in two stages to propagate these changes: first detecting direct changes, then detecting indirect changes. This isn't completely implemented -- we need to propagate 'extends' through more levels -- but I believe it's structurally correct and good enough until we actually document classes.

**Inheritance** Diviner currently has a very weak story on inheritance. I want to inherit a lot more metas/docs. If an interface documents a method, we should just pull that documentation in to every implementation by default (implementations can still override it if they want). It can be shown in grey or something, but it should be desirable and correct to omit documentation of a method implementation when you are implementing a parent. Similarly, I want to pull in inherited methods and @tasks and such. This diff sets up for that, by formalizing "extends" relationships between atoms.

**Overspecialization** Diviner currently specializes atoms (FileAtom, FunctionAtom, ClassAtom, etc.). This is pretty much not useful, because Atomizers (which produce the atoms) need to be highly specialized, and Renderers/Publishers (which consume the atoms) also need to be highly specialized. Nothing interesting actually lives in the atom specializations, and we don't benefit from having them -- it just costs us generality in storage/caches for them. In the new code, I've used a single Atom class to represent any type of atom.

**URIs** We have fairly hideous URIs right now, which are very cumbersome  For in-app doc links, I want to provide nice URIs ("/h/notfications" or similar) which are stable redirects, and probably add remarkup for it: !{notifications} or similar. This diff isn't related to that since it's too premature.

**Search** Once we have a database generation target, we can index the documentation.

**Design** Chad has some nice mocks.

Test Plan: Ran `bin/diviner generate`, `bin/diviner generate --clean`. Saw appropriate graph propagation after edits. This diff doesn't do anything very useful yet.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T988

Differential Revision: https://secure.phabricator.com/D4340
2013-01-07 14:04:23 -08:00
Mailson Menezes
712e22208c Store width and height metadata of image files
Summary: Also provide a way to update old files metadata.

Test Plan: Create a revision which includes a image file. Check whether the widht, height metadata exists. Run `scripts/files/manage_files.php metadata --all` to update previously uploaded files.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2101

Differential Revision: https://secure.phabricator.com/D4347
2013-01-07 09:46:43 -08:00
Bob Trahan
3448781c40 de-duplicate emails received by phabricator multiple times
Summary: this can happen if you have Phabricator and email lists co-mingling such that Phabricator receives an email multiple times. we can prevent this from then spamming everyone or otherwise taking the action multiple times by storing a message id hash and dropping the message if we have more than one message that matches.

Test Plan: simulated sending the same email multiple times on the command line. noted only the first one made it through.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1726

Differential Revision: https://secure.phabricator.com/D4328
2013-01-03 17:04:30 -08:00
Ricky Elrod
acadf51efe Increment year.
Summary:
- Change copyright year.
- Remove 2012 Calendar entries.
- Add 2016 Calendar entries.

Test Plan: Not majorly applicable here.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4335
2013-01-03 05:45:08 -08:00
epriestley
923dc42e1a Move all CSS "z-index" rules to "z-index.css", fix Differential z-index problem
Summary: We have enough z-index rules that they're fairly hard to visualize with "git grep". Consolidate them. Then fix T2253 (missing z-index on left menu background).

Test Plan: Made a Differential window really narrow, then scrolled it horizontally.

Reviewers: btrahan, chad, ender

Reviewed By: chad

CC: aran

Maniphest Tasks: T2253

Differential Revision: https://secure.phabricator.com/D4302
2012-12-30 09:30:21 -08:00
epriestley
ba489f9d85 Add a local configuration source and a non-environmental ENV config source
Summary:
See discussion in T2221. Before we can move configuration to the database, we have a bootstrapping problem: we need database credentials to live //somewhere// if we can't guess them (and we can only really guess localhost / root / no password).

Some options for this are:

  - Have them live in ENV variables.
    - These are often somewhat unfamiliar to users.
    - Scripts would become a huge pain -- you'd have to dump a bunch of stuff into ENV.
    - Some environments have limited ability to set ENV vars.
    - SSH is also a pain.
  - Have them live in a normal config file.
    - This probably isn't really too awful, but:
    - Since we deploy/upgrade with git, we can't currently let them edit a file which already exists, or their working copy will become dirty.
    - So they have to copy or create a file, then edit it.
    - The biggest issue I have with this is that it will be difficult to give specific, easily-followed directions from Setup. The instructions need to be like "Copy template.conf.php to real.conf.php, then edit these keys: x, y, z". This isn't as easy to follow as "run script Y".
  - Have them live in an abnormal config file with script access (this diff).
    - I think this is a little better than a normal config file, because we can tell users 'run phabricator/bin/config set mysql.user phabricator' and such, which is easier to follow than editing a config file.

I think this is only a marginal improvement over a normal config file and am open to arguments against this approach, but I think it will be a little easier for users to deal with than a normal config file. In most cases they should only need to store three values in this file -- db user/host/pass -- since once we have those we can bootstrap everything else. Normal config files also aren't going away for more advanced users, we're just offering a simple alternative for most users.

This also adds an ENVIRONMENT file as an alternative to PHABRICATOR_ENV. This is just a simple way to specify the environment if you don't have convenient access to env vars.

Test Plan: Ran `config set x y`, verified writes. Wrote to ENVIRONMENT, ran `PHABRICATOR_ENV= ./bin/repository`.

Reviewers: btrahan, vrana, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2221

Differential Revision: https://secure.phabricator.com/D4294
2012-12-30 06:16:15 -08:00
epriestley
9e6d59829c Consolidate environmental initialization
Summary:
We have a bunch of code duplication now between __init_script__.php and webroot/index.php. Consoldiate these methods and move them into PhabricatorEnv.

Merge PhabricatorRequestOverseer into PhabricatorStartup.

Test Plan: Loaded page, ran script. Wiped PHABRICATOR_ENV; loaded page, ran script; got errors.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2223

Differential Revision: https://secure.phabricator.com/D4283
2012-12-25 06:15:28 -08:00
epriestley
f6b1964740 Improve Search architecture
Summary:
The search indexing API has several problems right now:

  - Always runs in-process.
    - It would be nice to push this into the task queue for performance. However, the API currently passses an object all the way through (and some indexers depend on preloaded object attributes), so it can't be dumped into the task queue at any stage since we can't serialize it.
    - Being able to use the task queue will also make rebuilding indexes faster.
    - Instead, make the API phid-oriented.
  - No uniform indexing API.
    - Each "Editor" currently calls SomeCustomIndexer::indexThing(). This won't work with AbstractTransactions. The API is also just weird.
    - Instead, provide a uniform API.
  - No uniform CLI.
    - We have `scripts/search/reindex_everything.php`, but it doesn't actually index everything. Each new document type needs to be separately added to it, leading to stuff like D3839. Third-party applications can't provide indexers.
    - Instead, let indexers expose documents for indexing.
  - Not application-oriented.
    - All the indexers live in search/ right now, which isn't the right organization in an application-orietned view of the world.
    - Instead, move indexers to applications and load them with SymbolLoader.

Test Plan:
  - `bin/search index`
    - Indexed one revision, one task.
    - Indexed `--type TASK`, `--type DREV`, etc., for all types.
    - Indexed `--all`.
  - Added the word "saboteur" to a revision, task, wiki page, and question and then searched for it.
    - Creating users is a pain; searched for a user after indexing.
    - Creating commits is a pain; searched for a commit after indexing.
    - Mocks aren't currently loadable in the result view, so their indexing is moot.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: 20after4, aran

Maniphest Tasks: T1991, T2104

Differential Revision: https://secure.phabricator.com/D4261
2012-12-21 14:21:31 -08:00
epriestley
6dd0169873 Fix various issues with SSH receivers
Summary:
  - Original command is in SSH_ORIGINAL_COMMAND, not normal argv.
  - Use PhutilShellLexer to parse it.
  - Fix a protocol encoding issue with ConduitSSHWorkflow. I think I'm going to make this protocol accept multiple commands anyway because SSH pipes are crazy expensive to build (even locally, they're ~300ms).

Test Plan: With other changes, successfully executed "arc list --conduit-uri=ssh://localhost:2222".

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T550

Differential Revision: https://secure.phabricator.com/D4232
2012-12-19 11:11:32 -08:00
epriestley
e78898970a Implement SSHD glue and Conduit SSH endpoint
Summary:
  - Build "sshd-auth" (for authentication) and "sshd-exec" (for command execution) binaries. These are callable by "sshd-vcs", located [[https://github.com/epriestley/sshd-vcs | in my account on GitHub]]. They are based on precursors [[https://github.com/epriestley/sshd-vcs-glue | here on GitHub]] which I deployed for TenXer about a year ago, so I have some confidence they at least basically work.
    - The problem this solves is that normally every user would need an account on a machine to connect to it, and/or their public keys would all need to be listed in `~/.authorized_keys`. This is a big pain in most installs. Software like Gitosis/Gitolite solve this problem by giving you an easy way to add public keys to `~/.authorized_keys`, but this is pretty gross.
    - Roughly, instead of looking in `~/.authorized_keys` when a user connects, the patched sshd instead runs `echo <public key> | sshd-auth`. The `sshd-auth` script looks up the public key and authorizes the matching user, if they exist. It also forces sshd to run `sshd-exec` instead of a normal shell.
    - `sshd-exec` receives the authenticated user and any command which was passed to ssh (like `git receive-pack`) and can route them appropriately.
    - Overall, this permits a single account to be set up on a server which all Phabricator users can connect to without any extra work, and which can safely execute commands and apply appropriate permissions, and disable users when they are disabled in Phabricator and all that stuff.
  - Build out "sshd-exec" to do more thorough checks and setup, and delegate command execution to Workflows (they now exist, and did not when I originally built this stuff).
  - Convert @btrahan's conduit API script into a workflow and slightly simplify it (ConduitCall did not exist at the time it was written).

The next steps here on the Repository side are to implement Workflows for Git, SVN and HG wire protocols. These will mostly just proxy the protocols, but also need to enforce permissions. So the approach will basically be:

  - Implement workflows for stuff like `git receive-pack`.
  - These workflows will implement enough of the underlying protocol to determine what resource the user is trying to access, and whether they want to read or write it.
  - They'll then do a permissons check, and kick the user out if they don't have permission to do whatever they are trying to do.
  - If the user does have permission, we just proxy the rest of the transaction.

Next steps on the Conduit side are more simple:

  - Make ConduitClient understand "ssh://" URLs.

Test Plan: Ran `sshd-exec --phabricator-ssh-user epriestley conduit differential.query`, etc. This will get a more comprehensive test once I set up sshd-vcs.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603, T550

Differential Revision: https://secure.phabricator.com/D4229
2012-12-19 11:08:07 -08:00
epriestley
adfe84ffce Add HarbormasterRunnerWorker, for running CI tests
Summary:
This is very preliminary and doesn't actually do anything useful. In theory, it uses Drydock to check out a working copy and run tests. In practice, it's not actually capable of running any of our tests (because of complicated interdependency stuff), but does check out a working copy and //try// to run tests there.

Adds various sorts of utility methods to various things as well.

Test Plan: Ran `reparse.php --harbormaster --trace <commit>`, observed attempt to run tests via Drydock.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015, T1049

Differential Revision: https://secure.phabricator.com/D4215
2012-12-17 13:43:26 -08:00
epriestley
221562b294 Modernize file uploads
Summary:
Modernizes file uploads. In particular:

  - Adds a mobile menu, with an "Upload File" item.
  - Adds crumbs to the list view, detail view and upload view.
  - Adds "Upload File" action to crumbs.
  - Moves upload file to a separate page.
  - Removes the combined upload file + recent files page.
  - Makes upload file use a normal file control by default (works on mobile).
  - Home page, file list and file upload page are now global drop targets which accept files dropped anywhere on them. Dragging a file into the window shows a mask and an instructional message.
    - User education on this is a little weak but I think that's a big can of worms?
  - Fixes a bug where dropping multiple files into a Remarkup text area produced bad results (resolves T2190).

T879 is related, although it's specifically about Maniphest. I've declined to make global drop targets yet there because there are multiple drop targets on the page with different meanings. That UI needs updating in general.

@chad, do we have an "upload" icon (counterpart to "download")?

Test Plan: Uploaded files in Maniphest, Differential, Files, and from Home. Dragged and dropped multiple files into Differential. Used crumbs, mobile.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2190

Differential Revision: https://secure.phabricator.com/D4200
2012-12-16 16:34:01 -08:00
epriestley
e4bb9255be Allow leases to be explicitly released via web or CLI
Summary: Permit the forcible release of Drydock leases. The implementation isn't very exciting for now.

Test Plan: Released leases via web and CLI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D4181
2012-12-14 15:42:58 -08:00
epriestley
6fc5208433 Minor, update package definitions to include all the new sprite CSS. 2012-12-07 13:54:12 -08:00
epriestley
f306cab653 Use application icons for "Eye" menu and Crumbs
Summary:
Issues here:

  - Need an application-sized "eye", or a "home" icon for "Phabricator Home".
  - Some of the "apps_lb_2x" sliced images are the "_dark_" versions, not the light versions.
  - If you slice an application-sized "logout" (power off) icon and application-sized "help" (questionmark in circle) icon I can replace the current menu icons and nearly get rid of "autosprite".
  - To replace the icons on /applications/, the non-retina size is "4x", so we'd need "8x" for retina. Alternatively I can reduce the icon sizes by 50%.
  - The "Help", "Settings" and "Logout" items currently have a "glowing" hover state, which needs a variant (or we can drop it).
  - The /applications/ icons have a white hover state (or we can drop it).
  - The 1x application (14x14) icons aren't used anywhere right now, should they be? Maybe in the feed in the future, etc?
  - The "apps-2x" and "apps-large" sheets are the same image, but getting them to actually use the same file is a bit tricky, so I just left them separate for now.

Test Plan:
{F26698}
{F26699}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4108
2012-12-07 13:37:28 -08:00
epriestley
8cff6ea9cb Add eye icon to left menu button
Summary: Do we have an icon with 2x for the right menu?

Test Plan: {F26590}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4101
2012-12-07 13:36:35 -08:00
epriestley
1c9d1d6ad1 Add some textures/gradients to crumbs and menu
Summary:
This doesn't lay in everything, but:

  - Break the buttons gradient apart into components and rebuild it (along with other gradients) into a single gradient sprite (possible after {D4099}).
  - Use the sliced gradient for the crumbs background.
  - Use the sliced image for the crumb divider.
  - Adds the black/white app sheets, but I'm not generating them quite yet.

Test Plan: {F26537} {F26540}

Reviewers: chad, btrahan

Reviewed By: chad

CC: aran

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4100
2012-12-07 13:35:49 -08:00
epriestley
1c9a6be979 Add a breadcrumbs element
Summary:
Add a basic breadcrumbs element, and implement it in Paste.

This needs some polish but is most of the way there.

Test Plan:
{F26443}
{F26444}
{F26445}

(This element is not visible on devices.)

Reviewers: chad

Reviewed By: chad

CC: aran, btrahan

Maniphest Tasks: T1960

Differential Revision: https://secure.phabricator.com/D4087
2012-12-07 13:35:17 -08:00
vrana
2931cbcb55 Exit instead of throw from reparse.php without commits
Summary: I need to run this in `xargs`.

Test Plan:
  $ echo 'E' | xargs -n 1 ./reparse.php --message --all

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4094
2012-12-06 11:16:15 -08:00
epriestley
71c170661b Use "flame" notification icon and bubble instead of lightning bolt
Summary: Switch to the final versions of these

Test Plan: Will add screenshots...

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4032
2012-11-27 14:03:25 -08:00
Ricky Elrod
755e8eb7d6 Fix the RHEL install script, mostly for RHEL 5.
Summary:
A bunch of various fixes for the RHEL install script.
Most of them are stylistic, one of them fixes the EPEL repo release
RPM URL for RHEL 5. (enough acronyms there?)

Test Plan:
Tried installing on Fedora (which is treated as RHEL 6 due to how we handle
being unable to find version in the script).

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4038
2012-11-27 12:52:37 -08:00
epriestley
5cbc31644b Add a "close" action to Drydock resources
Summary: This does nothing fancy, just closes the resource and releases/breaks leases. They'll get cleaned up in some to-be-written GC process.

Test Plan: Closed resources from web UI and CLI.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3998
2012-11-27 12:48:03 -08:00
epriestley
6c88c76cac Generate an icon sprite map with new sprites and 2x retina graphics
Summary:
  - The filesystem is now the authority for which sprites are available. If you add new icons, the generation process will pick them up.
  - I broke out icon generation and added retina support. App icon generation still uses the old method.
  - Update ActionList and RemarkupControl to use the new sheet.
  - Use white icons on hover.
  - Also fixed a couple of minor issues with some stuff in Firefox/Chrome.

Test Plan:
{F25750}

{F25751}

{F25752}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2013

Differential Revision: https://secure.phabricator.com/D4027
2012-11-23 16:35:39 -08:00
epriestley
f8e6240b9d Move sprite sources from webroot/ to resources/
Summary:
  - Since we'll never serve these directly, move them to resources/. This makes generating the Celerity map faster and reduces the size of the result map, since we don't need to analyze resources we'll never serve.
  - Also Rename the 2x `subscribe-remove` to `subscribe-delete` since they were named inconsistently. Everything else is in good shape.

Test Plan: Generated sprites as per D4025

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2013

Differential Revision: https://secure.phabricator.com/D4026
2012-11-23 16:19:06 -08:00
epriestley
2dcc4cfb53 Update static resource packages
Summary: These are a bit out of date; bundle things together better.

Test Plan: Viewed page source for home, differential list, differential diff, maniphest list, maniphest task. Verified reasonable resource packaging.

Reviewers: chad, vrana, btrahan

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4002
2012-11-20 18:01:25 -08:00
vrana
ef8c43ac2a Simplify and optimize save_lint.php
Test Plan: Ran it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2038

Differential Revision: https://secure.phabricator.com/D3933
2012-11-16 15:58:09 -08:00
vrana
23a046b3cd Allow saving lint errors to database
Summary: This saves lint errors to the path change of current commit. It requires pushed revision. It doesn't save difference from previous commit mentioned in T2038#comment-4 - I don't plan doing it after all, everything would be much more complicated and the amount of data saved with this approach isn't that bad.

Test Plan: Applied patch, ran script, verified DB.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2038

Differential Revision: https://secure.phabricator.com/D3899
2012-11-08 15:39:43 -08:00
vrana
ef85f49adc Delete license headers from files
Summary:
This commit doesn't change license of any file. It just makes the license implicit (inherited from LICENSE file in the root directory).

We are removing the headers for these reasons:

- It wastes space in editors, less code is visible in editor upon opening a file.
- It brings noise to diff of the first change of any file every year.
- It confuses Git file copy detection when creating small files.
- We don't have an explicit license header in other files (JS, CSS, images, documentation).
- Using license header in every file is not obligatory: http://www.apache.org/dev/apply-license.html#new.

This change is approved by Alma Chao (Lead Open Source and IP Counsel at Facebook).

Test Plan: Verified that the license survived only in LICENSE file and that it didn't modify externals.

Reviewers: epriestley, davidrecordon

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2035

Differential Revision: https://secure.phabricator.com/D3886
2012-11-05 11:16:51 -08:00
epriestley
07dc943215 Modernize the drydock script
Summary: Add a bin/drydock symlink and break it into workflows. Nothing too special here.

Test Plan: Ran `bin/drydock wait-for-lease`, `bin/drydock lease`, `bin/drydock help`, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3867
2012-11-01 15:30:14 -07:00
epriestley
f0fdcf1a51 Undumb the Drydock resource allocator pipeline
Summary:
This was the major goal of D3859/D3855, and to a lesser degree D3854/D3852.

As Drydock is allocating a resource, it may need to allocate other resources first. For example, if it's allocating a working copy, it may need to allocate a host first.

Currently, we have the process basically queue up the allocation (insert a task into the queue) and sleep() until it finishes. This is problematic for a bunch of reasons, but the major one is that if allocation takes more resources (host, port, machine, DNS) than you have daemons, they could all end up sleeping and waiting for some other daemon to do their work. This is really stupid. Even if you only take up some of them, you're spending slots sleeping when you could be doing useful work.

To partially get around this and make the CLI experience less dumb, there's this goofy `synchronous` flag that gets passed around everywhere and pushes the workflow through a pile of special cases. Basically the `synchronous` flag causes us to do everything in-process. But this is dumb too because we'd rather do things in parallel if we can, and we have to have a lot of special case code to make it work at all.

Get rid of all of this. Instead of sleep()ing, try to work on the tasks that need to be worked on. If another daemon grabbed them already that's fine, but in the worst case we just gracefully degrade and do everything in process. So we get the best of both worlds: if we have parallelizable tasks and free daemons, things will execute in parallel. If we have nonparallelizable tasks or no free daemons, things will execute in process.

Test Plan: Ran `drydock_control.php --trace` and saw it perform cascading allocations without sleeping or special casing.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3861
2012-11-01 11:30:42 -07:00
epriestley
5903ed650c Move completed tasks to an "archive" table and delete them in the GC
Summary:
Currently, when taskmasters complete a task it is immediately deleted. This prevents us from doing some general things, like:

  - Supporting the idea of permanent failure (e.g., after N failures just stop trying).
  - Showing the user how fast taskmasters are completing tasks.
  - Showing the user how long tasks took to complete.

Having better visibility into this is important to Drydock, which builds on the task system. Also, generally buff debug output for task execution.

Test Plan: Ran `bin/phd debug taskmaster`. Ran `bin/phd debug garbage`. Queued some tasks via various systems.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D3852
2012-10-31 15:22:16 -07:00
epriestley
a7da4fad88 Add Drydock Application
Summary: Add an Application class for Drydock and move routing rules there.

Test Plan: Looked at /applications/, clicked around drydock.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3847
2012-10-31 09:57:57 -07:00
Bob Trahan
731a6900bd upgrade repository delete function to full-blown workflow
Summary: fancy title. really just make the delete() method aware of related objects and build a quick workflow which calls delete(). also make commit delete savvy about audit requests.

Test Plan: deleted a repository per the instructions given to me in the web UI

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1416, T1958, T1372

Differential Revision: https://secure.phabricator.com/D3822
2012-10-25 16:23:41 -07:00
epriestley
5d1bd51627 Add a script to migrate files between storage engines
Summary: Quora requested this (moving to S3) but it's also clearly a good idea.

Test Plan:
Ran with various valid/invalid options to test options. Error/sanity checking seemed OK.

Migrated individual local files.

Migrated all my local files back and forth between engines several times.

Uploaded some new files.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1950

Differential Revision: https://secure.phabricator.com/D3808
2012-10-25 11:36:38 -07:00
epriestley
70dc3f5004 Show all available action list icons in UIExamples
Summary:
Make the example page a little more useful by showing available icons.

Also replace the "new" image, it had a little arrow which I thought was a "+". Use the one with a "+".

Test Plan: {F21966}

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3794
2012-10-23 12:00:28 -07:00
epriestley
96b5d0e74a Generate Releeph GLYPHICON
Summary: Generate a gunsights stylesheet entry for use in Releeph.

Test Plan: None!

Reviewers: edward

Reviewed By: edward

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3773
2012-10-21 15:56:17 -07:00
epriestley
2f64a01579 Add table icon to remarkup spritesheet
Summary: See D3730.

Test Plan: Looked at sprite.

Reviewers: vrana

Reviewed By: vrana

CC: chad, aran

Differential Revision: https://secure.phabricator.com/D3731
2012-10-17 18:10:38 -07:00
epriestley
1a63938b33 Further improve various Phame UI things
Summary:
  - Better icons and action order.
  - "Move Post" action.
  - (Bugfix) Allow multiple blogs to be set to not having custom domains.
  - Make "Write Post" skip the "select a blog" step when coming from a blog view.
  - Sort blog list on "Write Post".
  - Show messages when a post is a draft or not on a blog.

Test Plan: Created posts, blogs, moved posts, preview/live'd posts, etc.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3708
2012-10-16 09:44:43 -07:00
epriestley
b072e937b5 Modernize blog UI
Summary:
Cleans up some of the mess I made in D3694. Basically:

  - All blogs have an "internal" view with posts that uses mobile-friendly UIs, etc., so we don't have to do as much work with skins -- they just have to look pretty.
  - Blogs now have a separate "live" view that we use to handle domains / skins.
  - Simplified some views and use IDs in some URLs for consistency.
  - Delete a bunch of edge/blogger/multi-blog code that's now obsolete.

Test Plan: Will attach screenshots.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1373

Differential Revision: https://secure.phabricator.com/D3695
2012-10-15 14:50:12 -07:00
epriestley
643d8070f1 Implement improved remarkup assistance panel
Summary:
  - I made a "?" icon for help/reference.
  - The `<>` icon was slightly too wide so I carved it down to 14x14.
  - All the icons are in `/Phabriactor/remarkup_icon_sources.psd` if you want to tweak anything.
  - Tooltips don't look like the mock but I'll tackle those separately.
  - Removed strikethrough.
  - Removed tag/image/text size for now since they don't have reasonable JS implementations yet.
  - I think everything else is accurate to the mock.

Test Plan:
Normal state:

{F20621, size=full}

Hover + Click states:

{F20622, size=full}

Clicked state:

{F20620, size=full}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1848

Differential Revision: https://secure.phabricator.com/D3650
2012-10-06 16:21:25 -07:00
epriestley
a7cdc312e6 Use new application icons and intended side nav icons
Summary: @chad, can you do the icon sheets based on 1.6? We're using a few icons not present in 1.5. I put the 1.6 "pro" source on Dropbox.

Test Plan:
Nav hover and selected states:

{F20598}

Launch hover state:

{F20596}

Reviewers: chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T1856

Differential Revision: https://secure.phabricator.com/D3649
2012-10-06 14:48:46 -07:00
lesha
26d62ee450 [Ubuntu Install Script] Automatically enable mod_rewrite
Summary: I needed to figure out how to do this, and it took a couple of minutes.

Test Plan: ran that command on my Ubuntu 12.04 system

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3479
2012-10-05 15:54:23 -07:00
epriestley
5b74b8b765 Add basic "Subscriptions" application
Summary:
Basic infrastructure for generalizing subscriptions/CCs for T1808, T1514 and T1663.

  - Implement `PhabricatorSubscribableInterface` and you'll get a subscribe/unsubscribe button for free.
  - If there are any auto-subscribed users (like the question author) you can specify them; this makes more sense for Tasks and Revisions than Ponder probably, but maybe the author should be auto-subscribed.
  - Subscriptions are either "explicit" (the user clicked 'subscribe') or "implicit" (the user did something which causes them to become subscribed naturally). If a user unsubscribes, they'll no longer be added by implicit subscriptions. This may or may not be relevant to Ponder but is an existing Herald feature in Differential.
  - Helper method on PhabricatorSubscribersQuery to load subscribers.
  - This doesn't handle actually sending email, etc. I think that's all so application-specific that it doesn't belong here.
  - Now seems to work.

Test Plan:
{F20552}
{F20553}

Reviewers: pieter, btrahan

Reviewed By: pieter

CC: aran

Maniphest Tasks: T1663, T1514, T1808

Differential Revision: https://secure.phabricator.com/D3637
2012-10-05 13:18:05 -07:00
Bob Trahan
3211238a95 Fix a bug found by vrana in audit
Summary: seen https://secure.phabricator.com/rPf5e71eb88920d4afb5590b0682eda66f09b2c024

Test Plan: i believe in vrana.

Reviewers: vrana

Reviewed By: vrana

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D3588
2012-10-01 15:15:07 -07:00
epriestley
92ff9c092b Move "Macros" to a first-class application
Summary:
This is mostly to unblock D3547.

  - Move "Macros" to a first-class application called "Macros".
  - After D3547, this application will also house "Memes" (macros with text on them).
  - This will also make them easier to find; the top navigational query I field is "where are image macros?" nowadays, since it's not intuitive they're part of files.
  - This makes some of the UI mobile-aware but doesn't set the `device` flag yet, since there are still some missing pieces.
  - I'll separate storage out and continue modernizing the UI as we unblock and integrate D3547.

Test Plan: Created, edited and deleted macros. Viewed files.

Reviewers: btrahan, vrana, teisenbe

Reviewed By: vrana

CC: aran

Maniphest Tasks: T175

Differential Revision: https://secure.phabricator.com/D3572
2012-10-01 14:04:03 -07:00
epriestley
a6b6a9766b Make star application icon available for Phame
Summary: See D3579.

Test Plan: Looked at sprite map, saw open-star.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3583
2012-10-01 13:53:16 -07:00
vrana
5f59532d6c Mark autosprite.css as generated
Test Plan: I wish I could tell that I ran it but I don't have the PNG files.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3582
2012-10-01 12:59:33 -07:00
epriestley
f817d81b12 Add applications for Owners, Repositories, PHID manager, PHPAST
Summary: See D3572.

Test Plan: Viewed /applications/, saw all these applications.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3579
2012-10-01 12:56:33 -07:00
Nick Harper
d591007d27 Add option to symbol import script to ignore errors in input
Summary:
I was running the generate_ctags_symbols script on a project, and some
combination of this script and ctags was generating invalid rows for this
script. It's nice if this script can ignore invalid input instead of having
to add another step to filter out invalid input.

Test Plan: run script with input that has some invalid lines

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3532
2012-09-27 15:36:38 -07:00
Eugene Yunak
f5e71eb889 Aphlict should respect the PATH set in the config.
Reviewed by: epriestley
2012-09-27 14:17:20 -07:00
vrana
8e7ae7b33a Optimize reindex_everything.php
Summary:
We have two troubles with this script:

# Our revisions and commits don't fit in the memory. (Our tasks do :-).)
# Reindexing revisions is slow.

Test Plan: Ran it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3483
2012-09-17 13:34:41 -07:00
vrana
a430d0f9df Use 'w' connection in bin/storage
Test Plan:
Built a DB provider using readonly connection for 'r' mode. Then:

  $ arc unit

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3484
2012-09-13 22:36:35 -07:00
epriestley
303ad93996 Modernize UIExamples
Summary:
  - Get rid of an AphrontSideNavView callsite.
  - Modernize and simplify the application implementation.
  - Doesn't work perfectly on tablet/phone but that's because not all the UI examples work there yet.

Test Plan: Looked at /applications/ and /uiexample/.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3431
2012-09-11 09:56:40 -07:00
Bob Trahan
cc0b74b01a bin/accountadmin - allow creation of system accounts and create workflow for system accounts that are in trouble
Summary: the former is self explanatory. the latter is necessary for installations that require email verification. since many system agents are given bogus email address there can become a problem where these accounts can't be verified

Test Plan: created system agent account from scratch. edited user and toggled system agent accountness. created system agent with unverified email address and verified it.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1656

Differential Revision: https://secure.phabricator.com/D3401
2012-08-29 11:07:29 -07:00
vrana
a44de4f76c Link interfaces as classes
Summary:
We link classes from `extends` and `instanceof` (which can be both interface and class) and from `implements` (which is always interface).

We don't need to split these two because they share the same namespace.
If we decide to then we will need to:

- Fix [[ https://secure.phabricator.com/diffusion/P/browse/master/src/applications/diffusion/controller/DiffusionSymbolController.php;66a300768a62bbc9$79 | DiffusionSymbolController ]]
- Fix [[ https://secure.phabricator.com/diffusion/P/browse/master/webroot/rsrc/js/application/repository/repository-crossreference.js;66a300768a62bbc9$24 | repository-crossreference ]]
- Allow more types in `DiffusionSymbolController`.

Test Plan: Ran the script on secure.phabricator.com, clicked on `ArcanistLintRenderer` symbol.

Reviewers: epriestley, alanh

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3382
2012-08-27 12:10:28 -07:00
epriestley
85bf88e400 Allow pastes to be flagged
Summary:
This does a few things:

  - Allows you to flag pastes. This is straightforward.
  - Allows Applications to register event listeners.
  - Makes object action lists emit a 'didrenderactions' event, so other applications can add more actions. The Flags application injects its action in this way. This should generally make it much easier to add actions to objects when we add new applications, with less code duplication and better modularity. We have a really hacky version of this in Differential that I want to get rid of in lieu of this more general approach. I'm going to make object lists do the same thing, so any application can jump in and add stuff.

Test Plan: Flagged and unflagged pastes. Viewed home page, differential, flags list.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D3377
2012-08-24 13:19:47 -07:00
epriestley
36e71a0601 Allow pastes to be edited
Summary: Permits the name and langauge of a paste to be edited. This will eventually allow the visibility policy to be edited as well.

Test Plan: Edited name/langauge of some pastes. Tried to edit a paste I didn't own, was harshly rebuffed.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1690

Differential Revision: https://secure.phabricator.com/D3376
2012-08-24 13:19:30 -07:00
epriestley
d9f241a9ca Fix some argument parsing stuff in reparse.php
Summary: Currently, "reparse.php --message rXnnn" fails because $reparse_what is an array. Allow multiple named commits.

Test Plan:
  $ ./scripts/repository/reparse.php --message rP9030fe8b0904b5291c69a22b0570a10013bba4b2 rP81946fc08d8a737b278255090e296ca92164d672
  Running 'PhabricatorRepositoryGitCommitMessageParserWorker'...
  Running 'PhabricatorRepositoryGitCommitMessageParserWorker'...

  Done.

Reviewers: nh, vrana, btrahan

Reviewed By: nh

CC: aran

Differential Revision: https://secure.phabricator.com/D3362
2012-08-22 14:03:04 -07:00
epriestley
93b0a501a4 Add local navigation to Differential
Summary:
Adds a flexible navigation menu to diffs that shows you your current position in the diff.

Anticipating some "this is the best thing ever" and some "this is the wosrt thing ever" on this, but let's see how much pushback we get? It seems pretty good to me.

Test Plan: Will attach screenshots.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1633, T1591

Differential Revision: https://secure.phabricator.com/D3355
2012-08-21 15:01:20 -07:00
vrana
b50cdc6e43 Highlight update time in revision list
Summary:
This is another experiment for reducing reviewers response time.
I stole the idea (and colors) from [[ http://www.reviewboard.org/media/screenshots/2009/02/02/dashboard.png | ReviewBoard ]].
I actually quite like it (except when everything is red) and I can image that people will review just to have better color balance.

The code is not production ready for these reasons:

- We load holidays again and again for each revision. I couldn't cache it to static variable because it could persist multiple requests, right?
- I don't know how to expand height to the whole cell (I'm really bad in CSS).
- CSS rules are probably in wrong file.
- We probably want to use different colors.

This is how it looks:
{F16406}

Test Plan: Displayed revision list.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3190
2012-08-20 17:59:13 -07:00
epriestley
bee112c575 Fix various app nav issues
Summary:
  - Fix width, corresponding to wider sprites.
  - Sprite the "Audit" icon.
  - Mark the meta-application as device-ready.
  - Fix some collapse/expand bugs with the draggable local navs.
  - Add texture to local nav.
  - Darken the application nav to make it more cohesive with the main nav. I think this is an improvement?

Test Plan: See screenshots.

Reviewers: chad, btrahan

Reviewed By: btrahan

CC: aran, netfoxcity

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3338
2012-08-20 14:13:15 -07:00
Alan Huang
470057c1a3 Support limiting symbol deletion to certain paths
Summary:
For Nefarious Facebook Internal Purposes, which may or may not
include incremental symbol database updates.

Give the import script an option to not clear all symbols from the
project, and make a new script that clears only symbols from paths given
on stdin.

(I'm not yet sure how much of the NFIPs is going to be ported over
here. So there might be a future diff that uses this. Conversely,
since there's no real use case out here, I'm fine just moving this to
the Facebook configuration if necessary.)

Test Plan: Run scripts in innumerable bizarre and eldritch configurations.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3305
2012-08-16 08:15:17 -07:00
epriestley
70a8e8b6e8 Modernize Paste
Summary:
  - Add a PhabricatorApplication.
  - Make most of the views work well on tablets / phones. The actual "Create" form doesn't, but everything else is good -- need to make device-friendly form layouts before I can do the form.

Test Plan: Will attach screenshots.

Reviewers: btrahan, chad, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3293
2012-08-15 10:45:06 -07:00
epriestley
012370c6ab Use sprites for (nearly) all application icons
Summary: Sprites for everyone.

Test Plan: Loaded `/applications/`.

Reviewers: btrahan, chad, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3280
2012-08-14 14:23:55 -07:00
epriestley
47b96fdebe Use sprites for all the menu icons
Summary:
See D3277, D3278.

  - Sprite all the menu icons.
  - Delete the unsprited versions.
  - Notification bolt now uses the same style as everything else.

Test Plan: Looked at page, hovered, clicked things.

Reviewers: btrahan, chad, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3279
2012-08-14 14:20:01 -07:00
epriestley
27b86d5132 Add a sprite generation script to Phabricator
Summary: This probably needs some tweaks, but gets the point across. I'll lay in the actual usage in the next diff.

Test Plan: Looked at generated CSS/PNG.

Reviewers: btrahan, chad, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1569

Differential Revision: https://secure.phabricator.com/D3278
2012-08-14 14:17:19 -07:00
Nick Harper
ef0f30b662 Use PhutilArgumentParser in reparse.php
Summary:
Update how arguments get parsed in reparse.php and add two new options
--force-local and --min-date (both to be used with --all). This diff also
fixes a bug in destroy_revision.php

Test Plan: ran the script

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3275
2012-08-13 19:35:30 -07:00
Bob Trahan
aa2ef37e94 quick, ghetto script to rescale profile pictures
Summary: we had a hole in scaling pics from oauth from a bit. that was patched in D2848. this cleans up the old data.

Test Plan: ran it in production. issue as described on task is resolved. also spot checked some profiles and they look good.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1634

Differential Revision: https://secure.phabricator.com/D3268
2012-08-13 15:29:34 -07:00
epriestley
3e29921f5e Use phutil_utf8_convert() in Phabricator
Summary: See D3252.

Test Plan: This one is nasty to test, I'm going to make some coffee first.

Reviewers: davidreuss, vrana, btrahan

Reviewed By: davidreuss

CC: aran

Maniphest Tasks: T452

Differential Revision: https://secure.phabricator.com/D3254
2012-08-12 08:50:19 -07:00
vrana
b1b2afce95 Support applications in aphrontpath.php
Test Plan:
  $ ./aphrontpath.php /diffusion/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3233
2012-08-10 10:48:08 -07:00
vrana
80d9a276f3 Automatically resolve merge conflicts in Celerity map
Summary:
We can use `.gitattributes` instead but there's no way how to set repository config for all users in Git, right?
So provide a script writting to `.git/info/attributes` instead so that we don't have to .gitignore `.gitattributes`.

Test Plan:
  $ scripts/celerity/install_merge.sh
  $ git pull # with merge conflict in Celerity map

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3228
2012-08-09 15:59:18 -07:00
Alan Huang
fd3ba9841b Update symbol generation scripts
Summary:
Scripts now return scoped symbols -- in particular, PHP class constants, fields, and methods. ctags gives some for other languages.

(Turns out XHPAST doesn't support traits. But no one uses traits anyway so it's probably fine.)

I couldn't find a list of the context types ctags uses (class/struct/union/enum/maybe others?), so the context code just ignores that. Also, it uses a blacklist for the symbol type instead of a whitelist because there are a ton, they vary by language, and I didn't want to unintentionally exclude anything (P480).

Test Plan: Scrape symbols from arcanist and phabricator. Upload them to sandbox. Search for things.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3202
2012-08-08 17:06:05 -07:00
Alan Huang
a8d6af0f42 Infrastructure changes to support scoped symbols
Summary:
 - import_project_symbols supports an optional extra field, which is the
   context of the symbol.
 - Symbol query can take a symbol argument, either as a parameter or a
   URL component (so you can now jump nav to `s Zerg/rush`, for
   example).
 - Conduit method not yet updated. Will do that later.

NOTE: Not providing a context is distinct from providing an empty
context, because an empty context stands for top-level context, i.e.
functions and classes for PHP. It will not find class methods, etc. It
is possible that we should use some weird token that could not normally
be a context name to stand in for empty context.

Test Plan: Do a bunch of symbol searches.

Reviewers: epriestley

Reviewed By: epriestley

CC: nh, aran, Korvin

Maniphest Tasks: T1602

Differential Revision: https://secure.phabricator.com/D3148
2012-08-07 09:28:49 -07:00
Marcel Beck
99e9a26192 Separates the PID and log directories of daemons
Summary: The Log and PID directory should be separable in the config file

Test Plan: Start the daemons, and check if the pid and log files are stored in directories that were specified in the config file.

Reviewers: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3149
2012-08-05 08:35:24 +02:00
epriestley
f0af273165 Add FactCursors and application fact datasources
Summary:
  - Add PhabricatorApplication. This is a general class that I have grand designs for, but used here to allow applications to provide objects for analysis by the facts appliction.
  - Add FactCursors, to keep track of where iterators are.
  - Make the daemon do something sort of useful.
  - Add `bin/fact cursors` for showing and managing objects and cursors.
  - Add some options to `bin/fact analyze`.

Test Plan:
  - `bin/fact cursors`, `bin/fact cursors --reset DifferentialRevision`, `bin/fact cursors --reset X`
  - `bin/fact analyze`, `bin/fact analyze --all`, `bin/fact analyze --iterator DifferentialRevision --skip-aggregates`
  - `bin/phd debug fact`

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3098
2012-07-30 10:43:49 -07:00
epriestley
7c934e4176 Add a basic "fact" application
Summary:
Basic "Fact" application with some storage, part of a daemon, and a control binary.

= Goals =

The general idea is that we have various statistics we'd like to compute, like the frequency of image macros, reviewer responsiveness, task close rates, etc. Computing these on page load is expensive and messy. By building an ETL pipeline and running it in a daemon, we can precompute statistics and just pull them out of "stats" tables.

One way to do this is just to completely hard-code everything, e.g. have a daemon that runs every hour which issues a big-ass query and dumps results into a table per-fact or per fact-group. But this has a bunch of drawbacks: adding new stuff to the pipeline is a pain, various fact aggregators can't share much code, updates are slow and expensive, we can never build generic graphs on top of it, etc.

I'm hoping to build an ETL pipeline which is generic enough that we can use it for most things we're interested in without needing schema changes, and so that installs can use it also without needing schema changes, while still being specific enough that it's fast and we can build useful stuff on top of it. I'm not sure if this will actually work, but it would be cool if it does so I'm starting pretty generally and we'll see how far I get. I haven't built this exact sort of thing before so I might be way off.

I'm basing the whole thing on analyzing entire objects, not analyzing changes to objects. So each part of the pipeline is handed an object and told "analyze this", not handed a change. It pretty much deletes all the old data about that thing and then writes new data. I think this is simpler to implement and understand, and it protects us from all sorts of weird issues where we end up with some kind of garbage in the DB and have to wipe the whole thing.

= Facts =

The general idea is that we extract "facts" out of objects, and then the various view interfaces just report those facts. This change has on type of fact, a "raw fact", which is directly derived from an object. These facts are concerete and relate specifically to the object they are derived from. Some examples of such facts might be:

  D123 has 9 comments.
  D123 uses macro "psyduck" 15 times.
  D123 adds 35 lines.
  D123 has 5 files.
  D123 has 1 object.
  D123 has 1 object of type "DREV".
  D123 was created at epoch timestamp 89812351235.
  D123 was accepted by @alincoln at epoch timestamp 8397981839.

The fact storage looks like this:

  <factType, objectPHID, objectA, valueX, valueY, epoch>

Currently, we supprot one optional secondary key (like a user PHID or macro PHID), two optional integer values, and an optional timestamp. We might add more later. Each fact type can use these fields if it wants. Some facts use them, others don't. For instance, this diff adds a "N:*" fact, which is just the count of total objects in the system. These facts just look like:

  <"N:*", "PHID-xxxx-yyyy", ...>

...where all other fields are ignored. But some of the more complex facts might look like:

  <"DREV:accept", "PHID-DREV-xxxx", "PHID-USER-yyyy", ..., ..., nnnn> # User 'yyyy' accepted at epoch 'nnnn'.
  <"FILE:macro", "PHID-DREV-xxxx", "PHID-MACR-yyyy", 17, ..., ...> # Object 'xxxx' uses macro 'yyyy' 17 times.

Facts have no uniqueness constraints. For @vrana's reviewer responsiveness stuff, we can insert multiple rows for each reviewer, e.g.

  <"DREV:reviewed", "PHID-DREV-xxxx", "PHID-USER-yyyy", nnnn, ..., mmmm> # User 'yyyy' reviewed revision 'xxxx' after 'nnnn' seconds at 'mmmm'.

The second value (valueY) is mostly because we need it if we sample anything (valueX = observed value, valueY = sample rate) but there might be other uses. We might need to add "objectB" at some point too -- currently we can't represent a fact like "User X used macro Y on revision Z", so it would be impossible to compute macro use rates //for a specific user// based on this schema. I think we can start here though and see how far we get.

= Aggregated Facts =

These aren't implemented yet, but the idea is that we can then take the "raw facts" and compute derived/aggregated/rollup facts based on the raw fact table. For example, the "count" fact can be aggregated to arrive at a count of all objects in the system. This stuff will live in a separate table which does have uniqueness constraints, and come in the next diff.

We might need some kind of time series facts too, not sure about that. I think most of our use cases today are covered by raw facts + aggregated facts.

Test Plan: Ran `bin/fact` commands and verified they seemed to do reasonable things.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran, majak

Maniphest Tasks: T1562

Differential Revision: https://secure.phabricator.com/D3078
2012-07-27 13:34:21 -07:00
epriestley
27f6cc3b27 Support PhabricatorOpaqueEnvelope for managing database passwords
Summary: Currently, MySQL/MySQLi connections store passwords in plain text on the object. Allow them to be stored in PhutilOpaqueEnvelopes instead. See D3053.

Test Plan: Loaded site.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D3054
2012-07-24 11:13:53 -07:00
epriestley
dd70c59465 Use OpaqueEnvelopes for all passwords in Phabricator
Summary:
See D2991 / T1526. Two major changes here:

  - PHP just straight-up logs passwords on ldap_bind() failures. Suppress that with "@" and keep them out of DarkConsole by enabling discard mode.
  - Use PhutilOpaqueEnvelope whenever we send a password into a call stack.

Test Plan:
  - Created a new account.
  - Reset password.
  - Changed password.
  - Logged in with valid password.
  - Tried to login with bad password.
  - Changed password via accountadmin.
  - Hit various LDAP errors and made sure nothing appears in the logs.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2993
2012-07-17 12:06:33 -07:00
epriestley
5d8b75b4da Use the unified markup cache for Maniphest
Summary:
  - See D2945.
  - Drop `cache` field from ManiphestTransaction.
  - Render task descriptions and transactions through PhabricatorMarkupEngine.
  - Also pull the list of macros more lazily.

Test Plan:
  - Verified transactions and transaction preview work correctly and interact with cache correctly.
  - Verified tasks descriptions and task preview work correctly.
  - Verified we don't hit the imagemacro table when we're rendering everything from cache anymore.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D2946
2012-07-11 11:40:10 -07:00
epriestley
ddf67fce58 Add an example event listener, improve documentation, and add a commit discovery event
Summary: Improve documentation around Phabricator events.

Test Plan: Generated and read documentation. Ran test script.

Reviewers: btrahan, jungejason

Reviewed By: jungejason

CC: aran

Maniphest Tasks: T1092

Differential Revision: https://secure.phabricator.com/D2917
2012-07-03 16:46:27 -07:00
Jason Ge
61b79b5359 Use binary_safe_diff from arcanist
Summary:
binary_safe_diff is needed in arcanist too. Moved it over to
arcanist. See D2915.

Test Plan: diffusion page rendered correctly on binary file.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D2916
2012-07-03 13:51:37 -07:00
epriestley
da0da83431 Fix display of admin "new value" in accountadmin script
Summary: I broke this when converting to PhabricatorUserEditor.

Test Plan:
Ran `accountadmin`, created an admin account, verified the "new value" column showed "Y".

  ACCOUNT SUMMARY

                 OLD VALUE                        NEW VALUE
      Username                                    derp
     Real Name                                    derprp
         Email                                    derp@derp.com
      Password                                    Unchanged
         Admin   N                                Y

Reviewers: btrahan, nikil

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1434

Differential Revision: https://secure.phabricator.com/D2908
2012-07-02 15:22:27 -07:00
epriestley
13d96e6377 Introduce "bin/repository" for repository management
Summary:
Nothing new or exciting here yet, just moving the random scripts/repositories/ things to bin/repository. Also add `repository list`.

(Console stuff comes from D2841.)

Test Plan: Ran `repository list`, `repository pull`, `repository discover`, `repository discover --verbose`, `repository help`.

Reviewers: jungejason, vrana

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D2849
2012-06-25 12:35:37 -07:00
epriestley
a705f336a3 Add vebose logging to PhutilRepositoryPullDaemon
Summary: Add verbose logging. This logging is activated by setting "phd.verbose" in the config, running "phd debug", or explicitly in scripts/repository/pull.php and scripst/repository/discover.php

Test Plan:
  >>> orbital ~/devtools/phabricator $ ./scripts/repository/discover.php GTEST
  Discovering 'GTEST'...
  <VERB> PhabricatorRepositoryPullLocalDaemon Discovering commits in repository 'GTEST'...
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '()_+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch '_+abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'abcd$100', at a37bc285a12efa7224fe19f3df54cd90fa2b897a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch', at 774c7737b2d560a291697126bf4513204ccf661a.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch-1', at dc97539bee07293f95990d71f4638335a2531d69.
  <VERB> PhabricatorRepositoryPullLocalDaemon Skipping, HEAD is known.
  <VERB> PhabricatorRepositoryPullLocalDaemon Examining branch 'arcpatch-2', at 1acfaec313c46dd3caa90448800181fb91b0270f.

Reviewers: jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2843
2012-06-24 15:06:40 -07:00
vrana
77f44ded98 Initialize translator after loading custom libraries
Summary: Installs may set global translation which may not be available before loading libraries.

Test Plan:
  $ aphrontpath.php /

Reviewers: nh

Reviewed By: nh

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D2819
2012-06-21 15:57:51 -07:00
Bob Trahan
26ca5cbbe3 phd - on start command accurately detect if daemons are running before erroring out
Summary: loadRunningDaemons loads daemons that may be running (have a file pid). other commands handle this appropriately so just make sure the start command knows whats up

Test Plan: phd start, stop, reload all seemed to work just fine.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1284

Differential Revision: https://secure.phabricator.com/D2808
2012-06-20 12:36:31 -07:00
epriestley
d4b6b095cb Provide a script to completely destroy revisions
Summary:
Someone may or may not have accidentally uploaded secrets to Differential. Provide an administrative mechanism to permanently destroy a revision.

Also fix some of the transaction handling code.

Test Plan:
  $ ./scripts/differential/destroy_revision.php --trace D1
  >>> [0] <connect>
  <<< [0] <connect> 1,060 us
  >>> [1] <query> SELECT * FROM `differential_revision` WHERE `id` = 1
  <<< [1] <query> 473 us

      Really destroy 'D1: asdbas' forever? [y/N] y

  >>> [2] <connect>
  <<< [2] <connect> 628 us
  >>> [3] <query> START TRANSACTION
  <<< [3] <query> 190 us
  >>> [4] <query> SELECT * FROM `differential_diff` WHERE revisionID = 1
  <<< [4] <query> 510 us
  >>> [5] <query> SAVEPOINT Aphront_Savepoint_1
  <<< [5] <query> 122 us
  >>> [6] <query> SELECT * FROM `differential_changeset` WHERE diffID = 1
  <<< [6] <query> 307 us
  >>> [7] <query> SAVEPOINT Aphront_Savepoint_2
  <<< [7] <query> 241 us
  >>> [8] <query> SELECT * FROM `differential_hunk` WHERE changesetID = 1
  <<< [8] <query> 212 us
  >>> [9] <query> DELETE FROM `differential_hunk` WHERE `id` = 1
  <<< [9] <query> 216 us
  >>> [10] <query> DELETE FROM `differential_changeset` WHERE `id` = 1
  <<< [10] <query> 154 us
  >>> [11] <query> SAVEPOINT Aphront_Savepoint_2
  <<< [11] <query> 118 us
  >>> [12] <query> SELECT * FROM `differential_hunk` WHERE changesetID = 2
  <<< [12] <query> 194 us
  >>> [13] <query> DELETE FROM `differential_hunk` WHERE `id` = 2
  <<< [13] <query> 179 us
  >>> [14] <query> DELETE FROM `differential_changeset` WHERE `id` = 2
  <<< [14] <query> 163 us
  >>> [15] <query> SAVEPOINT Aphront_Savepoint_2
  <<< [15] <query> 105 us
  >>> [16] <query> SELECT * FROM `differential_hunk` WHERE changesetID = 3
  <<< [16] <query> 211 us
  >>> [17] <query> DELETE FROM `differential_hunk` WHERE `id` = 3
  <<< [17] <query> 159 us
  >>> [18] <query> DELETE FROM `differential_changeset` WHERE `id` = 3
  <<< [18] <query> 152 us
  >>> [19] <query> SAVEPOINT Aphront_Savepoint_2
  <<< [19] <query> 124 us
  >>> [20] <query> SELECT * FROM `differential_hunk` WHERE changesetID = 4
  <<< [20] <query> 191 us
  >>> [21] <query> DELETE FROM `differential_hunk` WHERE `id` = 4
  <<< [21] <query> 155 us
  >>> [22] <query> DELETE FROM `differential_changeset` WHERE `id` = 4
  <<< [22] <query> 149 us
  >>> [23] <query> SELECT * FROM `differential_diffproperty` WHERE diffID = 1
  <<< [23] <query> 242 us
  >>> [24] <query> DELETE FROM `differential_diffproperty` WHERE `id` = 1
  <<< [24] <query> 196 us
  >>> [25] <query> DELETE FROM `differential_diff` WHERE `id` = 1
  <<< [25] <query> 169 us
  >>> [26] <query> DELETE FROM `differential_relationship` WHERE revisionID = 1
  <<< [26] <query> 178 us
  >>> [27] <query> DELETE FROM `differential_commit` WHERE revisionID = 1
  <<< [27] <query> 164 us
  >>> [28] <query> SELECT * FROM `differential_comment` WHERE revisionID = 1
  <<< [28] <query> 221 us
  >>> [29] <query> DELETE FROM `differential_comment` WHERE `id` = 1
  <<< [29] <query> 172 us
  >>> [30] <query> SELECT * FROM `differential_inlinecomment` WHERE revisionID = 1
  <<< [30] <query> 296 us
  >>> [31] <query> SELECT * FROM `differential_auxiliaryfield` WHERE revisionPHID = 'PHID-DREV-ooky7ozqukpmwget32oc'
  <<< [31] <query> 308 us
  >>> [32] <query> SELECT * FROM `differential_affectedpath` WHERE revisionID = 1
  <<< [32] <query> 4,173 us
  >>> [33] <query> DELETE FROM `differential_revision` WHERE `id` = 1
  <<< [33] <query> 231 us
  >>> [34] <query> COMMIT
  <<< [34] <query> 686 us
  OK, destroyed revision.

Reviewers: csilvers, vrana, jungejason

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2796
2012-06-19 11:52:50 -07:00
vrana
0acb7734cd Use pht()
Summary:
This is the first step in Phabricator internationalization.
It adds a translation selector and calls it at startup.
Installations can add custom selectors to override some texts.
We can add official translations in future.

Next step is to allow user to choose his translation which will override the global one.

This is currently used only for English plurals.

Test Plan: Displayed a diff with unit test error, verified that it says 'Detail' or 'Details' and not 'Detail(s)'.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1139

Differential Revision: https://secure.phabricator.com/D2753
2012-06-14 16:25:20 -07:00
epriestley
259638e900 Fix minor issues with D2630
Summary:
  - The config is called "resource-path" and the script references "resource-path", but the actual value checked for is "resource-map".
  - Use nonempty(), since defaulting with getEnvConfig() will give you null if the setting exists but is set to null. This default is nearly useless so maybe we should change it to use coalesce().
  - Remove Celerity map initialization from warmup. We don't currently initialize the environment in warmup, and Celerity initialization now depends on the environment.

Test Plan: Ran patch locally and on FPM-Warmup.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: hsb, aran

Differential Revision: https://secure.phabricator.com/D2662
2012-06-06 09:12:42 -07:00
epriestley
0a7b4591ef Allow usernames to include ".", "-" and "_"
Summary:
See T1303, which presents a reasonable case for inclusion of these characters in valid usernames.

Also, unify username validity handling.

Test Plan: Created a new user with a valid name. Tried to create a new user with an invalid name. Ran unit tests.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1303

Differential Revision: https://secure.phabricator.com/D2651
2012-06-06 07:09:05 -07:00
vrana
8883c9494f Allow specifying custom celerity resource map
Summary:
We have custom static resources.
We currently include them in Phabricator's celerity resource map which is causing some pain - we need to regenerate the file without our custom resources before pushing upstream, we need to discard our changes before pulling from upstream and we need to rebuild with our changes to run Phabricator.

This diff allows writing and reading the map in other location.
The plan is this - I will run `celerity_mapper.php` twice - once to build Phabricator-only resources (to push to upstream) and once to build Phabricator + ours resoruces to put in our directory.

Better solution would be to create a map just with our resources and read and combine it with Phabricator resources.
But it is complicated because we have dependencies on Phabricator resources.

Test Plan:
`celerity_mapper.php webroot`
`celerity_mapper.php webroot ../facebook/src/__celerity_resource_map__.php`
Delete Phabricator's celerity map, set 'celerity.resource-path' and successfully load Phabricator.

Reviewers: epriestley, edward

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T721

Differential Revision: https://secure.phabricator.com/D2630
2012-06-04 18:45:03 -07:00
vrana
ec9589fb3b Ignore errors in svn diff
Summary: Otherwise attaching the commit diff doesn't work.

Test Plan: Reparsed previously failing commit message.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2605
2012-06-01 21:45:33 -07:00
vrana
1ebf9186b4 Depend on class autoloading
Test Plan:
Run setup.
/differential/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Maniphest Tasks: T1103

Differential Revision: https://secure.phabricator.com/D2612
2012-05-30 16:57:21 -07:00
epriestley
70fd96037b Consolidate user editing code
Summary:
  - We currently have some bugs in account creation due to nontransactional user/email editing.
    - We save $user, then try to save $email. This may fail for various reasons, commonly because the email isn't unique.
    - This leaves us with a $user with no email.
  - Also, logging of edits is somewhat inconsistent across various edit mechanisms.
  - Move all editing to a `PhabricatorUserEditor` class.
  - Handle some broken-data cases more gracefully.

Test Plan:
  - Created and edited a user with `accountadmin`.
  - Created a user with `add_user.php`
  - Created and edited a user with People editor.
  - Created a user with OAuth.
  - Edited user information via Settings.
  - Tried to create an OAuth user with a duplicate email address, got a proper error.
  - Tried to create a user via People with a duplicate email address, got a proper error.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: tberman, aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2569
2012-05-25 07:30:44 -07:00
epriestley
f22d9131c6 Fix "phd repository-launch-readonly"
Summary:
  - Delete an unreachable block of code.
  - Pass "--" to terminate overseer args.

Test Plan: Ran "phd repository-launch-readonly", didn't get argument errors out of the daemon.

Reviewers: jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2456
2012-05-10 20:27:55 -07:00
epriestley
6fd88641c3 Minor, instructions suggest "phd list" but really mean "phd status".
"phd list" lists available (not running) daemons.

Auditors: csilvers
2012-05-09 11:33:10 -07:00
epriestley
b800df8c1b Simplify daemon management: "phd start"
Summary:
  - Merge CommitTask daemon into PullLocal daemon. This is another artifact of past instability (and order-dependent parsers). We still publish to the timeline, although this was the last consumer. Long term we'll probably delete timeline and move to webhooks, since everyone who has asked about this stuff has been eager to trade away the durability and ordering of the timeline for the ease of use of webhooks. There's also no reason to timeline this anymore since parsing is no longer order-dependent.
  - Add `phd start` to start all the daemons you need. Add `phd restart` to restart all the daemons you need. So cool~
  - Simplify and improve phd and Diffusion daemon documentation.

Test Plan:
  - Ran `phd start`.
  - Ran `phd restart`.
  - Generated/read documentation.
  - Imported some stuff, got clean parses.

Reviewers: btrahan, csilvers

Reviewed By: csilvers

CC: aran, jungejason, nh

Differential Revision: https://secure.phabricator.com/D2433
2012-05-09 10:29:37 -07:00
epriestley
d2b01aead0 Use one daemon to discover commits in all repositories, not one per repository
Summary:
See D2418. This merges the commit discovery daemon into the same single daemon, and applies all the same rules to it.

There are relatively few implementation changes, but a few things did change:

  - I simplified/improved Mercurial importing, by finding full branch tip hashes with "--debug branches" and using "parents --template {node}" so we don't need to do separate "--debug id" calls.
  - Added a new "--not" flag to exclude repositories, since I switched to real arg parsing anyway.
  - I removed a web UI notification that you need to restart the daemons, this is no longer true.
  - I added a web UI notification that no pull daemon is running on the machine.

NOTE: @makinde, this doesn't change anything from your perspective, but it something breaks this is the likely cause.

This implicitly resolves T792, because discovery no longer runs before pulling.

Test Plan:

  - Swapped databases to a fresh install.
  - Ran "pulllocal" in debug mode. Verified it correctly does nothing (fixed a minor issue with min() on empty array).
  - Added an SVN repository. Verified it cloned and discovered correctly.
  - Added a Mercurial repository. Verified it cloned and discovered correctly.
  - Added a Git repository. Verified it cloned and discovered correctly.
  - Ran with arguments to verify behaviors: "--not MTEST --not STEST", "P --no-discovery", "P".

Reviewers: btrahan, csilvers, Makinde

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T792

Differential Revision: https://secure.phabricator.com/D2430
2012-05-08 12:53:41 -07:00
epriestley
1c62a35710 Run one daemon to pull all working copies, not one daemon per working copy
Summary:
Allow the pull daemon to take a list of repositories. By default, pull all repositories.

Make some effort to respect pull frequencies, although we'll necessarily suffer a bit if running with only one process.

NOTE: We still launch one discovery daemon per working copy, so this only cuts the daemon count in half.

Test Plan:
  - Ran `phd debug pulllocal`, verified behavior.
  - Ran `pull.php P MTEST SVNTEST --trace`, verified it pulled the repos and ran the right commands.
  - Ran `phd repository-launch-master`, verified the right daemons launched, checked daemon console.
  - Ran `phd repository-launch-readonly`, verified the right daemon launched, checked daemon console.

Reviewers: btrahan, csilvers, davidreuss

Reviewed By: csilvers

CC: aran

Differential Revision: https://secure.phabricator.com/D2418
2012-05-07 15:01:10 -07:00
epriestley
87207b2f4e Allow users to have multiple email addresses, and verify emails
Summary:
  - Move email to a separate table.
  - Migrate existing email to new storage.
  - Allow users to add and remove email addresses.
  - Allow users to verify email addresses.
  - Allow users to change their primary email address.
  - Convert all the registration/reset/login code to understand these changes.
  - There are a few security considerations here but I think I've addressed them. Principally, it is important to never let a user acquire a verified email address they don't actually own. We ensure this by tightening the scoping of token generation rules to be (user, email) specific.
  - This should have essentially zero impact on Facebook, but may require some minor changes in the registration code -- I don't exactly remember how it is set up.

Not included here (next steps):

  - Allow configuration to restrict email to certain domains.
  - Allow configuration to require validated email.

Test Plan:
This is a fairly extensive, difficult-to-test change.

  - From "Email Addresses" interface:
    - Added new email (verified email verifications sent).
    - Changed primary email (verified old/new notificactions sent).
    - Resent verification emails (verified they sent).
    - Removed email.
    - Tried to add already-owned email.
  - Created new users with "accountadmin". Edited existing users with "accountadmin".
  - Created new users with "add_user.php".
  - Created new users with web interface.
  - Clicked welcome email link, verified it verified email.
  - Reset password.
  - Linked/unlinked oauth accounts.
  - Logged in with oauth account.
  - Logged in with email.
  - Registered with Oauth account.
  - Tried to register with OAuth account with duplicate email.
  - Verified errors for email verification with bad tokens, etc.

Reviewers: btrahan, vrana, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T1184

Differential Revision: https://secure.phabricator.com/D2393
2012-05-07 10:29:33 -07:00
vrana
74ef98e76e Delete __init_env__.php
Summary: This script is not used since D976.

Test Plan:
  grep init_env

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2379
2012-05-03 14:28:52 -07:00
vrana
73c82e5a94 Display holidays
Summary:
We will need it for two purposes:

- Status tool.
- Nagging tool - @aran suggested using "3 business days" and I don't want it to fall on New Year's Eve or such.

I don't plan working on any interface for editing this as this kind of data should be always imported.

Test Plan:
`bin/storage upgrade`
`scripts/calendar/import_us_holidays.php`
/calendar/

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2375
2012-05-03 09:22:52 -07:00
epriestley
f04d8ab1a7 Further improve unit/lint rendering
Summary:
I think this improves things, let me know if you have feedback.

Also addresses T840.

Test Plan: See screenshots...

Reviewers: vrana, btrahan, jungejason

Reviewed By: vrana

CC: aran, zeeg

Maniphest Tasks: T840

Differential Revision: https://secure.phabricator.com/D2357
2012-05-01 10:15:56 -07:00
epriestley
570feee199 Make default database namespace configurable
Summary: Allow the default namespace to be set in configuration, so you can juggle multiple copies of sandbox test data or whatever.

Test Plan: Changed default namespace, verified web UI and "storage" script respect it.

Reviewers: btrahan, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T345

Differential Revision: https://secure.phabricator.com/D2341
2012-04-30 11:56:58 -07:00
vrana
f362e5785f Ignore non-existing diffs in copied code detector script
Auditors: epriestley
2012-04-30 11:05:30 -07:00
vrana
e08b4cbb2c Inform about moved code and prefer it over copied code
Summary:
Also reduce the memory usage a little bit (before increasing it again).

I use the same CSS class as for the copied code.

Test Plan: Parsed 100 diffs and checked about 10 of them - looks good.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Koolvin

Differential Revision: https://secure.phabricator.com/D2339
2012-04-30 11:01:15 -07:00
epriestley
087cc0808a Make SQL patch management DAG-based and provide namespace support
Summary:
This addresses three issues with the current patch management system:

  # Two people developing at the same time often pick the same SQL patch number, and then have to go rename it. The system catches this, but it's silly.
  # Second/third-party developers can't use the same system to manage auxiliary storage they may want to add.
  # There's no way to build mock databases for unit tests that need to do reads.

To resolve these things, you can now name your patches whatever you want and conflicts are just merge conflicts, which are less of a pain to fix than filename conflicts.

Dependencies are now a DAG, with implicit dependencies created on the prior patch if no dependencies are specified. Developers can add new concrete subclasses of `PhabricatorSQLPatchList` to add storage management, and define the dependency branchpoint of their patches so they apply in the correct order (although, generally, they should not depend on the mainline patches, presumably).

The commands `storage upgrade --namespace test1234` and `storage destroy --namespace test1234` will allow unit tests to build and destroy MySQL storage.

A "quickstart" mode allows an upgrade from scratch in ~1200ms. Destruction takes about 200ms. These seem like fairily reasonable costs to actually use in tests. Building from scratch patch-by-patch takes about 6000ms.

Test Plan:
  - Created new databases from scratch with and without quickstart in a separate test namespace. Pointed the webapp at the test namespaces, browsed around, everything looked good.
  - Compared quickstart and no-quickstart dump states, they're identical except for mysqldump timestamps and a few similar things.
  - Upgraded a legacy database to the new storage format.
  - Destroyed / dumped storage.

Reviewers: edward, vrana, btrahan, jungejason

Reviewed By: btrahan

CC: aran, nh

Maniphest Tasks: T140, T345

Differential Revision: https://secure.phabricator.com/D2323
2012-04-30 07:54:00 -07:00
epriestley
20a5c9b261 Use "closed", not "committed", in Differential
Summary: "Committed" is SVN-specific language, and confusing in Git and Mercurial. Use neutral language instead.

Test Plan: Inspection.

Reviewers: btrahan, Makinde, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T909

Differential Revision: https://secure.phabricator.com/D2087
2012-04-23 17:40:57 -07:00
epriestley
3f6f8421b2 Update package definitions
Summary: So much Javascript. :(

Test Plan: Verified major Diffusion, Differential, Maniphest interfaces pull in no unpackaged CSS/JS. Rebuilt Celerity map.

Reviewers: btrahan, vrana, jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2248
2012-04-16 17:51:30 -07:00
vrana
88cba92477 Fix English
Summary: I usually don't dare to fix English but this one doesn't seem correct even to me.

Test Plan: Read.

Reviewers: epriestley, jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2214
2012-04-12 13:38:14 -07:00
vrana
32d2395a45 Unify links to www.phabricator.com and phabricator.com
Test Plan:
  scripts/sql/upgrade_schema.php

Verify links at /directory/2/.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Maniphest Tasks: T1096

Differential Revision: https://secure.phabricator.com/D2172
2012-04-09 14:32:03 -07:00
vrana
2c8e6f99bd Standardize mysql.configuration-provider
Summary: NOTE: BC break!

Test Plan: /

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, nh

Differential Revision: https://secure.phabricator.com/D2130
2012-04-08 21:32:15 -07:00
epriestley
e4df959064 Use Celerity to version all static resources
Summary:
We don't use versioned URIs for images, so when they change users may get old versions.

This was a particular issue with the recent logo change, which several users reported cache-related issues from.

Instead, use Celerity to manage image URI versions in addition to CSS/JS.

This is complicated, because we need to rewrite image URIs inside of CSS, which means the hash of a CSS file has to be derived from the current image data. Otherwise, when we updated an image the CSS wouldn't update, so we wouldn't be any better off.

So basically we:

  - Find all the "raw" files, and put them into the map.
  - Find all the CSS/JS, perform content-altering transformations on it (i.e., not minification) based on the partial map, and then put it into the map based on transformed hashes.

(If we wanted, we could now do CSS variables or whatever for "free", more or less.)

Test Plan:
  - Regenerated celerity map, browsed site, verified images generated with versioned URIs.
  - Moved "blue" flag image over "green" flag image, regenerated map, verified "green" flag image and the associated CSS changed hashes.
  - Added transformation unit tests; ran unit tests.

Reviewers: btrahan, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1073

Differential Revision: https://secure.phabricator.com/D2146
2012-04-08 10:07:51 -07:00
epriestley
b90d41dd90 Add an "--ignore-duplicates" flag to import_project_symbols.php
Summary: People are hella lazy and don't want to do this themselves.

Test Plan: Generated a symbol file with duplicates and piped it in, got an import under --ignore-duplicates.

Reviewers: kdeggelman, btrahan, vrana, jungejason

Reviewed By: jungejason

CC: aran

Differential Revision: https://secure.phabricator.com/D2145
2012-04-07 17:25:38 -07:00
vrana
d4c5761f41 Customizable MySQL implementation
Test Plan:
- /
- upgrade_schema.php
- Setup
- Try disabling mysql_connect.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2133
2012-04-07 10:54:12 -07:00
vrana
23988ca482 Support Windows
Test Plan:
Enable setup.
Disable setup.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2118
2012-04-06 09:34:06 -07:00
vrana
23d5d7a1a6 Delete unused CSS
Summary: Unused since D2062.

Test Plan:
View commit.
View task.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran

Differential Revision: https://secure.phabricator.com/D2108
2012-04-05 10:54:06 -07:00
Jason Ge
4aef876138 Add script to delete revisions without diff
Summary: detect all revisions that don't have a diff, then delete them.

Test Plan:
we have been using this script for several months in
Facebook and it's working well.

Reviewers: epriestley, nh, btrahan

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T605

Differential Revision: https://secure.phabricator.com/D2061
2012-03-30 14:14:38 -07:00
epriestley
eaa2ff71d3 Minify static resources
Summary: For production servers, minify CSS and JS by stripping comments, whitespace, etc.

Test Plan: Looked at CSS/JS, it was much smaller.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T264

Differential Revision: https://secure.phabricator.com/D2034
2012-03-28 10:13:53 -07:00
epriestley
a3028558ae Make includes free to the user (application support component)
Summary:
We spend a significant amount of time running includes, even with APC. However, we have rigidly structured includes and can safely run them all in workers before requests occur.

Right now, requests go like this:

  - Apache spawns a worker.
  - Client sends an HTTP request.
  - Apache interprets it.
  - Apache sees it's ".php", so it hands it off to the PHP SAPI.
  - The PHP SAPI starts the PHP interpreter in the worker.
  - The request is handled, etc.

Instead, we want to do this:

  - Worker spawns and loads the world.
  - Client sends an HTTP request.
  - Webeserver interprets it.
  - Sees it's a ".php", hands it off to the SAPI.
  - SAPI executes it on a loaded world.

No SAPIs I know of support this, but I added support to PHP-FPM fairly easily (in the sense that it took me 6 hours and I have a hacky, barely-working mess). Over HTTP (vs HTTPS) the performance improvement is pretty dramatic.

HPHP doesn't significantly defray this cost so we're probably quite a bit faster (to the user) under nginx+PHP-FPM than HPHP after this works for real.

I have the php-fpm half of this patch in a messy state, I'm going to try to port it to be vs php 5.4.

Test Plan: Ran a patched php-fpm, browsed around, site works, appears dramatically faster.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D2030
2012-03-27 16:06:13 -07:00
epriestley
914f044b62 More Drydock Stuff
Summary:
  - Still really really rough.
  - Adds a full synchronous mode for debugging.
  - Adds some logging.
  - It can now allocate EC2 machines and put webroots on them in a hacky, terrible way.
  - Adds a base query class.

Test Plan: oh hey look a test page? http://ec2-50-18-65-151.us-west-1.compute.amazonaws.com:2011/

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D2026
2012-03-26 20:54:26 -07:00
Julius Seporaitis
53b06d1a52 Project symbol import from 'ctags'
Summary:
I noticed that documentation said it is possible to have 'ctags' symbol import, so I hacked a quick version. I tested it on Python based project and successfuly imported symbols.

It is limited to classes right now, as the importer script complained about not-unique method names (there are a lot of 'get' & 'post' methods accross classes in my project).

If you would have any feedback about this, I would definetly try to wrap it up for possibly merging into main repository.

Test Plan:
Required 'ctags' tool (ctags.sourceforge.net/) Tested to work with version 5.8+ and didn't work with 3.x.

1. `find . -type f '*.py' | ./generate_ctags_symbols.php > /tmp/symbols`
2. `./import_project_symbols.php` < /tmp/symbols

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: seporaitis, aran, epriestley

Maniphest Tasks: T1034

Differential Revision: https://secure.phabricator.com/D1995
2012-03-25 09:50:42 -07:00
vrana
4fba549a99 Use PhabricatorEnv::newObjectFromConfig() wherever possible
Test Plan:
/mail/send/
scripts/aphront/aphrontpath.php /

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1983
2012-03-21 14:57:52 -07:00
epriestley
097e62b45c Use libphutil script initializer to initialize scripts in Phabricator
Summary:
libphutil has some basic environmental sanity checks that we should use when initializing scripts in Phabricator. Principally this:

https://secure.phabricator.com/diffusion/PHU/browse/master/scripts/__init_script__.php;db643ee9f5f524e7$26

Without this, the default ini may set CLI errors to go to some logfile, which means exceptions aren't shown on stderr.

See https://github.com/facebook/phabricator/issues/98/

Test Plan:
  - Ran "php -derror_log=/dev/null -f ./bin/phd debug adslkfnasdfnalks" prior to change; got confusing lack of output.
  - Ran "phd -derror_log=/dev/null -f ./bin/phd debug asdkflnaslfdnala" after change; got exception on stderr.

Reviewers: btrahan, killermonk

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1950
2012-03-19 19:21:10 -07:00
epriestley
d5c3d83a3f Add a little SQL-info script that is sort of useful maybe?
Summary: I'm not aware of an easy way to get this information through normal tools. I'm sure there's some fancy GUI client that has it but this seems worth keeping around.

Test Plan: ran script, got helpful information about data sizes

Reviewers: nh, btrahan

Reviewed By: nh

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1916
2012-03-15 14:16:40 -07:00
vrana
814f4adf6c Stabilize sort in celerity map
Summary:
Smaller diffs, better blame, less noise on a file which is usually ignored.

Less conflicts in merge!

Test Plan:   php -l __celerity_resource_map__.php

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1909
2012-03-15 11:08:14 -07:00
vrana
0c728bd7a3 Don't include requirements of non-providers in celerity map
Summary: Tests and docstubs don't provide anything and they are not required in celerity map.

Test Plan: View Options in Differential.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1893
2012-03-14 22:05:18 -07:00
epriestley
1cca22f3fd Provide a script to "undo" the negative effects of an accidental push in Differential
Summary: If someone accidentally pushes a bunch of commits, revisions might get marked as "Committed" incorrectly. This will restore them to their previous state without too much fuss.

Test Plan: Ran the script on some commits to undo them, it seemed to work correctly.

Reviewers: davidreuss, btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1877
2012-03-13 11:18:22 -07:00
vrana
4c1e356658 Convert database to UTF-8
Summary: This is the script used for conversion: P319

Test Plan:
Update diff with UTF-8 characters in description.
`sql/upgrade_schema.php`
Verify data in DB and that it looks good on web.

Reviewers: epriestley, nh

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T327

Differential Revision: https://secure.phabricator.com/D1830
2012-03-12 12:11:02 -07:00
epriestley
ef74b9b88b Improve messaging around passworded SSH keys
Summary: It would be better to test if a key is passworded, but I couldn't figure out how to do that.

Test Plan: Ran "test_connection.php"

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T924

Differential Revision: https://secure.phabricator.com/D1856
2012-03-12 10:34:27 -07:00
epriestley
4bf559baa0 Add a script to programmatically add image macros
Summary: Urgent request.

Test Plan: Ran "add_macro.php" on some things; verified results in web console.

Reviewers: btrahan, tcook

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1832
2012-03-09 08:55:11 -08:00
epriestley
ce919b0822 Resolve implicit fallthrough in Phabricator
Summary: New implicit fallthrough linter detected a few issues; none of these have behavioral impacts but they can clearly be tightened up. See D1824.

Test Plan: Lint; inspection.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1825
2012-03-08 12:46:29 -08:00
epriestley
adb7f9ed1c Add a script to close all open audits in a repository
Summary: If you import a repository you may trigger a large number of irrelevant audits. Provide a tool to nuke them.

Test Plan: Ran "audit.php Q" (does not exist), "audit.php P" (phabricator) from various repository states.

Reviewers: btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T904, T940

Differential Revision: https://secure.phabricator.com/D1791
2012-03-06 15:10:25 -08:00
Nick Harper
756faad2aa Provide upgrade script for transition away from Meta MTA daemon
Summary:
When switching from using the MetaMTADaemon to a Taskmaster for sending mail,
if there are messages queued for delivery, they need to be re-queued into the
task system. This patch does that.

Task ID: #

Blame Rev:

Test Plan:
Ran it.

Revert Plan:

Tags:

Reviewers: epriestley, btrahan, jungejason

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1780
2012-03-05 15:15:19 -08:00
Nick Harper
b5897ec9c0 Add dry-run, max version options to upgrade_schema.php script
Summary:
1) Dry run option allows you to run the script to see what patches would get
applied (without actually applying them).
2) Max version allows you to do a partial upgrade.

Task ID: #

Blame Rev:

Test Plan:
ran a dry run twice (to make sure it didn't do anything) and no max version
ran with a dry run and max version to check max version logic works correctly
ran with just a max version, and only patches up to that point got applied

Revert Plan:

Tags:

Reviewers: epriestley, btrahan, jungejason

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1758
2012-03-02 11:23:25 -08:00
epriestley
974efd6d65 Minor, remove metamta reference from update script. 2012-03-02 06:42:31 -08:00
epriestley
41a265b464 Update Javelin and packaging
Summary:
  - Update the Javelin submodule to pick up recent fixes (like D1749).
  - Update the package definitions do do a slightly better job of packaging
resources.

Test Plan:
Up and down work in tokenizers now. Pages load slightly fewer
resources.

Reviewers: vrana, btrahan

Reviewed By: btrahan

CC: aran, epriestley

Maniphest Tasks: T927

Differential Revision: https://secure.phabricator.com/D1751
2012-03-01 17:23:00 -08:00
vrana
9b56c7e8a1 Create script printing controller name based on URL
Summary:
With this script, I am able to add bookmarklet to my browser which edits the
controller responsible for displaying current page by a single click.
Plus it can be useful also for other uses.

Test Plan:
  ./aphrontpath.php /
  ./aphrontpath.php D123
  ./aphrontpath.php /D123
  ./aphrontpath.php /D123/ # doesn't exist
  ./aphrontpath.php https://secure.phabricator.com/D123
  ./aphrontpath.php https://secure.phabricator.com/D123?x=2
  ./aphrontpath.php https://secure.phabricator.com/D123#comment-1
  ./aphrontpath.php differential
  ./aphrontpath.php /differential/
  ./aphrontpath.php /w/ # rewritten by custom config

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley

Maniphest Tasks: T349

Differential Revision: https://secure.phabricator.com/D1746
2012-03-01 15:17:50 -08:00
epriestley
c8b4bfdcd1 Encode "<" and ">" in JSON/Ajax responses to prevent content-sniffing attacks
Summary:
Some browsers will still sniff content types even with "Content-Type" and
"X-Content-Type-Options: nosniff". Encode "<" and ">" to prevent them from
sniffing the content as HTML.

See T865.

Also unified some of the code on this pathway.

Test Plan: Verified Opera no longer sniffs the Conduit response into HTML for
the test case in T865. Unit tests pass.

Reviewers: cbg, btrahan

Reviewed By: cbg

CC: aran, epriestley

Maniphest Tasks: T139, T865

Differential Revision: https://secure.phabricator.com/D1606
2012-02-14 14:51:51 -08:00
epriestley
f19d5fbeae Fix existence test in "reconcile.php"
Summary:
"rev-parse --verify" is actually a terrible test, it only survived my test cases
because I put garbage into the tables with RAND() or similar, not
properly-formatted garbage.

Use "cat-file -t" instead to ensure we perform an object-existence test.

Test Plan: Ran "reconcile.php P" locally, ran "cat-file -t" and "rev-parse
--verify" on properly-formatted but invalid hashes like
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", worked with @rguerin to resolve
commit issues.

Reviewers: btrahan, rguerin

Reviewed By: rguerin

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1590
2012-02-07 10:46:26 -08:00
epriestley
a15130b47c Add a maintenance script for reconciling repositories to disk state
Summary:
@rguerin ran into an issue in his install where Phabricator appears to have
discovered commits which no longer exist, and thus is failing to proceed with
its repository import.

It's not clear how we got into this state. Previously, it was possible by, e.g.,
parsing a different repository's working copy and then switching them back, but
there are now safeguards against that.

I'm taking a three-pronged approach to try to sort this out:

  - Provide a script to get out of this state (this script) and reconcile
Phabricator's view of a repository with an authoritative copy of it. This
basically "un-discovers" any discovered commits which don't actually exist (any
queued tasks to parse them will fail permanently when they fail to load the
commit object).
  - Add more logging to the discovery daemon so we can figure out where commits
came from.
  - Improve Diffusion's UI when stuff is partially discovered (T776).

(This script should also clean up some nonsense on secure.phabricator.com from a
botched Diviner import.)

Test Plan: Ran "reconcile.php" with bogus commits and bogus differential/commit
links, had them expunged. Will work with @rguerin to see if this resolves
things.

Reviewers: btrahan, rguerin

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1552
2012-02-02 16:03:50 -08:00
epriestley
7d1f62409d Move resource allocation to task queue
Summary: Run the actual resource allocation for Drydock out-of-process via the
task queue.

Test Plan: Ran "drydock_control.php", saw it insert a task and wait for task
completion. Ran "phd debug taskmaster" and saw it run the task.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran, epriestley

Differential Revision: https://secure.phabricator.com/D1470
2012-01-24 09:44:14 -08:00
epriestley
d1ee08b2df Drydock Rough Cut
Summary:
Rough cut of Drydock. This is very basic and doesn't do much of use yet (it
//does// allocate EC2 machines as host resources and expose interfaces to them),
but I think the overall structure is more or less reasonable.

== Interfaces

Vision: Applications interact with Drydock resources through DrydockInterfaces,
like **command**, **filesystem** and **httpd** interfaces. Each interface allows
applications to perform some kind of operation on the resource, like executing
commands, reading/writing files, or configuring a web server. Interfaces have a
concrete, specific API:

  // Filesystem Interface
  $fs = $lease->getInterface('filesystem'); // Constants, some day?
  $fs->writeFile('index.html', 'hello world!');

  // Command Interface
  $cmd = $lease->getInterface('command');
  echo $cmd->execx('uptime');

  // HTTPD Interface
  $httpd = $lease->getInterface('httpd');
  $httpd->restart();

Interfaces are mostly just stock, although installs might add new interfaces if
they expose different ways to interact with resources (for instance, a resource
might want to expose a new 'MongoDB' interface or whatever).

Currently: We have like part of a command interface.

== Leases

Vision: Leases keep track of which resources are in use, and what they're being
used for. They allow us to know when we need to allocate more resources (too
many sandcastles on the existing hosts, e.g.) and when we can release resources
(because they are no longer being used). They also give applications something
to hold while resources are being allocated.

  // EXAMPLE: How this should work some day.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('sandcastle');
  $allocator->setAttributes(
    array(
      'diffID' => $diff->getID(),
    ));
  $lease = $allocator->allocate();
  $diff->setSandcastleLeaseID($lease->getID());

  // ...

  if ($lease->getStatus() == DrydockLeaseStatus::STATUS_ACTIVE) {
    $sandcastle_link = $lease->getInterface('httpd')->getURI('/');
  } else {
    $sandcastle_link = 'Still building your sandcastle...';
  }
  echo "Sandcastle for this diff: ".$sandcastle_link;

  // EXAMPLE: How this actually works now.
  $allocator = new DrydockAllocator();
  $allocator->setResourceType('host');
  // NOTE: Allocation is currently synchronous but will be task-driven soon.
  $lease = $allocator->allocate();

Leases are completely stock, installs will not define new lease types.

Currently: Leases exist and work but are very very basic.

== Resources

Vision: Resources represent some actual thing we've put somewhere, whether it's
a host, a block of storage, a webroot, or whatever else. Applications interact
through resources by acquiring leases to them, and then getting interfaces
through these leases. The lease acquisition process has a side effect of
allocating new resources if a lease can't be acquired on existing resources
(e.g., the application wants storage but all storage resources are full) and
things are configured to autoscale.

Resources may themselves acquire leases in order to allocate. For instance, a
storage resource might first acquire a lease to a host resource. A 'test
scaffold' resource might lease a storage resource and a mysql resource.

Not all resources are auto-allocate: the entry-level version of Drydock is that
you manually allocate a couple boxes and configure them through the web console.
Then, e.g.,  'storage' / 'webroot' resources allocate on top of them, but the
host pool itself does not autoscale.

Resources are completely stock, they are abstract shells representing any
arbitrary thing.

Currently: Resource exist ('host' only) but are very very basic.

== Blueprints

Vision: Blueprints contain instructions for building interfaces to, (possibly)
allocating, updating, managing, and destroying a specific type of resource in a
specific location. One way to think of them is that they are scripts for
creating and deleting resources. For example, the LocalHost, RemoteHost and
EC2Host blueprints can all manage 'host' resources.

Eventually, we will support more types of resources (storage, webroot,
sandcastle, test scaffold, phacility deployment) and more providers for resource
types, some of which will be in the Phabricator mainline and some of which will
be custom.

Blueprints are very custom and specific to application types, so installs will
define new blueprints if they are making significant use of Drydock.

Currently: They exist but have few capabilities. The stock blueprints do nearly
nothing useful. There is a technically functional blueprint for host allocation
in EC2.

== Allocator

This is just the actual code to execute the lease acquisition process.

Test Plan: Ran "drydock_control.php" script, it allocated a machine in EC2,
acquired a lease on it, interfaced with it, and then released the lease. Ran it
again, got a fresh lease on the existing resource.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D1454
2012-01-19 21:12:57 -08:00
epriestley
0402e4e06e Make "purge_cache.php --differential" also purge the inline comment cache
Summary: We purge the cache for primary comments only right now, not inline
comments.

Test Plan: Ran "purge_cache.php --differential".

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T683

Differential Revision: https://secure.phabricator.com/D1451
2012-01-18 15:20:33 -08:00
epriestley
82c0795e54 Unify logic for username validation
Summary: Revisit of D1254. Don't require lowercase, just standardize the logic.
The current implementation has nonuniform logic -- PeopleEditController forbids
uppercase.

Test Plan: Ran unit tests, see also D1254.

Reviewers: btrahan, jungejason, aran

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D1415
2012-01-16 11:52:59 -08:00
epriestley
5fd46dce66 Improve order of operations in upgrade script
Summary:
  - Run "phd stop" before stopping apache. This is essentially a smoke test for
PHABRICATOR_ENV being set.
  - Run documentation generation after everything else. Between the pull and the
restart we have some minor exposure to APC issues with deleted files and
out-of-date module definitions, and this limits that.
  - Pull commands out of (x && y) stuff, this prevents "set -e" from working
correctly.

Test Plan: Ran upgrade script locally.

Reviewers: btrahan, jungejason

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D1419
2012-01-16 11:52:01 -08:00
Bob Trahan
cf61f0e32d Adding an "ssh" client for conduit
Summary: ..."ssh" is in quotes 'cuz this is step 1 and there's no ssh in sight
at the moment.

Test Plan:
ran api.php PHID-USER-xee4ju2teq7mflitwfcs differential.query a few times...
 - tried valid input, it worked!
 - tried bad input, it worked in that it failed and told me so!
ran api.php crap_user differential.query a few times...
 - verified error message with respect to crap_user
ran api.php PHID-USER-xee4ju2teq7mflitwfcs crap_method a few times...
 - verified error message with respect to crap_method
visited http://phabricator.dev/conduit/method/differential.query a few times...
 - tried valid input, it worked!

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, btrahan, epriestley

Maniphest Tasks: T550

Differential Revision: https://secure.phabricator.com/D1357
2012-01-13 11:54:13 -08:00
jungejason
49cf1ca10c Support deleting individual changeset
Summary:
add support in purge_cache.php. We might want to add support
for maniphest comments and differential comments later.

Test Plan:
- purging all worked
- purging individual changeset worked
- when input is not integer, correct error message reported

Reviewers: epriestley, btrahan, nh

Reviewed By: nh

CC: nh, aran, jungejason, epriestley

Maniphest Tasks: T683

Differential Revision: https://secure.phabricator.com/D1372
2012-01-11 20:14:55 -08:00
epriestley
58f2cb2509 Provide a script for batch creating user accounts
Summary: Make it a little easier to create a bunch of accounts if your company
has more than like 5 employees.

Test Plan: Ran "add_user.php" to create new users. Created new users from the
web console.

Reviewers: btrahan, jungejason, rguerin

Reviewed By: btrahan

CC: aran, btrahan, rguerin

Differential Revision: https://secure.phabricator.com/D1336
2012-01-06 11:50:51 -08:00
epriestley
fd8303aa75 Document how to use the symbol importer
Summary: Some day we might have a fancy daemon for this, but for now at least
provide some instructions on using the existing importers, etc., to index
project symbols.

Test Plan:
  - Generated documentation, read over the result.
  - Ran the example code.

Reviewers: btrahan, jungejason, davidreuss

Reviewed By: jungejason

CC: aran, jungejason

Maniphest Tasks: T315

Differential Revision: https://secure.phabricator.com/D1262
2011-12-22 06:45:59 -08:00
epriestley
c93fc91e96 Update Javelin; improve package definitions
Summary:
  - Update Javelin to HEAD -- this doesn't pick up anything in particular, but
lets us smoke test some stuff like {D1217}.
  - Do a little more packaging since we've picked up a handful of 10-line
behaviors and such for various UI tweaks.

Test Plan:
  - Generally, this should be very low-risk.
  - Browed Maniphest, Differential, Diffusion and tried to hit all the JS
interactions.
  - Looked over the Javelin changes we're pulling in to see if I forgot
anything. The only API change I caught was removal of "JX.defer()", but that was
already cleared in Phabricator in D803.

Reviewers: aran, btrahan, jungejason

Reviewed By: aran

CC: aran

Differential Revision: 1240
2011-12-20 08:27:54 -08:00
epriestley
074bf4ed7d Add a script for purging long-lived caches
Summary: See task; installs occasionally need to do this themselves, and a
script is much better than me telling them to truncate tables.

Test Plan:
Ran various flavors of this command:

  - purge_cache.php
  - purge_cache.php derp
  - purge_cache.php --help
  - purge_cache.php --all
  - purge_cache.php --differential
  - purge_cache.php --differential --maniphest

Then I verified the actual behavior:

  - Visited a Differential revision with comments, observed cache update in
'Services' tab.
  - Visited a Maniphest task with comments, observed cache update in 'Services'
tab.
  - Reloaded a diff standalone view, got a cache update.

Reviewers: Makinde, btrahan, jungejason

Reviewed By: jungejason

CC: aran, jungejason

Maniphest Tasks: T676

Differential Revision: 1214
2011-12-16 12:43:31 -08:00
jungejason
c13b7da290 Add Related Commits for Owners
Summary:
For each commit, find the affected packages, and provide a way to
search by package.

Test Plan:
create commits that touch and don't touch two packages, and verify
that they display correctly in all the UI pages.

Reviewers: epriestley, blair, nh, tuomaspelkonen

Reviewed By: epriestley

CC: benmathews, aran, epriestley, btrahan, jungejason, mpodobnik, prithvi

Maniphest Tasks: T83

Differential Revision: 1208
2011-12-14 22:48:57 -08:00
David Reuss
dc0d3f3fef Fix encodings for mime headers and body if not UTF-8
Summary:
If mails are not sent in UTF-8 we cannot just it verbatim, so we have to
encode it into UTF-8 if it is not the case. Mime headers use different
encodings like "quoted-printable", which we have to handle.

It looks like "Subject: =?iso-8859-1?Q?opr=E6t_s=E5_den_task?=", and can
be decoded by ##iconv_mime_decode##.

Furthermore the body of the email might be in various encodings as well,
which we attempt to pull from the content-type header of the plain text
part of the mail.

Test Plan:
Attempted receiving mails in a variety of flavors. These could be
converted to test-cases once i know if this is a sane solution. Got
expected results from mails sent with Windows-1252 and ISO-8859-1.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, davidreuss

Differential Revision: 1093
2011-12-02 08:47:45 -08:00
Evan Priestley
0511bf36b3 Merge pull request #81 from moos3/master
Fix php binary location from /usr/bin/php to /usr/bin/env php
2011-11-30 09:26:48 -08:00
moos3
31c3f2a03e Fix to calling the php binary
Summary: by default it was looking for php to be in /usr/bin/php on some systems
this is incorrect. It should be using /usr/bin/env php for calling php with the
correct path.

Test Plan: patch and try to run.

Reviewers: epriestley

Reviewed By: epriestley

CC: svemir, aran, epriestley

Differential Revision: 1136
2011-11-30 12:21:41 -05:00
Bob Trahan
0795cd4baa Add cycle detection to celerity mapper
Summary: create CelerityResourceGraph, which extends AbstractDirectedGraph.
since we've done a bunch of work already to load the resource graph into memory
CelerityResourceGraph simply stores a copy and makes loadEdges work off that
stored copy.

Test Plan:
made phabricator-prefab require herald-rule-editor

~/code/phabricator> ./scripts/celerity_mapper.php webroot
Finding static resources...
Processing 154
files..........................................................................................................................................................
[2011-11-22 11:28:29] EXCEPTION: (Exception) Cycle detected in resource graph:
phabricator-prefab => herald-rule-editor => phabricator-prefab at
[/Users/btrahan/Dropbox/code/phabricator/scripts/celerity_mapper.php:173]

fixed phabricator-prefab requiring herald-rule-editor.  re-ran celerity_mapper
and no errors!

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, btrahan, epriestley

Differential Revision: 1132
2011-11-29 12:09:08 -08:00
adonohue
75e1a0d5a8 Fix celerity_mapper.php @generated issue
Summary: Easy

Test Plan: Run ##scripts/celerity_mapper.php webroot/##, verify
src/__celerity_resource_map__.php

Reviewers: epriestley, btrahan

Reviewed By: epriestley

CC: aran, epriestley

Differential Revision: 1133
2011-11-28 11:44:59 -08:00
David Reuss
13d15276a5 Catch any exceptions thrown when handling mail
Summary:
This makes debugging issues a lot easier, since any exceptions thrown
gets logged and can be seen at /mail/received/, for inspection.

Test Plan:
Created a task via the public-author functionality, and saw no errors
without this patch applied. Even worse, the exception was never shown at
command line either. Not sure what's up with that. Output buffering, or
whatever?

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, epriestley, davidreuss

Differential Revision: 1042
2011-10-28 08:05:27 -07:00
Jason Ge
13eee1a344 Add test to check all symbols can be loaded
Summary:
make sure all symboles can be loaded to avoid issues like missing
methods in descendants of abstract base class.

Test Plan:
ran it and verified it passes; remove a method in a descendant class
and verified that the test failed.

Reviewers: epriestley, nh

Reviewed By: nh

CC: aran, nh, jungejason

Differential Revision: 1023
2011-10-20 16:43:13 -07:00
John Chee
bafb75bdea Fix string matching bug in Ubuntu install script.
Commit f58e63a917 introduced a bug that
always set GIT to 'git-core'.
2011-10-17 10:18:18 -07:00
John Chee
f58e63a917 Install proper git package in Ubuntu install script.
'git-core' provided the 'git' command in Ubuntu LTS.
2011-10-14 17:06:12 -07:00
John Chee
d5431f6d9d Install php5-cli in Ubuntu install script.
php5-cli is necessary to run php on the command line.
2011-10-14 17:03:45 -07:00
epriestley
e4e5c39457 Merge __init_env__.php into __init_script__.php
Summary: There are currently two files, but all scripts require both of them,
which is clearly silly. In the longer term I want to rewrite all of this init
stuff to be more structured (e.g., merge webroot/index.php and __init_script__
better) but this reduces the surface area of the ad-hoc "include files" API we
have now, at least.

Test Plan:
  - Grepped for __init_env__.php (no hits)
  - Ran a unit test (to test unit changes)
  - Ran a daemon (to test daemon changes)

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 976
2011-10-02 11:48:09 -07:00
Ricky Elrod
10570635b5 Stop 'stop' from being in phd's list twice, and provide a way to kill one particular PID.
Summary:
This is a pretty bad, but working implmentation of a way to kill one particular PID that
is controlled by Phabricator. Also remove the second 'stop' from the ##phd help## list.

Test Plan:
  [ricky@rhelpad01 phabricator] (phd-stop-twice)$ ./bin/phd status
  PID  	Started                 	Daemon
  30154	Oct 1 2011, 2:38:08 AM  	PhabricatorMetaMTADaemon
  30172	Oct 1 2011, 2:38:09 AM  	PhabricatorMetaMTADaemon
  30190	Oct 1 2011, 2:38:09 AM  	PhabricatorMetaMTADaemon
  30210	Oct 1 2011, 2:38:09 AM  	PhabricatorMetaMTADaemon

  [ricky@rhelpad01 phabricator] (phd-stop-twice)$ ./bin/phd stop 30190
  Stopping daemon 'PhabricatorMetaMTADaemon' (30190)...
  Daemon 30190 exited normally.

  [ricky@rhelpad01 phabricator] (phd-stop-twice)$ ./bin/phd stop 123456
  123456 is not controlled by Phabricator. Not killing.

  [ricky@rhelpad01 phabricator] (phd-stop-twice)$ ./bin/phd stop
  Stopping daemon 'PhabricatorMetaMTADaemon' (30154)...
  Stopping daemon 'PhabricatorMetaMTADaemon' (30172)...
  Stopping daemon 'PhabricatorMetaMTADaemon' (30210)...
  Daemon 30210 exited normally.
  Daemon 30154 exited normally.
  Daemon 30172 exited normally.

Reviewers: epriestley

CC:

Differential Revision: 975
2011-10-01 17:31:20 -04:00
epriestley
fc5cc180ab Fix __init_script__.php issue with event engine. 2011-09-30 13:17:45 -07:00
epriestley
522e5b4779 Build an event dispatch mechanism into Phabricator
Summary:
This is an attempt to satisfy a lot of the one-off requests a little more
generally, by providing a relatively generic piece of event architecture.

Allow the registation of event listeners which can react to various application
events (currently, task editing).

I'll doc this a bit better but I wanted to see if anyone had massive objections
to doing this or the broad approach. The specific problem I want to address is
that one client wants to do a bunch of routing for tasks via email, so it's
either build a hook, or have them override most of ManiphestReplyHandler, or
something slightly more general like this.

Test Plan: Wrote a silly listener that adds "Quack!" to a task every time it is
edited and edited some tasks. I was justly rewarded.

Reviewers: nh, jungejason, tuomaspelkonen, aran

Reviewed By: aran

CC: aran, epriestley

Differential Revision: 881
2011-09-30 12:16:40 -07:00
epriestley
be26c6a5c1 Refactor repository reparse scripts to be more useful
Summary:
Splitting up D960 a bit, see that for context.

We currently have two scripts, "parse_one_commit.php" and
"reparse_all_commit_messages.php", but they're sort of silly and you can't do
certain things with them. Replace them with one script which is more flexible
and can do specific reparse steps on individual commits or entire repos.

I left the old scripts as stubs since I think there are some FB wiki docs and
stuff that mention them. I'll delete them in a month or whenever I remember or
something.

Test Plan: Ran "reparse.php" with various arguments, including vs-one-commit,
vs-repository, with --trace, and against different types of repos.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 964
2011-09-27 17:20:04 -07:00
epriestley
1c1f749eba Add an "arcanist.projectinfo" Conduit call
Summary:
We currently rely on "remote_hooks_enabled" in .arcconfig to determine whether
commands like "arc amend" and "arc merge" should imply "arc mark-committed".

However, this is a historical artifact that is now bad for a bunch of reasons:

  - The option name is confusing, it really means 'repository is tracked'.
  - The option is hard to discover and generally sucks.
  - We can empirically determine the right answer since we now know if a project
is in a tracked repository.

Add a call which arcanist can make on these workflows to figure out if it is
interacting with a project in a tracked repository or not.

Also added an "isTracked()" convenience method to reduce the number of magic
strings all over the place.

Test Plan: Ran "arcanist.projectinfo" for nonexistent, untracked and tracked
projects.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, epriestley, Makinde

Differential Revision: 945
2011-09-21 14:19:14 -07:00
epriestley
e0b86cc81b Add a Mercurial commit discovery daemon
Summary:
Repository import has three major steps:

  - Commit discovery (serial)
  - Message parsing (parallel, mostly VCS independent)
  - Change parsing (parallel, highly VCS dependent)

This implements commit discovery for Mercurial, similar to git's parsing:

  - List the heads of all the branches.
  - If we haven't already discovered them, follow them back to their roots (or
the first commit we have discovered).
  - Import all the newly discovered commits, oldest first.

This is a little complicated but it ensures we discover commits in depth order,
so the discovery process is robust against interruption/failure. If we just
inserted commits as we went, we might read the tip, insert it, and then crash.
When we ran again, we'd think we had already discovered commits older than HEAD.

This also allows later stages to rely on being able to find Phabricator commit
IDs which correspond to parent commits.

NOTE: This importer is fairly slow because "hg" has a large startup time
(compare "hg --version" to "git --version" and "svn --version"; on my machine,
hg has 60ms of overhead for any command) and we need to run many commands (see
the whole "hg id" mess). You can expect something like 10,000 per hour, which
means you may need to run overnight to discover a large repository (IIRC, the
svn/git discovery processes are both about an order of magnitude faster). We
could improve this with batching, but I want to keep it as simple as possible
for now.

Test Plan: Discovered all the commits in the main Mercurial repository,
http://selenic.com/repo/hg.

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: Makinde

CC: aran, Makinde

Differential Revision: 943
2011-09-16 11:08:52 -07:00
epriestley
8396c879b1 Support Mercurial in test_connection.php
Summary: Add Mercurial support to this script.

Test Plan: Connected to public and private Bitbucket mercurial repos,
successfully (with valid credentials) and unsuccessfully (without valid
credentials).

Reviewers: Makinde, jungejason, nh, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, jungejason

Differential Revision: 938
2011-09-15 14:50:29 -07:00
epriestley
77ed7ade66 Add symbol import scripts for crossref features
Summary: Adds a script to import symbols from a ctags-like format, and another
to generate that format for PHP files.

Test Plan:
Ran it on Phabricator:

  mysql> select * from repository_symbol limit 200, 20;

+-------------------+---------------------------------------------------+------------+----------------+--------+------------+
  | arcanistProjectID | symbolName                                        |
symbolType | symbolLanguage | pathID | lineNumber |

+-------------------+---------------------------------------------------+------------+----------------+--------+------------+
  |                 1 | DifferentialDiffViewController                    |
class      | php            |     52 |         19 |
  |                 1 | DifferentialInlineCommentEditController           |
class      | php            |    308 |         19 |
  |                 1 | DifferentialInlineCommentPreviewController        |
class      | php            |  10543 |         19 |
  |                 1 | DifferentialRevisionEditController                |
class      | php            |  10544 |         19 |
  |                 1 | DifferentialRevisionListController                |
class      | php            |  10545 |         19 |
  |                 1 | DifferentialRevisionViewController                |
class      | php            |    142 |         19 |
  |                 1 | DifferentialSubscribeController                   |
class      | php            |  10546 |         19 |
  |                 1 | DifferentialRevisionListData                      |
class      | php            |     58 |         19 |
  |                 1 | DifferentialCommentEditor                         |
class      | php            |     39 |         19 |
  |                 1 | DifferentialRevisionEditor                        |
class      | php            |     42 |         24 |
  |                 1 | DifferentialFieldSpecificationIncompleteException |
class      | php            |  10547 |         19 |
  |                 1 | DifferentialFieldDataNotAvailableException        |
class      | php            |  10548 |         19 |
  |                 1 | DifferentialFieldParseException                   |
class      | php            |  10549 |         19 |
  |                 1 | DifferentialFieldValidationException              |
class      | php            |  10550 |         19 |
  |                 1 | DifferentialFieldSelector                         |
class      | php            |  10551 |         19 |
  |                 1 | DifferentialDefaultFieldSelector                  |
class      | php            |  10552 |         19 |
  |                 1 | DifferentialApplyPatchFieldSpecification          |
class      | php            |  10553 |         19 |
  |                 1 | DifferentialArcanistProjectFieldSpecification     |
class      | php            |  10554 |         19 |
  |                 1 | DifferentialAuthorFieldSpecification              |
class      | php            |  10555 |         19 |
  |                 1 | DifferentialFieldSpecification                    |
class      | php            |  10556 |         35 |

+-------------------+---------------------------------------------------+------------+----------------+--------+------------+
  20 rows in set (0.00 sec)

Reviewers: jungejason, nh, tuomaspelkonen, aran

Reviewed By: tuomaspelkonen

CC: aran, tuomaspelkonen

Differential Revision: 898
2011-09-13 08:49:44 -07:00
epriestley
1df7d4039e Store repository credentials with repositories
Summary:
Move toward storing credentials in configuration so it's easier to get the
daemons working. This should eventually solve all the key juggling junk you have
to do right now.

This only gets us part of the way to actually using these credentials in the
daemons since I have to go swap everything for $repository->execBlah().

I tried to write a web "Test Connection" button but it was too much of a mess to
get git to work since git doesn't give you access to its SSH command and SSH has
a bunch of interactive prompts which you can't really do anything about without
it or a bunch of ~/.ssh/config editing. This is what Git recommends:

https://git.wiki.kernel.org/index.php/GitFaq#How_do_I_specify_what_ssh_key_git_should_use.3F

..but it's not a great match for this use case.

Test Plan:
  - Only partial.
  - Ran "test_connection.php" on a Git repo with and without SSH, and with and
without valid credentials. This part works properly.
  - Ran "test_connection.php" on a public SVN repo, but I don't have private or
WEBDAV repos set up at the moment.
  - Mercurial doesn't work yet.
  - Daemons haven't been converted yet.

Reviewers: jungejason, tuomaspelkonen, aran

Reviewed By: jungejason

CC: aran, abdul, nmalcolm, epriestley, jungejason

Differential Revision: 888
2011-09-06 08:58:00 -07:00
epriestley
d3558eb25f Explicitly detect missing pcntl and exit with an obvious error
Summary: We already detect this in setup but double-check just in case, see
gmail thread.
Test Plan: Rebuilt PHP without pcntl and tried to run phd, got an error.
Reviewed By: aran
Reviewers: aran, jungejason, tuomaspelkonen
CC: aran
Differential Revision: 819
2011-08-17 13:29:34 -07:00
epriestley
39b4d20ce5 Create AphrontWriteGuard, a backup mechanism for CSRF validation
Summary:
Provide a catchall mechanism to find unprotected writes.

  - Depends on D758.
  - Similar to WriteOnHTTPGet stuff from Facebook's stack.
  - Since we have a small number of storage mechanisms and highly structured
read/write pathways, we can explicitly answer the question "is this page
performing a write?".
  - Never allow writes without CSRF checks.
  - This will probably break some things. That's fine: they're CSRF
vulnerabilities or weird edge cases that we can fix. But don't push to Facebook
for a few days unless you're prepared to deal with this.
  - **>>> MEGADERP: All Conduit write APIs are currently vulnerable to CSRF!
<<<**

Test Plan:
  - Ran some scripts that perform writes (scripts/search indexers), no issues.
  - Performed normal CSRF submits.
  - Added writes to an un-CSRF'd page, got an exception.
  - Executed conduit methods.
  - Did login/logout (this works because the logged-out user validates the
logged-out csrf "token").
  - Did OAuth login.
  - Did OAuth registration.

Reviewers: pedram, andrewjcg, erling, jungejason, tuomaspelkonen, aran,
codeblock
Commenters: pedram
CC: aran, epriestley, pedram
Differential Revision: 777
2011-08-16 13:29:57 -07:00
James Ide
c7e1fa626d Fix typo in Celerity mapper
Summary: s/pacakge/package/ (I love pfff, too, btw)
Test Plan: ☃
Reviewed By: epriestley
Reviewers: epriestley
CC: aran, epriestley
Differential Revision: 782
2011-08-03 23:25:40 -07:00
epriestley
467308dd12 Script to selectively convert MyISAM tables to InnoDB
Summary:
A bunch of the .sql patch files don't explicitly specify the table engine, but
we should always use InnoDB with the exception of one table which needs MyISAM
for FULLTEXT.

MySQL doesn't no-op an ALTER TABLE statment that changes the engine back to
itself and converting large tables can be time consuming, so convert only the
required tables.

Test Plan: Ran on secure.phabricator.com and my local box, it fixed all the
issues in about 3 seconds on secure.phabricator.com and <<1 second on my local.
Reviewed By: codeblock
Reviewers: codeblock, tuomaspelkonen, jungejason, aran
CC: aran, epriestley, codeblock
Differential Revision: 641
2011-07-11 11:42:28 -07:00
tuomaspelkonen
49310391e0 Added subscriber view to Maniphest.
Summary:
People want to see all the tasks they have subscribed to in one
view. A new table was added for this to make queries faster.

Test Plan:
Tested that the view was initially empty. After running the
reindex_maniphest.php script, I saw the correct tasks there. Added
myself as a subscriber to one task and made sure the view was
updated. Removed myself as a subscriber from one task and made sure
the view was updated again.

Reviewed By: epriestley
Reviewers: epriestley, jungejason, codeblock
CC: aran, rm, epriestley
Differential Revision: 603
2011-07-07 14:08:52 -07:00
epriestley
4ef918e213 Add a garbage collector daemon
Summary:
Phabricator generates a bunch of data that we don't need to keep around forever,
add a GC daemon to get rid of it with some basic configuration options.

This needs a couple more diffs to get some of the details but I think this is a
reasonable start.

I also fixed a couple of UI things related to this, e.g. the daemon logs page
going crazy when a daemon gets stuck in a loop and dumps tons of data to stdout.

Test Plan:
  - Ran gc daemon in 'phd debug' mode and saw it delete stuff, then sleep once
it had cleaned everything up.
  - Mucked around with TTLs and verified they work correctly.
  - Viewed gc'd transcripts in the web interface and made sure they displayed
okay.
  - Viewed daemon logs before/after garbage collection.
  - Running some run-at / run-for tests now, I'll update if the daemon doesn't
shut off in ~10-15 minutes. :P

Reviewed By: tuomaspelkonen
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, tuomaspelkonen, epriestley
Differential Revision: 583
2011-07-05 13:49:11 -07:00
epriestley
f6c3a902b5 Install "mysql-server" in the RHEL-derivatives script
Summary: We don't currently install this component, but should.
Test Plan: iiam
Reviewed By: codeblock
Reviewers: codeblock
CC: aran, codeblock
Differential Revision: 560
2011-06-30 12:17:10 -07:00
epriestley
de0c89261e Allow Maniphest tasks to be filtered by Project
Summary:
Major things taking place here:

  - A new table for storing <task, project> relationships.
  - Moved all task query logic into a dedicated class.
  - Added a "projects" filter to the UI.

I was originally going to try to drive this off the main search index but the
perf benefits of a custom schema make an overwhelming argument in favor of doing
it this way.

Test Plan:
Filtered tasks by author and owner and zero, one, and more than one project.
Exercised all the group/sort options. Ran the index script over my 100k task
corpus. Edited task-project membership and verified the index updated.

Reviewed By: cadamo
Reviewers: gc3, jungejason, cadamo, tuomaspelkonen, aran
CC: aran, cadamo, epriestley
Differential Revision: 556
2011-06-29 21:56:47 -07:00
epriestley
6a3eb19876 Provide a basic update script for Phabricator
Summary:
This isn't completely cross-system compatible but it's definitely better than
nothing.

Test Plan:
Pushed secure.phabricator.com a bunch of times.

Reviewed By: moskov
Reviewers: moskov
CC: aran, moskov
Differential Revision: 558
2011-06-29 20:05:28 -07:00
epriestley
b61e325f05 Show logs to the console in 'phd debug'
Summary:
Currently we send logs to the logfile in 'phd debug', but we should send them to
the console instead.

Also fixed some %C stuff which could theoretically cause problems if a user had
percentage symbols in their paths (heaven forbid).

fratrik, this or D535 might have been involved in frustrating your efforts to
debug the "sudo" stuff.

Test Plan:
Ran "phd debug irc derpderp" and "phd launch irc derpderp". In the former case,
the exception appeared in the console. In the latter, it appeared in the log.

Reviewed By: codeblock
Reviewers: codeblock, jungejason, tuomaspelkonen, aran, fratrik
CC: aran, codeblock
Differential Revision: 536
2011-06-26 20:41:08 -07:00
Jason Ge
d282549166 Add commits to the Phabricator search index
Summary:
create the indexer for commit

Test Plan:
run the reindex_one_commit.php against one existing commit
and it is returned in search result on the webpage; run parse_one_commit
against another commit; modified reindex_everything.php to let it only
parse one commit after loading all commits.

Reviewed By: epriestley
Reviewers: epriestley, aran
CC: aran, jungejason, epriestley, debow
Differential Revision: 490
2011-06-25 13:15:54 -07:00
epriestley
a632b220a8 Index users in search results.
Summary:
Add users to the search results. I need to follow this up with a patch to make
the search results stop being terrible. I'll do that.

Test Plan:
Searched for users, ran "reindex_all_users.php"

Reviewed By: jungejason
Reviewers: tomo, jungejason, aran
CC: aran, jungejason
Differential Revision: 508
2011-06-23 16:56:10 -07:00
epriestley
8f63873d57 Ping conduit from PHD and set daemon logfiles. 2011-06-13 18:39:23 -07:00
Jason Ge
f2efdd07a7 Use DatabaseConfigurationProvider to get DB info
Summary:
remove accessing the db config info directly. Use
DatabaseConfigurationProvider instead. Also fixed a minor issue where
different number of newlines are output in PhabricatorSetup.php's output.

Test Plan:
executed upgrade_schema.php; executed PhabricatorSetup.php by
setting 'phabricator.setup' to true.

Reviewed By: epriestley
Reviewers: epriestley
CC: aran, jungejason, epriestley
Differential Revision: 443
2011-06-13 14:59:28 -07:00
epriestley
cb0cbc50ad Add a "phd debug" command
Summary:
Make it easier to find obvious problems in daemons by letting them run
undaemonized in the console without requiring the user to know the magical
incantations for loading libraries, etc.

Test Plan:
Ran "phd debug nice", simulated some failures (e.g., bringing down Phabricator,
daemon fatal) and got useful error messages.

Reviewed By: jungejason
Reviewers: toulouse, jungejason, tuomaspelkonen, aran
CC: aran, jungejason
Differential Revision: 448
2011-06-13 13:42:01 -07:00
epriestley
d710fc097f Improve messaging and setup process for configuring outbound email.
Summary:
- Make the instructional text generally more useful.
  - Show the current configured adapter.
  - When the configuration prevents outbound email from being delivered, show a
warning.
  - Detect 'curl' extension during setup since it's more-or-less required
  - Add curl extension to the install scripts

codeblock: can you verify the rhel-derivs changes are correct?

Test Plan:
Set adapter to test, verified warning; entered setup mode and verified curl. Ran
apt-get on an ubuntu box. Ran yum on an amazon linux box.

Reviewed By: toulouse
Reviewers: toulouse, codeblock
Commenters: codeblock
CC: aran, jungejason, tuomaspelkonen, codeblock, epriestley, toulouse
Differential Revision: 438
2011-06-12 22:40:18 -07:00
epriestley
fedbd475b5 Create phd PID directory if it doesn't exist, and give the user a meaningful
error message if that doesn't work

Summary:
This workflow is needlessly bad right now, make it not terrible.

Also removed some related, unused code.

Test Plan:
Ran 'phd launch nice' with no directory and with a failing mkdir command.

Reviewed By: toulouse
Reviewers: hsb, toulouse, codeblock
CC: aran, toulouse
Differential Revision: 440
2011-06-12 22:40:07 -07:00
epriestley
8ca5581a9c Skip attaching 'inline' text attachments
Summary:
Mail clients can send messages where the body is represented as 'inline'
attachments. Don't treat any such text attachments as actual attachments.

Test Plan:
toulouse, can you verify this fixes the issue?

Reviewed By: toulouse
Reviewers: toulouse
CC: aran, toulouse, epriestley
Differential Revision: 441
2011-06-12 22:38:57 -07:00
epriestley
49d6854f95 Document how to set a MySQL port
Summary:
This already pretty much works, document it explicitly.

Test Plan:
Moved my MySQL server over to port 3307.

Reviewed By: aran
Reviewers: jungejason, aran
CC: aran
Differential Revision: 411
2011-06-08 10:29:57 -07:00