1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2025-01-25 14:08:19 +01:00
Commit graph

2172 commits

Author SHA1 Message Date
Nick Pellegrino
f0682941b6 Testing that PhabricatorEnv::getEnvConfig throws an exception if config option is not found.
Summary: Unit test for T2345

Test Plan: Ran unit tests, checked that it passed.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4552
2013-01-19 17:05:45 -08:00
Nick Pellegrino
3e6fa43658 getConfigEnv fails fast when key is not found and no default value is given.
Summary:
T2345
getConfig throws an Exception when the key does not exist.
Also removes dead code that throws an Exception.

Test Plan:
Reloaded the Phabricator home page.  In the process, found
2 Exceptions thrown due to nonexistent keys.  After addressing these problems,
the home page loads without Exceptions.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4541
2013-01-19 12:11:28 -08:00
vrana
dd5da0fedb Handle errors in reading cache
Summary: I've stored `PhutilSafeHTML` instance to cache on devbox and then wasn't able to read it in production.

Test Plan: Displayed revision with unreadable cache, saw error in error log but not fatal.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4520
2013-01-19 09:32:38 -08:00
epriestley
a8bd1f4998 Move Facebook setup checks to post-install
Summary: Technically we should have these for all the OAuth providers but I don't think anyone really has trouble with them and it can probably be done generically after T1536. Preserve the functionality, at least.

Test Plan: Broke my config, verified warnings appeared.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2228

Differential Revision: https://secure.phabricator.com/D4506
2013-01-19 08:42:15 -08:00
epriestley
e6b7ad2e00 Migrate max_allowed_packet and GD checks to new setup stuff
Summary: These are nonblocking warnings and can move to post-install.

Test Plan: Broke my environment and observed the warnings.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran, asherkin

Maniphest Tasks: T2228

Differential Revision: https://secure.phabricator.com/D4505
2013-01-19 08:41:45 -08:00
epriestley
14c6df0605 Migrate local disk storage setup check
Summary:
Migrate to the new hotness. Also:

  - Remove a string test, which is now impossible since the config will repair itself and raise a type error.
  - Restore the header even in /config/ -- this check is kind of hacky and it feels a bit more natural now that it's above the menu.

Test Plan: Set my local disk path to something invalid, verified I got a setup error.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2228

Differential Revision: https://secure.phabricator.com/D4497
2013-01-19 08:39:27 -08:00
epriestley
d5c29e1135 Make timezone configuration impossible to get wrong
Summary: Fixes T2269. If the user manages to mess up both the PHP and Phabricator configurations, set the timezone to UTC. We basically never use this anyway (we always render into the user's time), PHP just gets angry at us if we don't set it. (We do use it for logged-out users, I suppose.)

Test Plan: Set PHP and Phabricator timezones to goofy nonsense, verified we recover sensibly from it.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2228, T2269

Differential Revision: https://secure.phabricator.com/D4496
2013-01-19 08:38:37 -08:00
Asher Baker
da9315b145 Read default values of custom config options
Summary: Because the Default configuration provider is loaded before custom libraries, any config options specified in them don't get a default values.

Test Plan: Looked at /config/

Reviewers: epriestley, codeblock, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4532
2013-01-19 08:36:08 -08:00
John Watson
ff53b7942a Initial PhabricatorIRCFeedNotificationHandler
Summary:
Follows Phabricator's feed and puts notifications into channels
that are configured.

~~notification.all - bool - 1:1 stories to messages~~
notification.types - array - Specific story types to notify for - ["differential", "maniphest"]
notification.verbosity - int - Range of 0-3 for verbosity
notification.max_pages - int - Maximum number of pages to go back per poll
notification.page_size - int - Size of pages (limit) to poll
~~notification.channels - array - Array of channels to send messages to~~
~~notification.sleep - int - Seconds to sleep between polls~~

Test Plan: Run phabot with various configuration options

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, asherkin

Differential Revision: https://secure.phabricator.com/D4418
2013-01-19 05:45:17 -08:00
epriestley
08bca2a87e Move mail-related setup issues to setup checks
Summary: Ports mail stuff from the existing setup process to the more modular setup checks.

Test Plan: Configured my local install to have all these errors, verified setup raised them.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2228

Differential Revision: https://secure.phabricator.com/D4494
2013-01-18 13:28:30 -08:00
epriestley
9e2ec82094 Minor, fix initialization order so config initialization does not fatal on phlog(). 2013-01-18 11:29:18 -08:00
epriestley
0e36c802c5 Apply a hack to unbreak bin/storage upgrade across the DB config boundary
Summary: This is gross, but fixes an issue where `bin/storage upgrade` tries to access DB config which doesn't exist yet. We need a version of this for `bin/config` anyway. I'll sort this out into a proper sequenced startup process in a followup.

Test Plan: `bin/storage upgrade` no longer fatals when upgrading across the config boundary.

Reviewers: asherkin, codeblock, btrahan, vrana

Reviewed By: codeblock

CC: aran

Differential Revision: https://secure.phabricator.com/D4512
2013-01-18 07:50:52 -08:00
epriestley
5beaafb952 Load libraries before adding database config
Summary:
If your configuration overrides the connection adapter, we need to load libraries before we can setup the database config source.

Also lock this since it won't work when edited from the web anymore, and so sneaky users can't upload stuff and then edit their config to run arbitrary code.

Test Plan: See chatlog in #phabricator. This is a problem for Facebook only.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4498
2013-01-17 18:59:58 -08:00
epriestley
2d74f9ec76 Show a warning banner to administrators when there are unresolved setup warnings
Summary: We now issue only valid setup warnings, so we can let administrators know when we detect problems.

Test Plan:
Banner:

{F29568}

Created a fake issue; saw banner. No banner inside /config/. Resovled the issue, banner went away.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2221

Differential Revision: https://secure.phabricator.com/D4493
2013-01-17 16:25:47 -08:00
epriestley
b0d815d157 Repair invalid configuration by setting values back to defaults
Summary:
When configuration is set incorrectly (e.g., of the wrong type), detect and repair it by setting it to the default value. A setup warning will be raised separately.

Notably, this removes the need to hard-code all the class types.

This runs separately from the "invalid config" check because we need to run it on every page, but do setup checks only once per restart (some of them are slow).

Also dirty setup when we edit configuration.

Test Plan: Set config incorrectly on purpose, saw Phabricator correct it on restart and on every subsequent page load until it was fixed.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2292

Differential Revision: https://secure.phabricator.com/D4492
2013-01-17 16:25:38 -08:00
epriestley
74cb7a8971 Add database configuration source to the source stack
Summary:
Read configuration from the new database source.

This adds an extra MySQL connect + query to every page. They're very cheap so I think we can suffer them for now, but I'd like to put cache in front of this at some point. The difficulties are:

  - If we use APC, multi-frontend installs (Facebook) can't dirty it (major problem), and the CLI can't dirty it (fine for now, maybe a major problem later).
  - If we use Memcache, we need to add config stuff.
  - We could use APC in all non-Facebook installs if we can make it dirtyable from the CLI, but I don't see a reasonable way to do that.
  - We don't have any other caches which are faster than the database.

So I'll probably implement Memcache support at some point, although this is a lame excuse for it.

Test Plan: Added some config values via web UI, saw them active on the install.

Reviewers: btrahan, codeblock, vrana

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2221

Differential Revision: https://secure.phabricator.com/D4296
2013-01-17 15:10:21 -08:00
vrana
f896696fde Don't pop invalid test environment
Summary: If `unset($env)` throws then we pop some other environment instead which is impossible to pop later.

Test Plan:
  $ arc unit src/infrastructure/env/__tests__ src/applications/calendar/storage/__tests__

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4488
2013-01-17 12:01:46 -08:00
vrana
aa3e95cbeb Cache connection in bin/storage
Summary: Connection takes .3s from dev server to master.

Test Plan:
  $ bin/storage --trace upgrade --namespace x
  $ bin/storage --trace destroy --namespace x

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4480
2013-01-17 11:00:36 -08:00
vrana
9f549ba75e Fix whitespace around else 2013-01-16 12:16:37 -08:00
epriestley
d905d3cb2a Sprite menu label and hover states
Summary: These can all fit into the gradient sprite.

Test Plan: Looked at menu with selected item, hovered over menu.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4453
2013-01-15 15:49:48 -08:00
John Watson
ec19c3332a Break IRCSymbolHandler from IRCObjectNameHandler
Summary: Allows to easily disable responding to "where is..."

Test Plan: Run ircbot with and without the handler

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4444
2013-01-15 13:17:40 -08:00
epriestley
cbe9aea876 Enable log discard modes for all scripts
Summary:
Fixes T2273. We currently discard logs, service calls, etc., for daemons, but not for other scripts. However, other scripts may be long-running or issue a large body of service calls (e.g., `bin/search index --all`). We never retrieve this information from scripts (it is used to build darkconsole; in scripts, we echo it immediately under --trace), so discard it immediately to prevent these scripts from requiring a large amount of memory.

(When the daemons load `__init_script__.php` they end up calling this code, so this doesn't change anything for them. They hit another ServiceProfiler discard along the daemon pathways in libphutil, but the call is idempotent.)

Test Plan: Ran `bin/search index --all` and saw increasing memory usage before this patch, but steady memory usage after this patch.

Reviewers: btrahan, vrana, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2273

Differential Revision: https://secure.phabricator.com/D4364
2013-01-08 15:54:08 -08:00
Mailson Menezes
712e22208c Store width and height metadata of image files
Summary: Also provide a way to update old files metadata.

Test Plan: Create a revision which includes a image file. Check whether the widht, height metadata exists. Run `scripts/files/manage_files.php metadata --all` to update previously uploaded files.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2101

Differential Revision: https://secure.phabricator.com/D4347
2013-01-07 09:46:43 -08:00
vrana
7fe5f218ba Translate setup issues
Test Plan:
  $ php -l PhabricatorBaseEnglishTranslation.php

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4345
2013-01-03 19:05:13 -08:00
Bob Trahan
3448781c40 de-duplicate emails received by phabricator multiple times
Summary: this can happen if you have Phabricator and email lists co-mingling such that Phabricator receives an email multiple times. we can prevent this from then spamming everyone or otherwise taking the action multiple times by storing a message id hash and dropping the message if we have more than one message that matches.

Test Plan: simulated sending the same email multiple times on the command line. noted only the first one made it through.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1726

Differential Revision: https://secure.phabricator.com/D4328
2013-01-03 17:04:30 -08:00
Ricky Elrod
f12af03836 Make phabot never private message anyone.
Summary:
This replaces D4175 and makes it so phabot doesn't message anyone.

The reasons for this are twofold:

- It was possible to get information from the bot, by private messaging it, even
  if the bot was only in a +i channel (on a public network) -- meaning that if
  someone knew the nickname of the bot, they could obtain e.g. ticket names
  or diff titles.
- The other time it messaged people was when you typed e.g. "somenick: T123".
  Most times when this is triggered, it's done so on accident.

See discussion on the old revision (D4175).

Test Plan:
  15:29:33 ::: Irssi: Starting query in quartz with cb-phabot
  15:29:38 <relrod> T2
  (nothing back)

and

  15:29:21 <@relrod> rublets: T1
  15:29:21 < cb-phabot> T1: asdfasdf (Priority: Needs Triage) - http://local.elrod.me/T1

Reviewers: epriestley, btrahan, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4339
2013-01-03 12:44:36 -08:00
epriestley
0902543fc8 Port MySQL settings to PHP
Summary:
  - Ports MySQL settings to PHP.
  - Removes "mysql.retries" -- this existed only because Magic Numbers Are Bad, but there is no concievable reason it should ever be set to anything other than 3.
  - Introduced "Hidden" config, which isn't visible from the web (for SaaS, we'll just mark anything with secret keys as "hidden").
  - Introduced "Masked" config, which will be masked in darkconsole once that gets updated.
  - "Hidden" implies "Masked" and "Locked".
  - Moved "storage.default-namespace" here -- it probably makes more sense than core; this was my bad in T2255.
  - Put cancel button back for hidden/locked config.
  - Introduce 'class' config type.

Test Plan: Viewed MySQL options. None are editable.

Reviewers: codeblock, btrahan

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4326
2013-01-03 06:01:14 -08:00
Bob Trahan
2f6104bf8b add code to drop old databases
Summary: these existed once, are no more, and don't get cleaned up in the current code path

Test Plan: storage destroy --dryrun -- noted the correct database names

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2237

Differential Revision: https://secure.phabricator.com/D4329
2013-01-02 19:02:31 -08:00
epriestley
ea52c30bca Fix issue where "{Fnnn, size}" renders as though "size=full"
Summary: We interpret "size" as "size => true", and "true == 'full'", so we hit the wrong branch in the switch(). String cast explicitly.

Test Plan: Typed `{Fnnn, size}`; saw it render as a thumb instead of full.

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: frozendevil, aran

Differential Revision: https://secure.phabricator.com/D4323
2013-01-02 14:03:29 -08:00
epriestley
a3fdb20a8e Move GC into PHP and simplify it
Summary:
  - Move GC options into PHP.
  - Remove the "run at" and "run for" options. The GC daemon doesn't actually do any table scans, is very gentle, and runs for like 3 seconds per day in any normal install. Just limit it to running once every 4 hours when it's caught up and call it a day.

Test Plan: Edited GC options.

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4321
2013-01-02 14:03:08 -08:00
epriestley
32e4a7a37f Use transactions to show edit history for Configuration
Summary: Use ApplicationTransactions in Config to create an edit history. Resolves T2256.

Test Plan: {F28477}

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2256

Differential Revision: https://secure.phabricator.com/D4314
2013-01-01 18:14:41 -08:00
epriestley
21efc7cb64 Show all configuration defaults when editing configuration
Summary: Show the value for all loaded configuration sources.

Test Plan:
{F28469}

{F28470}

{F28471}

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4312
2013-01-01 14:10:33 -08:00
epriestley
ba489f9d85 Add a local configuration source and a non-environmental ENV config source
Summary:
See discussion in T2221. Before we can move configuration to the database, we have a bootstrapping problem: we need database credentials to live //somewhere// if we can't guess them (and we can only really guess localhost / root / no password).

Some options for this are:

  - Have them live in ENV variables.
    - These are often somewhat unfamiliar to users.
    - Scripts would become a huge pain -- you'd have to dump a bunch of stuff into ENV.
    - Some environments have limited ability to set ENV vars.
    - SSH is also a pain.
  - Have them live in a normal config file.
    - This probably isn't really too awful, but:
    - Since we deploy/upgrade with git, we can't currently let them edit a file which already exists, or their working copy will become dirty.
    - So they have to copy or create a file, then edit it.
    - The biggest issue I have with this is that it will be difficult to give specific, easily-followed directions from Setup. The instructions need to be like "Copy template.conf.php to real.conf.php, then edit these keys: x, y, z". This isn't as easy to follow as "run script Y".
  - Have them live in an abnormal config file with script access (this diff).
    - I think this is a little better than a normal config file, because we can tell users 'run phabricator/bin/config set mysql.user phabricator' and such, which is easier to follow than editing a config file.

I think this is only a marginal improvement over a normal config file and am open to arguments against this approach, but I think it will be a little easier for users to deal with than a normal config file. In most cases they should only need to store three values in this file -- db user/host/pass -- since once we have those we can bootstrap everything else. Normal config files also aren't going away for more advanced users, we're just offering a simple alternative for most users.

This also adds an ENVIRONMENT file as an alternative to PHABRICATOR_ENV. This is just a simple way to specify the environment if you don't have convenient access to env vars.

Test Plan: Ran `config set x y`, verified writes. Wrote to ENVIRONMENT, ran `PHABRICATOR_ENV= ./bin/repository`.

Reviewers: btrahan, vrana, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2221

Differential Revision: https://secure.phabricator.com/D4294
2012-12-30 06:16:15 -08:00
Ricky Elrod
a774620042 Start of a config web interface.
Summary:
This is somewhat clowny, particularly in how it handles JSON encode/decode, but
I've commented why I did things the way I did. The goal is to store minified JSON
but show pretty-printed JSON where possible, to the user editing it.

Test Plan:
* Went to /config/ and saw a list of keys from the `default` config.
* Clicked on one of them, submitted the default value successfully.
* Changed the value to invalid JSON and got a decent error.
* Changed the value to valid JSON and checked the DB to confirm it saved.
* Confirmed the DB values were minified.
* Confirmed the user-facing values were pretty-printed where they could be.
* Confirmed that PHIDs were getting assigned properly and that isDeleted
  properly defaulted to false/0.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2246

Differential Revision: https://secure.phabricator.com/D4290
2012-12-27 15:21:21 -08:00
epriestley
19b2c3d3d0 Formalize configuration sources and source stacks
Summary: Currently, we have a configuration stack for unit tests, but they're built in to `PhabricatorEnv`. Pull them out and formalize them, so we can add more configuration sources (e.g., database).

Test Plan: Ran unit tests, web requests, scripts. This code had fairly good existing test coverage.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2223, T2221

Differential Revision: https://secure.phabricator.com/D4284
2012-12-25 06:44:29 -08:00
epriestley
9e6d59829c Consolidate environmental initialization
Summary:
We have a bunch of code duplication now between __init_script__.php and webroot/index.php. Consoldiate these methods and move them into PhabricatorEnv.

Merge PhabricatorRequestOverseer into PhabricatorStartup.

Test Plan: Loaded page, ran script. Wiped PHABRICATOR_ENV; loaded page, ran script; got errors.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2223

Differential Revision: https://secure.phabricator.com/D4283
2012-12-25 06:15:28 -08:00
epriestley
ed58f6c5f4 Move a lot of pre-request checks to PhabricatorStartup
Summary:
We have a lot of mess to get through before we can load libphutil and enter Phabricator code properly. Move it to a dedicated class.

I'm probably going to merge PhabricatorRequestOverseer into this, although the check that lives there now is kind of weird. It also does not really need to be a pre-load check and could be handled better.

I stopped shoving stuff in here once I got to ENV stuff, I'm going to tackle that next.

Test Plan: Ran phabricator normally; introduced fatals and misconfigurations. Grepped for changed symbols.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, asherkin

Maniphest Tasks: T2223

Differential Revision: https://secure.phabricator.com/D4282
2012-12-25 06:11:39 -08:00
vrana
1c2e7e5daa Fix Paste translation
Summary: This syntax is extremely stupid, I'll probably rewrite it later.

Test Plan:
  pht('%s Line(s)', number_format(1000));

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4272
2012-12-21 15:39:25 -08:00
epriestley
f6b1964740 Improve Search architecture
Summary:
The search indexing API has several problems right now:

  - Always runs in-process.
    - It would be nice to push this into the task queue for performance. However, the API currently passses an object all the way through (and some indexers depend on preloaded object attributes), so it can't be dumped into the task queue at any stage since we can't serialize it.
    - Being able to use the task queue will also make rebuilding indexes faster.
    - Instead, make the API phid-oriented.
  - No uniform indexing API.
    - Each "Editor" currently calls SomeCustomIndexer::indexThing(). This won't work with AbstractTransactions. The API is also just weird.
    - Instead, provide a uniform API.
  - No uniform CLI.
    - We have `scripts/search/reindex_everything.php`, but it doesn't actually index everything. Each new document type needs to be separately added to it, leading to stuff like D3839. Third-party applications can't provide indexers.
    - Instead, let indexers expose documents for indexing.
  - Not application-oriented.
    - All the indexers live in search/ right now, which isn't the right organization in an application-orietned view of the world.
    - Instead, move indexers to applications and load them with SymbolLoader.

Test Plan:
  - `bin/search index`
    - Indexed one revision, one task.
    - Indexed `--type TASK`, `--type DREV`, etc., for all types.
    - Indexed `--all`.
  - Added the word "saboteur" to a revision, task, wiki page, and question and then searched for it.
    - Creating users is a pain; searched for a user after indexing.
    - Creating commits is a pain; searched for a commit after indexing.
    - Mocks aren't currently loadable in the result view, so their indexing is moot.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: 20after4, aran

Maniphest Tasks: T1991, T2104

Differential Revision: https://secure.phabricator.com/D4261
2012-12-21 14:21:31 -08:00
epriestley
aae5f9efd3 Implement a more compact, general database-backed key-value cache
Summary:
See discussion in D4204. Facebook currently has a 314MB remarkup cache with a 55MB index, which is slow to access. Under the theory that this is an index size/quality problem (the current index is on a potentially-384-byte field, with many keys sharing prefixes), provide a more general index with fancy new features:

  - It implements PhutilKeyValueCache, so it can be a component in cache stacks and supports TTL.
  - It has a 12-byte hash-based key.
  - It automatically compresses large blocks of data (most of what we store is highly-compressible HTML).

Test Plan:
  - Basics:
    - Loaded /paste/, saw caches generate and save.
    - Reloaded /paste/, saw the page hit cache.
  - GC:
    - Ran GC daemon, saw nothing.
    - Set maximum lifetime to 1 second, ran GC daemon, saw it collect the entire cache.
  - Deflate:
    - Selected row formats from the database, saw a mixture of 'raw' and 'deflate' storage.
    - Used profiler to verify that 'deflate' is fast (12 calls @ 220us on my paste list).
  - Ran unit tests

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4259
2012-12-21 14:17:56 -08:00
epriestley
4af2e3c4e2 Add PhabricatorHash::digestForIndex()
Summary: Does this seem reasonable? It's a bit more compact than digest() (6 bits / byte instead of 4 bits / byte) and 72 bits of entropy @ 12 bytes instead of 128 bits of entropy @ 32 bytes. I feel like it's important to preserve the printability, though, and this seemed like a fairly good balance of concerns.

Test Plan: unit tests

Reviewers: vrana

Reviewed By: vrana

CC: aran, yemao932

Differential Revision: https://secure.phabricator.com/D4253
2012-12-21 05:43:33 -08:00
epriestley
e78898970a Implement SSHD glue and Conduit SSH endpoint
Summary:
  - Build "sshd-auth" (for authentication) and "sshd-exec" (for command execution) binaries. These are callable by "sshd-vcs", located [[https://github.com/epriestley/sshd-vcs | in my account on GitHub]]. They are based on precursors [[https://github.com/epriestley/sshd-vcs-glue | here on GitHub]] which I deployed for TenXer about a year ago, so I have some confidence they at least basically work.
    - The problem this solves is that normally every user would need an account on a machine to connect to it, and/or their public keys would all need to be listed in `~/.authorized_keys`. This is a big pain in most installs. Software like Gitosis/Gitolite solve this problem by giving you an easy way to add public keys to `~/.authorized_keys`, but this is pretty gross.
    - Roughly, instead of looking in `~/.authorized_keys` when a user connects, the patched sshd instead runs `echo <public key> | sshd-auth`. The `sshd-auth` script looks up the public key and authorizes the matching user, if they exist. It also forces sshd to run `sshd-exec` instead of a normal shell.
    - `sshd-exec` receives the authenticated user and any command which was passed to ssh (like `git receive-pack`) and can route them appropriately.
    - Overall, this permits a single account to be set up on a server which all Phabricator users can connect to without any extra work, and which can safely execute commands and apply appropriate permissions, and disable users when they are disabled in Phabricator and all that stuff.
  - Build out "sshd-exec" to do more thorough checks and setup, and delegate command execution to Workflows (they now exist, and did not when I originally built this stuff).
  - Convert @btrahan's conduit API script into a workflow and slightly simplify it (ConduitCall did not exist at the time it was written).

The next steps here on the Repository side are to implement Workflows for Git, SVN and HG wire protocols. These will mostly just proxy the protocols, but also need to enforce permissions. So the approach will basically be:

  - Implement workflows for stuff like `git receive-pack`.
  - These workflows will implement enough of the underlying protocol to determine what resource the user is trying to access, and whether they want to read or write it.
  - They'll then do a permissons check, and kick the user out if they don't have permission to do whatever they are trying to do.
  - If the user does have permission, we just proxy the rest of the transaction.

Next steps on the Conduit side are more simple:

  - Make ConduitClient understand "ssh://" URLs.

Test Plan: Ran `sshd-exec --phabricator-ssh-user epriestley conduit differential.query`, etc. This will get a more comprehensive test once I set up sshd-vcs.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603, T550

Differential Revision: https://secure.phabricator.com/D4229
2012-12-19 11:08:07 -08:00
Ricky Elrod
93c4030732 Fix IRC bot Dxxx Conduit call.
Summary: Some fallout from D4191.

Test Plan: Sent "D12" in IRC and got a response.

Reviewers: epriestley, vrana, zeeg

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4235
2012-12-18 20:41:52 -08:00
Nick Harper
4a81ae6d6d Add data information to daemon task view
Summary:
Load the data for daemon worker tasks when viewing them, and present
the information in a useful way. This defaults to printing the json data,
but for some classes of worker it will also link to the corresponding
object, to make debugging problems with workers easier.

Test Plan:
load /daemon/task/NNN for a CommitParserWorker and a MetaMTAWorker, and
see the addition of a data field with useful content and link.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4226
2012-12-17 17:12:55 -08:00
epriestley
7e37eb4827 Provide a highlighter cache for Paste
Summary:
D4188 adds a preview of Paste contents to the list, but gets slow for large lists if you have Pygments installed since it has to spend ~200ms/item highlighting them. Instead, cache the highlighted output.

  - Adds a Paste highlighting cache. This uses RemarkupCache because it has automatic GC and we don't have a more generalized cache for the moment.
  - Uses the Paste cache on paste list and paste detail.
  - Adds a little padding to the summary.
  - Adds "..." if there's more content.
  - Adds line count and language, if available. These are hidden on Mobile.
  - Nothing actually uses needRawContent() but I left it there since it doesn't hurt anything and is used internally (I thought the detail view did, but it uses the file content directly, and must for security reasons).

Test Plan:
{F27710}

  - Profiled paste list, saw good performance and few service calls.
  - Viewed paste.
  - Viewed raw paste content.

Reviewers: codeblock, btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D4204
2012-12-16 16:33:42 -08:00
epriestley
4c7c518c63 Throw a richer exception when updating tasks with expired leases
Summary: Include task ID and class when raising this exception. I took a brief stab at doing this generically, but (a) we specifically raise this exception outside of normal try/catch because we can't follow normal recovery rules for it and (b) we don't have a reasonable PhutilProxyException or similar right now which would preserve stack traces, and don't have builtin exception nesting support until PHP 5.3.

Test Plan: Faked this exception, verified we get more information in the logs.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2193

Differential Revision: https://secure.phabricator.com/D4205
2012-12-16 16:27:32 -08:00
vrana
b05b9f2408 Link hashes in Remarkup commit links
Summary: Also generalize the code a bit.

Test Plan:
rPF57bc7f#inline-874
rPF57bc7f#874

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4168
2012-12-14 17:50:50 -08:00
epriestley
4041a7e0f6 Add ApplicationTransaction handling for transactions with no effect
Summary:
When a user submits an action with no effect (like an empty comment, an "abandon" on an already-accepted revision, or a "close, resolved" on a closed task) we want to alert them that their action isn't effective. These warnings fall into two general buckets:

  - User is submitting two or more actions, and some aren't effective but some are. Prompt them to apply the effective actions only.
    - A special case of this is where the only effective action is a comment. We provide tailored text ("Post Comment") in this case.
  - User is submitting one action, which isn't effective. Tell them they're out of luck.
    - A special case of this is an empty comment. We provide tailored text in this case.

By default, the transaction editor throws when transactions have no effect. The caller can then deal with this, or use `PhabricatorApplicationTransactionNoEffectResponse` to provide a standard dialog that gives the user information as above. For cases where we expect transactions to have no effect (notably, "edit" forms) we just continue on no-effect unconditionally.

Also fix an issue where new, combined or filtered transactions would not be represented properly in the Ajax response (i.e., return final transactions from `applyTransactions()`), and translate some strings.

Test Plan:
  - Submitted empty and nonempy comments in Macro and Pholio.
  - Submitted comments with new and existing "@mentions".
  - Submitted edits in both applications.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T912, T2104

Differential Revision: https://secure.phabricator.com/D4160
2012-12-11 17:27:40 -08:00
epriestley
dd669c6d4e Make AphrontProxyResponse reduce to a real response instead of building a string
Summary: Currently, AphrontProxyResponse is expected to build a string. This prevents some response types (like Dialog) from being proxied, because they have special rules. Instead, make proxy responses reduce into a non-proxied response so it's possible to proxy any type of response and hit all the normal rules for it.

Test Plan: Built a proxied DialogResponse on top of this.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2104, T912

Differential Revision: https://secure.phabricator.com/D4159
2012-12-11 17:27:25 -08:00
epriestley
ba7723d905 Modernize Macro application
Summary: Adds feed, email, notifications, comments, partial editing, subscriptions, enable/disable, flags and crumbs to Macro.

Test Plan:
{F26839}
{F26840}
{F26841}
{F26842}
{F26843}
{F26844}
{F26845}

Reviewers: vrana, btrahan, chad

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2157, T175, T2104

Differential Revision: https://secure.phabricator.com/D4141
2012-12-11 14:01:03 -08:00