1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-11 09:22:40 +01:00
Commit graph

476 commits

Author SHA1 Message Date
epriestley
809f380aa6 Fix FileTreeView sprites
Summary: D4453 and D4427 sailed past one another, like ships in the night.

Test Plan: Verified Differential hover and selected states.

Reviewers: asherkin, chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4577
2013-01-22 09:10:21 -08:00
epriestley
baa9d96082 Increase the power of bin/config
Summary:
Fixes T2254. Make the CLI for config more powerful:

  - Add validation for `set`.
  - Add `get`.
  - Add `list`.
  - Add `delete`.

The `get` command produces fairly verbose JSON to support flags like `--all`, or `--source database` later. The other commands are straightforward.

Test Plan:
Tested `config set`:

  $ ./bin/config set
  Usage Exception: Specify a configuration key and a value to set it to.
  $ ./bin/config set x
  Usage Exception: Specify a value to set the key 'x' to.
  $ ./bin/config set phabricator.base-uri
  Usage Exception: Specify a value to set the key 'phabricator.base-uri' to.
  $ ./bin/config set phabricator.base-uri x
  Usage Exception: Config option 'phabricator.base-uri' is invalid. The URI must start with 'http://' or 'https://'.
  $ ./bin/config set phabricator.base-uri http://x
  Usage Exception: Config option 'phabricator.base-uri' is invalid. The URI must contain a dot ('.'), like 'http://example.com/', not just a bare name like 'http://example/'. Some web browsers will not set cookies on domains with no TLD.
  $ ./bin/config set phabricator.base-uri http://x.com
  Set 'phabricator.base-uri' in local configuration.
  $

Tested `config get`:

  $ ./bin/config get pygments.enabled
  {
    "config" : []
  }
  $ ./bin/config set pygments.enabled true
  Set 'pygments.enabled' in local configuration.
  $ ./bin/config get pygments.enabled
  {
    "config" : [
      {
        "key"    : "pygments.enabled",
        "source" : "local",
        "value"  : true
      }
    ]
  }
  $

Tested `config delete`:

  $ ./bin/config delete
  Usage Exception: Specify a configuration key to delete.
  $ ./bin/config delete x x
  Usage Exception: Too many arguments: expected one key.
  $ ./bin/config delete x
  Usage Exception: No such configuration key 'x'! Use `config list` to list all keys.
  $ ./bin/config delete pygments.enabled
  Deleted 'pygments.enabled' from local configuration.
  $ ./bin/config delete pygments.enabled
  Usage Exception: Configuration key 'pygments.enabled' is not set in local configuration!
  $

Tested `config list`:

  $ ./bin/config list
  account.editable
  account.minimum-password-length
  amazon-ec2.access-key
  amazon-ec2.secret-key
  amazon-s3.access-key
  amazon-s3.endpoint
  amazon-s3.secret-key
  amazon-ses.access-key
  amazon-ses.secret-key
  aphront.default-application-configuration-class
  audit.can-author-close-audit
  auth.email-domains
  auth.login-message
  auth.password-auth-enabled
  auth.require-email-verification
  auth.sessions.conduit
  auth.sessions.web
  auth.sshkeys.enabled
  cache.enable-deflate
  celerity.force-disk-reads
  celerity.minify
  celerity.resource-hash
  celerity.resource-path
  config.hide
  config.lock
  config.mask
  controller.oauth-registration
  darkconsole.always-on
  darkconsole.enabled
  debug.profile-rate
  debug.stop-on-redirect
  differential.allow-reopen
  differential.allow-self-accept
  differential.always-allow-close
  differential.anonymous-access
  differential.custom-remarkup-block-rules
  differential.custom-remarkup-rules
  differential.days-fresh
  differential.days-stale
  differential.enable-email-accept
  differential.expose-emails-prudently
  differential.field-selector
  differential.generated-paths
  differential.require-test-plan-field
  differential.revision-custom-detail-renderer
  differential.show-host-field
  differential.show-test-plan-field
  differential.whitespace-matters
  disqus.application-id
  disqus.application-secret
  disqus.auth-enabled
  disqus.auth-permanent
  disqus.registration-enabled
  disqus.shortname
  environment.append-paths
  events.listeners
  facebook.application-id
  facebook.application-secret
  facebook.auth-enabled
  facebook.auth-permanent
  facebook.registration-enabled
  facebook.require-https-auth
  feed.http-hooks
  feed.public
  files.image-mime-types
  files.viewable-mime-types
  gcdaemon.ttl.daemon-logs
  gcdaemon.ttl.differential-parse-cache
  gcdaemon.ttl.general-cache
  gcdaemon.ttl.herald-transcripts
  gcdaemon.ttl.markup-cache
  gcdaemon.ttl.task-archive
  github.application-id
  github.application-secret
  github.auth-enabled
  github.auth-permanent
  github.registration-enabled
  google.application-id
  google.application-secret
  google.auth-enabled
  google.auth-permanent
  google.registration-enabled
  ldap.activedirectory_domain
  ldap.anonymous-user-name
  ldap.anonymous-user-password
  ldap.auth-enabled
  ldap.base_dn
  ldap.hostname
  ldap.port
  ldap.real_name_attributes
  ldap.referrals
  ldap.search-first
  ldap.search_attribute
  ldap.start-tls
  ldap.username-attribute
  ldap.version
  load-libraries
  log.access.format
  log.access.path
  maniphest.custom-fields
  maniphest.custom-task-extensions-class
  maniphest.default-priority
  maniphest.enabled
  metamta.can-send-as-user
  metamta.default-address
  metamta.differential.attach-patches
  metamta.differential.inline-patches
  metamta.differential.patch-format
  metamta.differential.reply-handler
  metamta.differential.reply-handler-domain
  metamta.differential.subject-prefix
  metamta.differential.unified-comment-context
  metamta.diffusion.attach-patches
  metamta.diffusion.byte-limit
  metamta.diffusion.inline-patches
  metamta.diffusion.reply-handler
  metamta.diffusion.reply-handler-domain
  metamta.diffusion.subject-prefix
  metamta.diffusion.time-limit
  metamta.domain
  metamta.herald.show-hints
  metamta.insecure-auth-with-reply-to
  metamta.macro.reply-handler-domain
  metamta.macro.subject-prefix
  metamta.mail-adapter
  metamta.maniphest.default-public-author
  metamta.maniphest.public-create-email
  metamta.maniphest.reply-handler
  metamta.maniphest.reply-handler-domain
  metamta.maniphest.subject-prefix
  metamta.one-mail-per-recipient
  metamta.package.reply-handler
  metamta.package.subject-prefix
  metamta.pholio.reply-handler-domain
  metamta.pholio.subject-prefix
  metamta.placeholder-to-recipient
  metamta.precedence-bulk
  metamta.public-replies
  metamta.re-prefix
  metamta.recipients.show-hints
  metamta.reply.show-hints
  metamta.send-immediately
  metamta.single-reply-handler-prefix
  metamta.user-address-format
  metamta.vary-subjects
  mysql.configuration-provider
  mysql.host
  mysql.implementation
  mysql.pass
  mysql.user
  notification.client-uri
  notification.debug
  notification.enabled
  notification.log
  notification.pidfile
  notification.server-uri
  notification.user
  phabricator.application-id
  phabricator.application-secret
  phabricator.auth-enabled
  phabricator.auth-permanent
  phabricator.base-uri
  phabricator.csrf-key
  phabricator.env
  phabricator.mail-key
  phabricator.oauth-uri
  phabricator.production-uri
  phabricator.registration-enabled
  phabricator.serious-business
  phabricator.setup
  phabricator.show-beta-applications
  phabricator.show-error-callout
  phabricator.show-stack-traces
  phabricator.timezone
  phame.skins
  phd.log-directory
  phd.pid-directory
  phd.start-taskmasters
  phd.trace
  phd.verbose
  phid.external-loaders
  phpmailer.mailer
  phpmailer.smtp-host
  phpmailer.smtp-password
  phpmailer.smtp-port
  phpmailer.smtp-protocol
  phpmailer.smtp-user
  phriction.enabled
  policy.allow-public
  pygments.dropdown-choices
  pygments.enabled
  recaptcha.enabled
  recaptcha.private-key
  recaptcha.public-key
  remarkup.enable-embedded-youtube
  repository.default-local-path
  search.elastic.host
  search.engine-selector
  security.alternate-file-domain
  security.hmac-key
  security.require-https
  sendgrid.api-key
  sendgrid.api-user
  storage.default-namespace
  storage.engine-selector
  storage.local-disk.path
  storage.mysql-engine.max-size
  storage.s3.bucket
  storage.upload-size-limit
  style.monospace
  syntax-highlighter.engine
  syntax.filemap
  test.value
  tokenizer.ondemand
  translation.override
  translation.provider
  uri.allowed-protocols
  $

Reviewers: btrahan, codeblock

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2254

Differential Revision: https://secure.phabricator.com/D4570
2013-01-21 15:27:42 -08:00
epriestley
08e61c6ff1 Phabricator changes for TOC markup
Summary: See T2333 / D4565. Fixes the Phriction rule.

Test Plan: See D4565.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2333

Differential Revision: https://secure.phabricator.com/D4566
2013-01-21 11:57:58 -08:00
epriestley
d5b006b2cc Remove pcntl/php CLI setup checks
Summary:
As far as I know, we never actually need `php` to be available from the web UI. I think the history here is:

  - Long ago, we checked for 'pcntl' as an extension during setup.
  - Someone had an install where 'pcntl' was available from the CLI, but not the web UI. So we switched the check to use the CLI.
  - Someone had an install where the CLI binary was php-fpm, which caused the 'pcntl' check to loop endlessly, so we added more checks.

But we don't actually need to do any of this -- when the user tries to run the daemons, they get an explicit message that they need to install pcntl already, and we never (as far as I know) try to run PHP scripts from the web UI other than the pcntl_available.php check (we only run `git`, `svn`, `hg`, `ssh-agent`, `diff`, `xhpast` and `pygmentize`, I think).

Test Plan: Thought carefully about places we might execute PHP scripts from the web UI. Looked through /scripts/ to try to identfiy anything we might execute.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D4568
2013-01-21 11:57:24 -08:00
Nick Pellegrino
f0682941b6 Testing that PhabricatorEnv::getEnvConfig throws an exception if config option is not found.
Summary: Unit test for T2345

Test Plan: Ran unit tests, checked that it passed.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4552
2013-01-19 17:05:45 -08:00
Nick Pellegrino
3e6fa43658 getConfigEnv fails fast when key is not found and no default value is given.
Summary:
T2345
getConfig throws an Exception when the key does not exist.
Also removes dead code that throws an Exception.

Test Plan:
Reloaded the Phabricator home page.  In the process, found
2 Exceptions thrown due to nonexistent keys.  After addressing these problems,
the home page loads without Exceptions.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4541
2013-01-19 12:11:28 -08:00
vrana
dd5da0fedb Handle errors in reading cache
Summary: I've stored `PhutilSafeHTML` instance to cache on devbox and then wasn't able to read it in production.

Test Plan: Displayed revision with unreadable cache, saw error in error log but not fatal.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4520
2013-01-19 09:32:38 -08:00
epriestley
a8bd1f4998 Move Facebook setup checks to post-install
Summary: Technically we should have these for all the OAuth providers but I don't think anyone really has trouble with them and it can probably be done generically after T1536. Preserve the functionality, at least.

Test Plan: Broke my config, verified warnings appeared.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2228

Differential Revision: https://secure.phabricator.com/D4506
2013-01-19 08:42:15 -08:00
epriestley
e6b7ad2e00 Migrate max_allowed_packet and GD checks to new setup stuff
Summary: These are nonblocking warnings and can move to post-install.

Test Plan: Broke my environment and observed the warnings.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran, asherkin

Maniphest Tasks: T2228

Differential Revision: https://secure.phabricator.com/D4505
2013-01-19 08:41:45 -08:00
epriestley
14c6df0605 Migrate local disk storage setup check
Summary:
Migrate to the new hotness. Also:

  - Remove a string test, which is now impossible since the config will repair itself and raise a type error.
  - Restore the header even in /config/ -- this check is kind of hacky and it feels a bit more natural now that it's above the menu.

Test Plan: Set my local disk path to something invalid, verified I got a setup error.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2228

Differential Revision: https://secure.phabricator.com/D4497
2013-01-19 08:39:27 -08:00
epriestley
d5c29e1135 Make timezone configuration impossible to get wrong
Summary: Fixes T2269. If the user manages to mess up both the PHP and Phabricator configurations, set the timezone to UTC. We basically never use this anyway (we always render into the user's time), PHP just gets angry at us if we don't set it. (We do use it for logged-out users, I suppose.)

Test Plan: Set PHP and Phabricator timezones to goofy nonsense, verified we recover sensibly from it.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2228, T2269

Differential Revision: https://secure.phabricator.com/D4496
2013-01-19 08:38:37 -08:00
Asher Baker
da9315b145 Read default values of custom config options
Summary: Because the Default configuration provider is loaded before custom libraries, any config options specified in them don't get a default values.

Test Plan: Looked at /config/

Reviewers: epriestley, codeblock, btrahan

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4532
2013-01-19 08:36:08 -08:00
John Watson
ff53b7942a Initial PhabricatorIRCFeedNotificationHandler
Summary:
Follows Phabricator's feed and puts notifications into channels
that are configured.

~~notification.all - bool - 1:1 stories to messages~~
notification.types - array - Specific story types to notify for - ["differential", "maniphest"]
notification.verbosity - int - Range of 0-3 for verbosity
notification.max_pages - int - Maximum number of pages to go back per poll
notification.page_size - int - Size of pages (limit) to poll
~~notification.channels - array - Array of channels to send messages to~~
~~notification.sleep - int - Seconds to sleep between polls~~

Test Plan: Run phabot with various configuration options

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin, asherkin

Differential Revision: https://secure.phabricator.com/D4418
2013-01-19 05:45:17 -08:00
epriestley
08bca2a87e Move mail-related setup issues to setup checks
Summary: Ports mail stuff from the existing setup process to the more modular setup checks.

Test Plan: Configured my local install to have all these errors, verified setup raised them.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2228

Differential Revision: https://secure.phabricator.com/D4494
2013-01-18 13:28:30 -08:00
epriestley
9e2ec82094 Minor, fix initialization order so config initialization does not fatal on phlog(). 2013-01-18 11:29:18 -08:00
epriestley
0e36c802c5 Apply a hack to unbreak bin/storage upgrade across the DB config boundary
Summary: This is gross, but fixes an issue where `bin/storage upgrade` tries to access DB config which doesn't exist yet. We need a version of this for `bin/config` anyway. I'll sort this out into a proper sequenced startup process in a followup.

Test Plan: `bin/storage upgrade` no longer fatals when upgrading across the config boundary.

Reviewers: asherkin, codeblock, btrahan, vrana

Reviewed By: codeblock

CC: aran

Differential Revision: https://secure.phabricator.com/D4512
2013-01-18 07:50:52 -08:00
epriestley
5beaafb952 Load libraries before adding database config
Summary:
If your configuration overrides the connection adapter, we need to load libraries before we can setup the database config source.

Also lock this since it won't work when edited from the web anymore, and so sneaky users can't upload stuff and then edit their config to run arbitrary code.

Test Plan: See chatlog in #phabricator. This is a problem for Facebook only.

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4498
2013-01-17 18:59:58 -08:00
epriestley
2d74f9ec76 Show a warning banner to administrators when there are unresolved setup warnings
Summary: We now issue only valid setup warnings, so we can let administrators know when we detect problems.

Test Plan:
Banner:

{F29568}

Created a fake issue; saw banner. No banner inside /config/. Resovled the issue, banner went away.

Reviewers: btrahan, chad

Reviewed By: chad

CC: aran

Maniphest Tasks: T2221

Differential Revision: https://secure.phabricator.com/D4493
2013-01-17 16:25:47 -08:00
epriestley
b0d815d157 Repair invalid configuration by setting values back to defaults
Summary:
When configuration is set incorrectly (e.g., of the wrong type), detect and repair it by setting it to the default value. A setup warning will be raised separately.

Notably, this removes the need to hard-code all the class types.

This runs separately from the "invalid config" check because we need to run it on every page, but do setup checks only once per restart (some of them are slow).

Also dirty setup when we edit configuration.

Test Plan: Set config incorrectly on purpose, saw Phabricator correct it on restart and on every subsequent page load until it was fixed.

Reviewers: btrahan, vrana

Reviewed By: vrana

CC: aran

Maniphest Tasks: T2292

Differential Revision: https://secure.phabricator.com/D4492
2013-01-17 16:25:38 -08:00
epriestley
74cb7a8971 Add database configuration source to the source stack
Summary:
Read configuration from the new database source.

This adds an extra MySQL connect + query to every page. They're very cheap so I think we can suffer them for now, but I'd like to put cache in front of this at some point. The difficulties are:

  - If we use APC, multi-frontend installs (Facebook) can't dirty it (major problem), and the CLI can't dirty it (fine for now, maybe a major problem later).
  - If we use Memcache, we need to add config stuff.
  - We could use APC in all non-Facebook installs if we can make it dirtyable from the CLI, but I don't see a reasonable way to do that.
  - We don't have any other caches which are faster than the database.

So I'll probably implement Memcache support at some point, although this is a lame excuse for it.

Test Plan: Added some config values via web UI, saw them active on the install.

Reviewers: btrahan, codeblock, vrana

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2221

Differential Revision: https://secure.phabricator.com/D4296
2013-01-17 15:10:21 -08:00
vrana
f896696fde Don't pop invalid test environment
Summary: If `unset($env)` throws then we pop some other environment instead which is impossible to pop later.

Test Plan:
  $ arc unit src/infrastructure/env/__tests__ src/applications/calendar/storage/__tests__

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4488
2013-01-17 12:01:46 -08:00
vrana
aa3e95cbeb Cache connection in bin/storage
Summary: Connection takes .3s from dev server to master.

Test Plan:
  $ bin/storage --trace upgrade --namespace x
  $ bin/storage --trace destroy --namespace x

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4480
2013-01-17 11:00:36 -08:00
vrana
9f549ba75e Fix whitespace around else 2013-01-16 12:16:37 -08:00
epriestley
d905d3cb2a Sprite menu label and hover states
Summary: These can all fit into the gradient sprite.

Test Plan: Looked at menu with selected item, hovered over menu.

Reviewers: chad

Reviewed By: chad

CC: aran

Differential Revision: https://secure.phabricator.com/D4453
2013-01-15 15:49:48 -08:00
John Watson
ec19c3332a Break IRCSymbolHandler from IRCObjectNameHandler
Summary: Allows to easily disable responding to "where is..."

Test Plan: Run ircbot with and without the handler

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4444
2013-01-15 13:17:40 -08:00
epriestley
cbe9aea876 Enable log discard modes for all scripts
Summary:
Fixes T2273. We currently discard logs, service calls, etc., for daemons, but not for other scripts. However, other scripts may be long-running or issue a large body of service calls (e.g., `bin/search index --all`). We never retrieve this information from scripts (it is used to build darkconsole; in scripts, we echo it immediately under --trace), so discard it immediately to prevent these scripts from requiring a large amount of memory.

(When the daemons load `__init_script__.php` they end up calling this code, so this doesn't change anything for them. They hit another ServiceProfiler discard along the daemon pathways in libphutil, but the call is idempotent.)

Test Plan: Ran `bin/search index --all` and saw increasing memory usage before this patch, but steady memory usage after this patch.

Reviewers: btrahan, vrana, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2273

Differential Revision: https://secure.phabricator.com/D4364
2013-01-08 15:54:08 -08:00
Mailson Menezes
712e22208c Store width and height metadata of image files
Summary: Also provide a way to update old files metadata.

Test Plan: Create a revision which includes a image file. Check whether the widht, height metadata exists. Run `scripts/files/manage_files.php metadata --all` to update previously uploaded files.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2101

Differential Revision: https://secure.phabricator.com/D4347
2013-01-07 09:46:43 -08:00
vrana
7fe5f218ba Translate setup issues
Test Plan:
  $ php -l PhabricatorBaseEnglishTranslation.php

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4345
2013-01-03 19:05:13 -08:00
Bob Trahan
3448781c40 de-duplicate emails received by phabricator multiple times
Summary: this can happen if you have Phabricator and email lists co-mingling such that Phabricator receives an email multiple times. we can prevent this from then spamming everyone or otherwise taking the action multiple times by storing a message id hash and dropping the message if we have more than one message that matches.

Test Plan: simulated sending the same email multiple times on the command line. noted only the first one made it through.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T1726

Differential Revision: https://secure.phabricator.com/D4328
2013-01-03 17:04:30 -08:00
Ricky Elrod
f12af03836 Make phabot never private message anyone.
Summary:
This replaces D4175 and makes it so phabot doesn't message anyone.

The reasons for this are twofold:

- It was possible to get information from the bot, by private messaging it, even
  if the bot was only in a +i channel (on a public network) -- meaning that if
  someone knew the nickname of the bot, they could obtain e.g. ticket names
  or diff titles.
- The other time it messaged people was when you typed e.g. "somenick: T123".
  Most times when this is triggered, it's done so on accident.

See discussion on the old revision (D4175).

Test Plan:
  15:29:33 ::: Irssi: Starting query in quartz with cb-phabot
  15:29:38 <relrod> T2
  (nothing back)

and

  15:29:21 <@relrod> rublets: T1
  15:29:21 < cb-phabot> T1: asdfasdf (Priority: Needs Triage) - http://local.elrod.me/T1

Reviewers: epriestley, btrahan, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4339
2013-01-03 12:44:36 -08:00
epriestley
0902543fc8 Port MySQL settings to PHP
Summary:
  - Ports MySQL settings to PHP.
  - Removes "mysql.retries" -- this existed only because Magic Numbers Are Bad, but there is no concievable reason it should ever be set to anything other than 3.
  - Introduced "Hidden" config, which isn't visible from the web (for SaaS, we'll just mark anything with secret keys as "hidden").
  - Introduced "Masked" config, which will be masked in darkconsole once that gets updated.
  - "Hidden" implies "Masked" and "Locked".
  - Moved "storage.default-namespace" here -- it probably makes more sense than core; this was my bad in T2255.
  - Put cancel button back for hidden/locked config.
  - Introduce 'class' config type.

Test Plan: Viewed MySQL options. None are editable.

Reviewers: codeblock, btrahan

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4326
2013-01-03 06:01:14 -08:00
Bob Trahan
2f6104bf8b add code to drop old databases
Summary: these existed once, are no more, and don't get cleaned up in the current code path

Test Plan: storage destroy --dryrun -- noted the correct database names

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2237

Differential Revision: https://secure.phabricator.com/D4329
2013-01-02 19:02:31 -08:00
epriestley
ea52c30bca Fix issue where "{Fnnn, size}" renders as though "size=full"
Summary: We interpret "size" as "size => true", and "true == 'full'", so we hit the wrong branch in the switch(). String cast explicitly.

Test Plan: Typed `{Fnnn, size}`; saw it render as a thumb instead of full.

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: frozendevil, aran

Differential Revision: https://secure.phabricator.com/D4323
2013-01-02 14:03:29 -08:00
epriestley
a3fdb20a8e Move GC into PHP and simplify it
Summary:
  - Move GC options into PHP.
  - Remove the "run at" and "run for" options. The GC daemon doesn't actually do any table scans, is very gentle, and runs for like 3 seconds per day in any normal install. Just limit it to running once every 4 hours when it's caught up and call it a day.

Test Plan: Edited GC options.

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4321
2013-01-02 14:03:08 -08:00
epriestley
32e4a7a37f Use transactions to show edit history for Configuration
Summary: Use ApplicationTransactions in Config to create an edit history. Resolves T2256.

Test Plan: {F28477}

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2256

Differential Revision: https://secure.phabricator.com/D4314
2013-01-01 18:14:41 -08:00
epriestley
21efc7cb64 Show all configuration defaults when editing configuration
Summary: Show the value for all loaded configuration sources.

Test Plan:
{F28469}

{F28470}

{F28471}

Reviewers: btrahan, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2255

Differential Revision: https://secure.phabricator.com/D4312
2013-01-01 14:10:33 -08:00
epriestley
ba489f9d85 Add a local configuration source and a non-environmental ENV config source
Summary:
See discussion in T2221. Before we can move configuration to the database, we have a bootstrapping problem: we need database credentials to live //somewhere// if we can't guess them (and we can only really guess localhost / root / no password).

Some options for this are:

  - Have them live in ENV variables.
    - These are often somewhat unfamiliar to users.
    - Scripts would become a huge pain -- you'd have to dump a bunch of stuff into ENV.
    - Some environments have limited ability to set ENV vars.
    - SSH is also a pain.
  - Have them live in a normal config file.
    - This probably isn't really too awful, but:
    - Since we deploy/upgrade with git, we can't currently let them edit a file which already exists, or their working copy will become dirty.
    - So they have to copy or create a file, then edit it.
    - The biggest issue I have with this is that it will be difficult to give specific, easily-followed directions from Setup. The instructions need to be like "Copy template.conf.php to real.conf.php, then edit these keys: x, y, z". This isn't as easy to follow as "run script Y".
  - Have them live in an abnormal config file with script access (this diff).
    - I think this is a little better than a normal config file, because we can tell users 'run phabricator/bin/config set mysql.user phabricator' and such, which is easier to follow than editing a config file.

I think this is only a marginal improvement over a normal config file and am open to arguments against this approach, but I think it will be a little easier for users to deal with than a normal config file. In most cases they should only need to store three values in this file -- db user/host/pass -- since once we have those we can bootstrap everything else. Normal config files also aren't going away for more advanced users, we're just offering a simple alternative for most users.

This also adds an ENVIRONMENT file as an alternative to PHABRICATOR_ENV. This is just a simple way to specify the environment if you don't have convenient access to env vars.

Test Plan: Ran `config set x y`, verified writes. Wrote to ENVIRONMENT, ran `PHABRICATOR_ENV= ./bin/repository`.

Reviewers: btrahan, vrana, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2221

Differential Revision: https://secure.phabricator.com/D4294
2012-12-30 06:16:15 -08:00
Ricky Elrod
a774620042 Start of a config web interface.
Summary:
This is somewhat clowny, particularly in how it handles JSON encode/decode, but
I've commented why I did things the way I did. The goal is to store minified JSON
but show pretty-printed JSON where possible, to the user editing it.

Test Plan:
* Went to /config/ and saw a list of keys from the `default` config.
* Clicked on one of them, submitted the default value successfully.
* Changed the value to invalid JSON and got a decent error.
* Changed the value to valid JSON and checked the DB to confirm it saved.
* Confirmed the DB values were minified.
* Confirmed the user-facing values were pretty-printed where they could be.
* Confirmed that PHIDs were getting assigned properly and that isDeleted
  properly defaulted to false/0.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Maniphest Tasks: T2246

Differential Revision: https://secure.phabricator.com/D4290
2012-12-27 15:21:21 -08:00
epriestley
19b2c3d3d0 Formalize configuration sources and source stacks
Summary: Currently, we have a configuration stack for unit tests, but they're built in to `PhabricatorEnv`. Pull them out and formalize them, so we can add more configuration sources (e.g., database).

Test Plan: Ran unit tests, web requests, scripts. This code had fairly good existing test coverage.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2223, T2221

Differential Revision: https://secure.phabricator.com/D4284
2012-12-25 06:44:29 -08:00
epriestley
9e6d59829c Consolidate environmental initialization
Summary:
We have a bunch of code duplication now between __init_script__.php and webroot/index.php. Consoldiate these methods and move them into PhabricatorEnv.

Merge PhabricatorRequestOverseer into PhabricatorStartup.

Test Plan: Loaded page, ran script. Wiped PHABRICATOR_ENV; loaded page, ran script; got errors.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2223

Differential Revision: https://secure.phabricator.com/D4283
2012-12-25 06:15:28 -08:00
epriestley
ed58f6c5f4 Move a lot of pre-request checks to PhabricatorStartup
Summary:
We have a lot of mess to get through before we can load libphutil and enter Phabricator code properly. Move it to a dedicated class.

I'm probably going to merge PhabricatorRequestOverseer into this, although the check that lives there now is kind of weird. It also does not really need to be a pre-load check and could be handled better.

I stopped shoving stuff in here once I got to ENV stuff, I'm going to tackle that next.

Test Plan: Ran phabricator normally; introduced fatals and misconfigurations. Grepped for changed symbols.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran, asherkin

Maniphest Tasks: T2223

Differential Revision: https://secure.phabricator.com/D4282
2012-12-25 06:11:39 -08:00
vrana
1c2e7e5daa Fix Paste translation
Summary: This syntax is extremely stupid, I'll probably rewrite it later.

Test Plan:
  pht('%s Line(s)', number_format(1000));

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4272
2012-12-21 15:39:25 -08:00
epriestley
f6b1964740 Improve Search architecture
Summary:
The search indexing API has several problems right now:

  - Always runs in-process.
    - It would be nice to push this into the task queue for performance. However, the API currently passses an object all the way through (and some indexers depend on preloaded object attributes), so it can't be dumped into the task queue at any stage since we can't serialize it.
    - Being able to use the task queue will also make rebuilding indexes faster.
    - Instead, make the API phid-oriented.
  - No uniform indexing API.
    - Each "Editor" currently calls SomeCustomIndexer::indexThing(). This won't work with AbstractTransactions. The API is also just weird.
    - Instead, provide a uniform API.
  - No uniform CLI.
    - We have `scripts/search/reindex_everything.php`, but it doesn't actually index everything. Each new document type needs to be separately added to it, leading to stuff like D3839. Third-party applications can't provide indexers.
    - Instead, let indexers expose documents for indexing.
  - Not application-oriented.
    - All the indexers live in search/ right now, which isn't the right organization in an application-orietned view of the world.
    - Instead, move indexers to applications and load them with SymbolLoader.

Test Plan:
  - `bin/search index`
    - Indexed one revision, one task.
    - Indexed `--type TASK`, `--type DREV`, etc., for all types.
    - Indexed `--all`.
  - Added the word "saboteur" to a revision, task, wiki page, and question and then searched for it.
    - Creating users is a pain; searched for a user after indexing.
    - Creating commits is a pain; searched for a commit after indexing.
    - Mocks aren't currently loadable in the result view, so their indexing is moot.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: 20after4, aran

Maniphest Tasks: T1991, T2104

Differential Revision: https://secure.phabricator.com/D4261
2012-12-21 14:21:31 -08:00
epriestley
aae5f9efd3 Implement a more compact, general database-backed key-value cache
Summary:
See discussion in D4204. Facebook currently has a 314MB remarkup cache with a 55MB index, which is slow to access. Under the theory that this is an index size/quality problem (the current index is on a potentially-384-byte field, with many keys sharing prefixes), provide a more general index with fancy new features:

  - It implements PhutilKeyValueCache, so it can be a component in cache stacks and supports TTL.
  - It has a 12-byte hash-based key.
  - It automatically compresses large blocks of data (most of what we store is highly-compressible HTML).

Test Plan:
  - Basics:
    - Loaded /paste/, saw caches generate and save.
    - Reloaded /paste/, saw the page hit cache.
  - GC:
    - Ran GC daemon, saw nothing.
    - Set maximum lifetime to 1 second, ran GC daemon, saw it collect the entire cache.
  - Deflate:
    - Selected row formats from the database, saw a mixture of 'raw' and 'deflate' storage.
    - Used profiler to verify that 'deflate' is fast (12 calls @ 220us on my paste list).
  - Ran unit tests

Reviewers: vrana, btrahan

Reviewed By: vrana

CC: aran

Differential Revision: https://secure.phabricator.com/D4259
2012-12-21 14:17:56 -08:00
epriestley
4af2e3c4e2 Add PhabricatorHash::digestForIndex()
Summary: Does this seem reasonable? It's a bit more compact than digest() (6 bits / byte instead of 4 bits / byte) and 72 bits of entropy @ 12 bytes instead of 128 bits of entropy @ 32 bytes. I feel like it's important to preserve the printability, though, and this seemed like a fairly good balance of concerns.

Test Plan: unit tests

Reviewers: vrana

Reviewed By: vrana

CC: aran, yemao932

Differential Revision: https://secure.phabricator.com/D4253
2012-12-21 05:43:33 -08:00
epriestley
e78898970a Implement SSHD glue and Conduit SSH endpoint
Summary:
  - Build "sshd-auth" (for authentication) and "sshd-exec" (for command execution) binaries. These are callable by "sshd-vcs", located [[https://github.com/epriestley/sshd-vcs | in my account on GitHub]]. They are based on precursors [[https://github.com/epriestley/sshd-vcs-glue | here on GitHub]] which I deployed for TenXer about a year ago, so I have some confidence they at least basically work.
    - The problem this solves is that normally every user would need an account on a machine to connect to it, and/or their public keys would all need to be listed in `~/.authorized_keys`. This is a big pain in most installs. Software like Gitosis/Gitolite solve this problem by giving you an easy way to add public keys to `~/.authorized_keys`, but this is pretty gross.
    - Roughly, instead of looking in `~/.authorized_keys` when a user connects, the patched sshd instead runs `echo <public key> | sshd-auth`. The `sshd-auth` script looks up the public key and authorizes the matching user, if they exist. It also forces sshd to run `sshd-exec` instead of a normal shell.
    - `sshd-exec` receives the authenticated user and any command which was passed to ssh (like `git receive-pack`) and can route them appropriately.
    - Overall, this permits a single account to be set up on a server which all Phabricator users can connect to without any extra work, and which can safely execute commands and apply appropriate permissions, and disable users when they are disabled in Phabricator and all that stuff.
  - Build out "sshd-exec" to do more thorough checks and setup, and delegate command execution to Workflows (they now exist, and did not when I originally built this stuff).
  - Convert @btrahan's conduit API script into a workflow and slightly simplify it (ConduitCall did not exist at the time it was written).

The next steps here on the Repository side are to implement Workflows for Git, SVN and HG wire protocols. These will mostly just proxy the protocols, but also need to enforce permissions. So the approach will basically be:

  - Implement workflows for stuff like `git receive-pack`.
  - These workflows will implement enough of the underlying protocol to determine what resource the user is trying to access, and whether they want to read or write it.
  - They'll then do a permissons check, and kick the user out if they don't have permission to do whatever they are trying to do.
  - If the user does have permission, we just proxy the rest of the transaction.

Next steps on the Conduit side are more simple:

  - Make ConduitClient understand "ssh://" URLs.

Test Plan: Ran `sshd-exec --phabricator-ssh-user epriestley conduit differential.query`, etc. This will get a more comprehensive test once I set up sshd-vcs.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T603, T550

Differential Revision: https://secure.phabricator.com/D4229
2012-12-19 11:08:07 -08:00
Ricky Elrod
93c4030732 Fix IRC bot Dxxx Conduit call.
Summary: Some fallout from D4191.

Test Plan: Sent "D12" in IRC and got a response.

Reviewers: epriestley, vrana, zeeg

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4235
2012-12-18 20:41:52 -08:00
Nick Harper
4a81ae6d6d Add data information to daemon task view
Summary:
Load the data for daemon worker tasks when viewing them, and present
the information in a useful way. This defaults to printing the json data,
but for some classes of worker it will also link to the corresponding
object, to make debugging problems with workers easier.

Test Plan:
load /daemon/task/NNN for a CommitParserWorker and a MetaMTAWorker, and
see the addition of a data field with useful content and link.

Reviewers: epriestley, vrana

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D4226
2012-12-17 17:12:55 -08:00
epriestley
7e37eb4827 Provide a highlighter cache for Paste
Summary:
D4188 adds a preview of Paste contents to the list, but gets slow for large lists if you have Pygments installed since it has to spend ~200ms/item highlighting them. Instead, cache the highlighted output.

  - Adds a Paste highlighting cache. This uses RemarkupCache because it has automatic GC and we don't have a more generalized cache for the moment.
  - Uses the Paste cache on paste list and paste detail.
  - Adds a little padding to the summary.
  - Adds "..." if there's more content.
  - Adds line count and language, if available. These are hidden on Mobile.
  - Nothing actually uses needRawContent() but I left it there since it doesn't hurt anything and is used internally (I thought the detail view did, but it uses the file content directly, and must for security reasons).

Test Plan:
{F27710}

  - Profiled paste list, saw good performance and few service calls.
  - Viewed paste.
  - Viewed raw paste content.

Reviewers: codeblock, btrahan, chad

Reviewed By: btrahan

CC: aran

Differential Revision: https://secure.phabricator.com/D4204
2012-12-16 16:33:42 -08:00
epriestley
4c7c518c63 Throw a richer exception when updating tasks with expired leases
Summary: Include task ID and class when raising this exception. I took a brief stab at doing this generically, but (a) we specifically raise this exception outside of normal try/catch because we can't follow normal recovery rules for it and (b) we don't have a reasonable PhutilProxyException or similar right now which would preserve stack traces, and don't have builtin exception nesting support until PHP 5.3.

Test Plan: Faked this exception, verified we get more information in the logs.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2193

Differential Revision: https://secure.phabricator.com/D4205
2012-12-16 16:27:32 -08:00