Summary:
Uploaded an example email with a lot of accents called 'test_accents.mbox' and
expected headers in the file 'test_accents.headers.txt'.
Better than nothing.
This change also includes a minor refactor in the library loading.
Ref T15960
Test Plan:
Manually run the new unit test and see green lights:
arc unit src/applications/metamta/externals/__tests__/PhabricatorExternalMimeMailParserTestCase.php
Double-check that the new class is already recorded:
arc liberate
Just as extra care, re-apply the same test plan of:
D25839
So, for example, run this, and see no exceptions:
./scripts/mail/mail_handler.php < src/applications/metamta/externals/__tests__/data/test_accents.mbox
Reviewers: aklapper, taavi, O1 Blessed Committers
Reviewed By: aklapper, O1 Blessed Committers
Subscribers: tobiaswiese, Matthew, Cigaryno
Maniphest Tasks: T15960
Differential Revision: https://we.phorge.it/D25844
Summary:
rPa76444a8e238f647dc96f756e6c88aa2fafcdbfe updated our 13 year old copy of the mimemailparser library.
That included a behaviour change in the library not covered by Phorge code: The library now decodes MIME encoded UTF8 data in headers. Phorge passes that header to the `iconv_mime_decode()` PHP function which does not accept already encoded content.
```
EXCEPTION: (RuntimeException) iconv_mime_decode(): Detected an illegal character in input string at [<arcanist>/src/error/PhutilErrorHandler.php:273]
arcanist(head=master, ref.master=29ca3df1122b), phorge(head=master, ref.master=6ec5c88bee24)
#0 PhutilErrorHandler::handleError(integer, string, string, integer)
#1 iconv_mime_decode(string, integer, string) called at [<arcanist>/src/utils/utils.php:1759]
#2 phutil_decode_mime_header(string) called at [<phorge>/scripts/mail/mail_handler.php:64]
```
Closes T15960
Test Plan: * Have an email file called `tmp.mbox` with a UTF-8 encoded `From:` header. In `scripts/mail/mail_handler.php`, replace `file_get_contents('php://stdin')` with `file_get_contents('./tmp.mbox')`. Insert `echo $headers['subject']; echo "\n"; echo $headers['from'];` statements for debugging. Run `php ./mail_handler.php`.
Reviewers: O1 Blessed Committers, taavi, valerio.bozzolan
Reviewed By: O1 Blessed Committers, taavi, valerio.bozzolan
Subscribers: taavi, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15960
Differential Revision: https://we.phorge.it/D25839
Summary:
Bump to version 8.0.4 from 2024-09-11 per https://github.com/php-mime-mail-parser/php-mime-mail-parser/releases before this ancient code copy falls apart.
`scripts/mail/mail_handler.php` (used for incoming (!) mail) is the only consumer.
Closes T15940
Test Plan: Feed `mail_handler.php` with various test emails (formats: plain text, HTML, multipart; encodings: UTF-8, ASCII, ISO-8859-something) by manually replacing `php://stdin` with corresponding text files and adding some `phlog`s for output checking as I don't have mail server glue handy. Get only expected errors for broken emails.
Reviewers: O1 Blessed Committers, 20after4
Reviewed By: O1 Blessed Committers, 20after4
Subscribers: 20after4, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15940
Differential Revision: https://we.phorge.it/D25829
Summary:
* Properly handle when no mail headers at all can be parsed
* Properly handle when mail headers can be parsed but no subject line can be found
```
EXCEPTION: (RuntimeException) Undefined index: subject
```
Closes T15769
Test Plan: See T15769
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: 20after4, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15769
Differential Revision: https://we.phorge.it/D25565
Summary:
Just a small set of replacements in locations that seem innocuous (user-facing messages, documentation, etc.)
Ref T15006
Test Plan:
Nothing should change in terms of behavior. The places where these changes were made should now say "phorge".
Example tests:
- Manage a single User and click on Delete User and see the popup
- Run a test email and check the output
./bin/mail send-test --to username
- Visit /maniphest/, shift+click on at least 1 Task, click on Bulk Edit Selected,
Continue, see the popup
- See the mentioned documentation with your big eyes. Eyes do not explode \o/
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15006
Differential Revision: https://we.phorge.it/D25473
Summary:
Add `./bin/celerity sprites`, to replace script `./scripts/celerity/generate_sprites.php`.
Also make new workflow run `./bin/celerity map` at the same time.
Fixes T15437.
Test Plan: Changes a file that goes in the sprites, run new command
Reviewers: O1 Blessed Committers, speck
Reviewed By: O1 Blessed Committers, speck
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15437
Differential Revision: https://we.phorge.it/D25274
Summary:
Fix a regression introduced here:
96ae4ba13a
I reproduced this exception executing "svn commit" on a hosted repository.
That crash happened because the PHP getenv() function can return false.
But, that is a very terrible value that blasts the non-string-empty check.
So, now the default getenv() value is skipped, without causing problems.
Closes T15253
Ref T15190
Test Plan: - I've run `svn commit` and I have not encountered any issue now
Reviewers: O1 Blessed Committers, avivey
Reviewed By: O1 Blessed Committers, avivey
Subscribers: speck, tobiaswiese, Matthew, Cigaryno
Maniphest Tasks: T15253, T15190
Differential Revision: https://we.phorge.it/D25122
Summary:
This is a fix for PHP 8.1 deprecation of strlen(NULL), for these Phorge components:
- scripts
- aphront
- project
The strlen() was used in Phabricator to check if a generic value was a non-empty string.
For this reason, Phorge adopts phutil_nonempty_string() that checks that.
Note: this may highlight other absurd input values that might be worth correcting
instead of just ignoring. If your phutil_nonempty_string() throws an exception, just
report it to Phorge to evaluate and fix together that specific corner case.
Closes T15223
Ref T15190
Ref T15064
Test Plan: - check with your big eyes that there are no obvious typos
Reviewers: O1 Blessed Committers, avivey
Reviewed By: O1 Blessed Committers, avivey
Subscribers: speck, tobiaswiese, Matthew, Cigaryno
Maniphest Tasks: T15223, T15190, T15064
Differential Revision: https://we.phorge.it/D25105
Summary: Ref T13658. I used the linter in D21763 to identify these and `split` them into arbitrary groups of 10 files.
Test Plan:
This test plan is non-exhaustive, because some of these strings are difficult to reach.
- Looked at "Create Service" in Almanac.
- Used "bin/auth" to go through a one-time auth workflow (not all related strings can be hit on a single workflow).
- Started the "Generate Keypair" worfklow in "SSH Public Keys".
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21765
Summary:
Ref T13588. This configuration value may not be set.
Also fix an issue in `bin/storage` and whatever else I hit between now and this diff actually uploading.
Also fix a MySQLi report mode difference, beginning in PHP 8.1.
Also update a bunch of "static" property usage in Lisk.
Test Plan: Ran `bin/files ...` locally under PHP 8.1.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21744
Summary: Ref PHI2157. Like other low-level tools, "bin/celerity" does not need databases configured in order to execute.
Test Plan: Ran `bin/celerity map` with and without the database available.
Differential Revision: https://secure.phabricator.com/D21730
Summary:
Set the Reposity links to actually install Phorge, Update Documentation links to the new Website.
Notes:
- Github Mirrors are not set up. Use one anyway? Or just use the Repos at we.phorge.it?
- Documentation Links still contain "phabricator". The Docs are changed, but the Diviner Books are not rebuilt. (Add a Cronjob or do it after every Commit?)
- mysql-server is not included anymore in Debian environments. Should mariadb-server be used?
- Should we print somewhere that Phorge is a Fork of Phabricator?
Test Plan:
Run the install script.
It should install Phorge and display Links to we.phore.it
Reviewers: O1 Blessed Committers, speck
Reviewed By: O1 Blessed Committers, speck
Subscribers: speck, tobiaswiese
Differential Revision: https://we.phorge.it/D25019
Summary: Ref T13660. Clean up callsites to "PhutilExecPassthru->execute()" to prepare to deprecate it.
Test Plan:
- Grepped for "PhutilExecPassthru" and looked for callsites.
- Ran `GIT_SSH=.../ssh-connect git ls-remote origin` to execute the "ssh-connect" code.
- The two passthru future methods have no callers and could possibly be removed, but I'm just letting sleeping dogs lie for now.
Reviewers: cspeckmim
Reviewed By: cspeckmim
Maniphest Tasks: T13660
Differential Revision: https://secure.phabricator.com/D21703
Summary:
Ref T13658. This just scrubs some of the simple references from the codebase.
Most of what's left is in documentation which won't be relevant for a fork and/or which I need to separately revise (or more-or-less delete) at some point anyway.
I removed the "install RHEL" and "install Ubuntu" scripts outright since I don't have any reasonable way to test them and don't plan to maintain them.
Test Plan: Grepped for "phacility", "epriestley"; ran unit tests.
Reviewers: cspeckmim
Reviewed By: cspeckmim
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21678
Summary:
Ref T13641. Phabricator sometimes makes intracluster requests that authenticate as a device.
Forbid these requests from authenticating as a disabled device.
Test Plan:
- Ran `bin/ssh-exec --phabricator-ssh-device ...` as an enabled/disabled device (worked; sensible error).
- Made Conduit calls as an enable/disabled device (worked; sensible error).
Maniphest Tasks: T13641
Differential Revision: https://secure.phabricator.com/D21635
Summary:
Ref T13395. "libphutil/" was stripped for parts, but some documentation still references it. This is mostly minor corrections, but:
- Removes "Javelin at Facebook", long obsolete.
- Removes "php FPM warmup", which was always a prototype and is obsoleted by PHP preloading in recent PHP.
Test Plan: `grep` / reading
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D21624
Summary: Ref T13624. Depends on D21578. In "sshd" subprocess contexts, use "PhutilErrorLog" to direct errors to both stderr and, if configured, a logfile on disk.
Test Plan:
- Confiugured an error log.
- Forced `ssh-auth` to fatal.
- Saw errors on stderr and in log.
Maniphest Tasks: T13624
Differential Revision: https://secure.phabricator.com/D21579
Summary:
Ref T13395. Libphutil has merged into Arcanist and no longer needs to be installed or upgraded. Additionally:
- The minimum PHP version is now PHP 5.5.
- Although older versions of PHP should still install APC, modern versions come with Opcache and do not need APC. Setup issues guide administrators thorugh the correct install procedure now.
Test Plan: Read documentation.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D21550
Summary: See PHI1784. Currently, users who pass an invalid SSH command to Phabricator's SSH handler get an unhelpful error message. Make it more helpful.
Test Plan: Ran `./bin/ssh-exec` with no arguments (old, helpful error), invalid arguments (before: unhelpful error; after: helpful error), and valid arguments (old, helpful behavior).
Differential Revision: https://secure.phabricator.com/D21362
Summary: Ref T13395. Since there's very little code which really makes sense in "scripts/", I've moved most of it to other places.
Test Plan: Ran `bin/phd`.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D20994
Summary: Ref T13395. Moves a small amount of remaining "libphutil/" code into "phabricator/" and stops us from loading "libphutil/".
Test Plan: Browsed around; there are likely remaining issues.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D20981
Summary:
Ref T13436. There's no real security value to doing this comparison, it just wards off evil "security researchers" who get upset if you ever compare two strings with a non-constant-time algorithm.
In practice, SSH public keys are pretty long, pretty public, and have pretty similar lengths. This leads to a relatively large amount of work to do constant-time comparisons on them (we frequently can't abort early after identifying differing string length).
Test Plan: Ran `bin/ssh-auth --sshd-key ...` on `secure` with ~1K keys, saw runtime drop by ~50% (~400ms to ~200ms) with `===`.
Maniphest Tasks: T13436
Differential Revision: https://secure.phabricator.com/D20875
Summary: Depends on D20873. Ref T13436. Allow callers to configure "bin/ssh-auth --sshd-key %k" as an "AuthorizedKeysCommand"; if they do, and we recognize the key, emit just that key in the output.
Test Plan:
- Used `git pull` locally, still worked fine.
- Instrumented things, saw the public key lookup actually work and emit a single key.
- Ran without "--sshd-key", got a full key list as before.
Maniphest Tasks: T13436
Differential Revision: https://secure.phabricator.com/D20874
Summary:
Ref T13436. Historically, this script could be used with a forked copy of "sshd" to do lower-cost per-key auth.
Relatively modern "sshd" supports "%f" to "AuthorizedKeysCommand", which effectively moots this.
Users have never been instructed to use this script for anything, and we moved away from this specific patch to "sshd" some time ago.
Test Plan: Grepped for "ssh-auth-key", no hits.
Maniphest Tasks: T13436
Differential Revision: https://secure.phabricator.com/D20873
Summary:
Fixes T13382. Depends on D20724. These ancient scripts are no longer necessary since we've had a smooth web-based onboarding process for a long time.
I retained `bin/user empower` and `bin/user enable` for recovering from situations where you accidentally delete or disable all administrators. This is normally difficult, but some users are industrious.
Test Plan: Grepped for `accountadmin` and `add_user.php`, found no more hits.
Maniphest Tasks: T13382
Differential Revision: https://secure.phabricator.com/D20725
Summary:
Ref T13382.
- Remove "bin/people profileimage" which previously generated profile image caches but now feels obsolete.
- Replace it with "bin/user", with "enable" and "empower" flows. This command is now focused on regaining access to an install after you lock your keys inside.
- Document the various ways to unlock objects and accounts from the CLI.
Test Plan:
- Ran `bin/user enable` and `bin/user empower` with various flags.
- Grepped for `people profileimage` and found no references.
- Grepped for `bin/people` and found no references.
- Read documentation.
Maniphest Tasks: T13382
Differential Revision: https://secure.phabricator.com/D20724
Summary: Depends on D20069. Ref T13232. This is a very, very weak dependency and we can reasonably polyfill it.
Test Plan: Grepped for `iconv` in libphutil, arcanist, and Phabricator.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13232
Differential Revision: https://secure.phabricator.com/D20070
Summary: If you go through the `accountadmin` flow and change nothing, you get an exception about the transaction not having any effect. Instead, let the `applyTransactions` call continue even on no effect.
Test Plan: Ran `accountadmin` without changing anything for an existing user. No longer got an exception about no-effect transactions.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D20009
Summary:
Ref PHI1027. Currently, `PhabricatorUser` has a couple of mail-related methods which shouldn't really be there in the long term. Immediately, I want to make some adjusments to the welcome email.
Move "Welcome" mail generation to a separate class and consolidate all the error handling. (Eventually, "invite" and "verify address" email should move to similar subclasses, too.) Previously, a bunch of errors/conditions got checked in multiple places.
The only functional change is that we no longer allow you to send welcome mail to disabled users.
Test Plan:
- Used "Send Welcome Mail" from profile pages to send mail.
- Hit "not admin", "disabled user", "bot/mailing list" errors.
- Used `scripts/user/add_user.php` to send welcome mail.
- Used "Create New User" to send welcome mail.
- Verified mail with `bin/mail show-outbound`. (Cleaned up a couple of minor display issues here.)
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19989
Summary:
After T13217 import_repository_symbols.php was showing a lot of warnings, using %LQ fixes that.
I'm aware, that there are changes planned to the whole managing the symbols complex but until then less warnings are nice.
Test Plan: No more warnings when updating symbols
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D19962
Summary:
Ref T920. Over time, mail has become much more complex and I think considering "mail", "sms", "postcards", "whatsapp", etc., to be mostly-the-same is now a more promising avenue than building separate stacks for each one.
Throw away all the standalone SMS code, including the Twilio config options. I have a separate diff that adds Twilio as a mail adapter and functions correctly, but it needs some more work to bring upstream.
This permanently destroys the `sms` table, which no real reachable code ever wrote to. I'll call this out in the changelog.
Test Plan:
- Grepped for `SMS` and `Twilio`.
- Ran storage upgrade.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T920
Differential Revision: https://secure.phabricator.com/D19939
Summary: Ref T13216. See D19666. It's currently tricky to profile Herald test runs since you have to submit a form and repeating them is a bit of a mess. Provide a simple CLI wrapper so we can use `--xprofile`. This is also maybe nice-to-have if we're ever debugging anything here.
Test Plan: Ran `bin/herald test --object ... --type ...` and got a sensible looking transcript in the UI.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13216
Differential Revision: https://secure.phabricator.com/D19806
Summary: Depends on D19796. Simplify some timing code by using phutil_microseconds_since() instead of duplicate casting and arithmetic.
Test Plan: Grepped for `1000000` to find these. Pulled, pushed, made a conduit call. This isn't exhaustive but it should be hard for these to break in a bad way since they're all just diagnostic.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19797
Summary:
Depends on D19779. Ref T13216. The push logs currently record the "hostWait", which is roughly "locking + subprocess cost". We also record locking separately, so we can figure out "subprocess cost" alone by subtracting the lock costs.
However, the subprocess (normally `git receive-pack`) runs hooks, and we don't have an easy way to figure out how much time was spent doing actual `git` stuff vs spent doing commit hook processing. This would have been useful in diagnosing at least one recent issue.
Track at least a rough hook cost and record it in the push logs.
Test Plan: Pushed to a repository, saw a reasonable hook cost appear in the database table.
Reviewers: amckinley
Reviewed By: amckinley
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13216
Differential Revision: https://secure.phabricator.com/D19780
Summary:
Ref T13195. See PHI842. Alternative to D19638.
Instead of doing all the stuff in D19638, //just// remove the `rebuild_summaries.php` script. This script is outdated, copy/pastes the rebuild logic, and doesn't understand unreachable commits.
If we had some use for it it should move to `bin/repository rebuild-summary ...` or similar, but it's not clear there's any use for it. The incremental summary rebuilds seem to work fine as-is.
Test Plan: Grepped for callers or documentation referencing this script, found nothing.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13195
Differential Revision: https://secure.phabricator.com/D19643
Summary: Currently the symbol generation scripts fail if passed a list containing no files because `explode("\n", $input)` returns `array("")` rather than `array()`. This means that a generic Harbormaster Build Plan with a step which executes `find . -type f -name '*.php' | ./scripts/generate_php_symbols.php` won't work because it fails in repositories that don't contain any PHP code.
Test Plan: Ran `echo | generate_php_symbols` and saw no output instead of an exception.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D19588
Summary:
Fixes T12397. Ref T13164. See PHI801.
Several installs have hit various use cases where the path on disk where Phabricator lives changes at runtime. Currently, `bin/ssh-auth` caches a flat file which includes the path to `bin/ssh-exec`, so this may fall out of date if `phabricator/` moves.
These use cases have varying strengths of legitimacy, but "we're migrating to a new set of hosts and the pool is half old machines and half new machines" seems reasonably compelling and not a problem entirely of one's own making.
Test Plan:
- Compared output on `master` to output after change, found them byte-for-byte identical.
- Moved `phabricator/` to `phabricator2/`, ran `bin/ssh-auth`, got updated output.
- Added a new SSH key, saw it appear in the output.
- Grepped for `AUTHFILE_CACHEKEY` (no hits).
- Dropped the cache, verified that the file regenerates cleanly.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13164, T12397
Differential Revision: https://secure.phabricator.com/D19568
Summary:
Makes `ssh-connect` compatible with Git v2 wire protocol over SSH
More details about git V2 wire: https://opensource.googleblog.com/2018/05/introducing-git-protocol-version-2.html
`git` command (2.18+) passes extra options (`-o "SendEnv GIT_PROTOCOL"`) to underlying `ssh` command to enable v2 wire protocol (environment variable enabling new protocol).
Phabricator `ssh-connect` command doesn't understand `-o` options and interprets it as host parts hence when you enable git v2 all clones/ls-remotes crash with:
```
#0 ExecFuture::resolvex() called at [<phabricator>/src/applications/repository/storage/PhabricatorRepository.php:525]
#1 PhabricatorRepository::execxRemoteCommand(string, PhutilOpaqueEnvelope) called at [<phabricator>/src/applications/repository/engine/PhabricatorRepositoryPullEngine.php:400]
#2 PhabricatorRepositoryPullEngine::loadGitRemoteRefs(PhabricatorRepository) called at [<phabricator>/src/applications/repository/engine/PhabricatorRepositoryPullEngine.php:343]
#3 PhabricatorRepositoryPullEngine::executeGitUpdate() called at [<phabricator>/src/applications/repository/engine/PhabricatorRepositoryPullEngine.php:126]
#4 PhabricatorRepositoryPullEngine::pullRepositoryWithLock() called at [<phabricator>/src/applications/repository/engine/PhabricatorRepositoryPullEngine.php:40]
#5 PhabricatorRepositoryPullEngine::pullRepository() called at [<phabricator>/src/applications/repository/management/PhabricatorRepositoryManagementUpdateWorkflow.php:59]
#6 PhabricatorRepositoryManagementUpdateWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:441]
#7 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:333]
#8 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/repository/manage_repositories.php:22]
COMMAND
git ls-remote '********'
STDOUT
(empty)
STDERR
ssh: Could not resolve hostname -o: Name or service not known
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
at [<phutil>/src/future/exec/ExecFuture.php:369]
```
Test Plan:
How to reproduce:
1. add repository to Phabricator which is accessed via `ssh`
2. Use git 2.18+
3. Enable wire protocol in `/etc/gitconfig`:
```
[protocol]
version = 2
```
4. Try refreshing repository: `phabricator/bin/repository update somecallsing`
5. Repository update fails with `ssh: Could not resolve hostname -o: Name or service not known`
after this changes - updates will succeed
Reviewers: epriestley, Pawka, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: Korvin
Differential Revision: https://secure.phabricator.com/D19542
Summary:
Ref T4200. Since the last time this script was written, Ubuntu has made lots of changes.
Try to keep up with those.
Test Plan:
Ran this on frash installs of Ubuntu 16.04 LTS and 18.04 LTS (Pre-release).
Got to see Phabricator running.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: amckinley, Korvin
Maniphest Tasks: T4200
Differential Revision: https://secure.phabricator.com/D19394
Summary:
Depends on D19247. Ref T13109. When we receive an SSH request, generate a random unique ID for the request. Then thread it down through the process tree.
The immediate goal is to let the `ssh-exec` process coordinate with `commit-hook` process and log information about read and write lock wait times. Today, there's no way for `ssh-exec` to interact with the `PushEvent`, but this is the most helpful place to store this data for users.
Test Plan: Made pushes, saw the `PushEvent` table populate with a random request ID. Exported data and saw the ID preserved in the export.
Maniphest Tasks: T13109
Differential Revision: https://secure.phabricator.com/D19249
Summary:
Fixes T12994. We need `MYSQLI_ASYNC` to implement client-side query timeouts, and we need MySQLi + MySQL Native Driver to get `MYSQLI_ASYNC`.
Recommend users install MySQLi and MySQL Native Driver if they don't have them. These are generally the defaults and best practice anyway, but Ubuntu makes it easy to use the older stuff.
All the cases we're currently aware of stem from `apt-get install php5-mysql` (which explicitly selects the non-native driver) so issue particular guidance about `php5-mysqlnd`.
Test Plan:
- Faked both issues locally, reviewed the text.
- Will deploy to `secure`, which currently has the non-native driver.
Maniphest Tasks: T12994
Differential Revision: https://secure.phabricator.com/D19216
Summary: Depends on D19173. Ref T13096. Adds an optional, disabled-by-default lock log to make it easier to figure out what is acquiring and holding locks.
Test Plan: Ran `bin/lock log --enable`, `--disable`, `--name`, etc. Saw sensible-looking output with log enabled and daemons restarted. Saw no additional output with log disabled and daemons restarted.
Maniphest Tasks: T13096
Differential Revision: https://secure.phabricator.com/D19174
Summary:
Depends on D19072. Ref T13073. Currently, you can leave leases stranded by using `^C` to interrupt the script. Handle signals and release leases on destruction if they haven't activated yet.
Also, print out more useful information before and after activation.
Test Plan: Mashed ^C while runnning `bin/drydock lease ... --trace`, saw the lease release.
Subscribers: yelirekim, PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13073
Differential Revision: https://secure.phabricator.com/D19073
Summary: Ref T11330. Adds general support for webhooks. This is still rough and missing a lot of pieces -- and not yet useful for anything -- but can make HTTP requests.
Test Plan: Used `bin/webhook call ...` to complete requests to a test endpoint.
Maniphest Tasks: T11330
Differential Revision: https://secure.phabricator.com/D19045
Summary:
Ref T13060. See PHI343. Triaging this bug required figuring out where in the pipeline UTF8 was being dropped, and bisecting the pipeline required making calls to Conduit.
Currently, there's no easy way to debug/inspect arbitrary Conduit calls, especially when they are `diffusion.*` calls which route to a different host (even if you have a real session and use the web console for these, you just see an HTTP service call to the target host in DarkConsole).
Add a `bin/conduit` utility to make this kind of debugging easier, with an eye toward the Phacility production cluster (or other similar clusters) specifically.
Test Plan:
- Ran `echo '{}' | bin/conduit call --method conduit.ping --input -` and similar.
- Used a similar approach to successfully diagnose the UTF8 issue in T13060.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13060
Differential Revision: https://secure.phabricator.com/D18987
Summary:
See PHI305. Ref T13046.
The SSH workflows currently extend `PhabricatorManagementWorkflow` to benefit from sharing all the standard argument parsing code. Sharing the parsing code is good, but it also means they inherit a `getViewer()` method which returns the ommnipotent viewer.
This is appropriate for everything else which extends `ManagementWorkflow` (like `bin/storage`, `bin/auth`, etc.) but not appropriate for SSH workflows, which have a real user.
This caused a bug with the pull logs where `pullerPHID` was not recorded properly. We used `$this->getViewer()->getPHID()` but the correct code was `$this->getUser()->getPHID()`.
To harden this against future mistakes:
- Don't extend `ManagementWorkflow`. Extend `PhutilArgumentWorkflow` instead. We **only** want the argument parsing code.
- Rename `get/setUser()` to `get/setSSHUser()` to make them explicit.
Then, fix the pull log bug by calling `getSSHUser()` instead of `getViewer()`.
Test Plan:
- Pulled and pushed to a repository over SSH.
- Grepped all the SSH stuff for the altered symbols.
- Saw pulls record a valid `pullerPHID` in the pull log.
- Used `echo {} | ssh ... conduit conduit.ping` to test conduit over SSH.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13046
Differential Revision: https://secure.phabricator.com/D18912
