1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-27 17:22:42 +01:00
Commit graph

1347 commits

Author SHA1 Message Date
epriestley
daa893e508 Extend TransactionCommentQuery for Diffusion
Summary: Ref T2009. Ref T1460. Reduces the amount of garbage involved in loading inline comments and routes more pathways through the proper Query layer.

Test Plan: Viewed, edited, previewed, submitted inline comments in Diffusion.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009, T1460

Differential Revision: https://secure.phabricator.com/D12028
2015-03-09 14:11:22 -07:00
epriestley
7427a6e648 Extend TransactionCommentQuery for Differential
Summary: Ref T2009. Ref T1460. Replace hard-coded garbage with a real Query-layer query.

Test Plan: Submitted inline comments in Differential.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009, T1460

Differential Revision: https://secure.phabricator.com/D12027
2015-03-09 14:11:20 -07:00
epriestley
4d86d51125 Prepare TransactionCommentQuery for extension
Summary:
Ref T2009. Ref T1460. The way Diffusion and Differential load inlines is horrible garbage right now:

  - Differential does an ad-hoc query to get the PHIDs, then does a real load to policy check.
  - Diffusion completely fakes things. In practice this is not a policy violation, but it's dangerous.

Make TransactionCommentQuery extensible so we can subclass it and get the query building correctly in the right Query layer.

Specifically, the Diffusion and Differential subclasses of this Query will add appropriate `withX()` methods to let us express the query in SQL.

Test Plan: Loaded, previewed, edited, and submitted inlines in Differential and Diffusion

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009, T1460

Differential Revision: https://secure.phabricator.com/D12026
2015-03-09 14:11:18 -07:00
epriestley
2972894a4d Write "hasReplies" to database for inline comments
Summary:
Ref T1460. Ref T2618.

When publishing a draft inline, mark the inline it replies to (if any) as replied to.

Also, don't load deleted comments as drafts (sets the stage for T2618).

I'll make an effort to clean up the loading mess here in the next revision, and find some more appropriate home for the shared code.

Test Plan: Made and replied to comments in Differential and Diffusion. Saw comments get marked as "Has Replies" and "Is Reply".

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2618, T1460

Differential Revision: https://secure.phabricator.com/D12025
2015-03-09 14:11:16 -07:00
epriestley
f1f2c5d01d Reduce code duplication in inline right/left side tracking
Summary: Ref T2009. These subclasses have a mixture of similar methods, move them all to the base class.

Test Plan: Created/edited/undo/submitted comments on the left and right sides of a diff.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D12024
2015-03-09 12:53:40 -07:00
epriestley
56a9709008 Reduce code duplication for inline "Undo"
Summary:
Ref T2009. This is another almost-identical copy of the row scaffolding, which has the same 1up/2up bugs as the 8 other copies of this code.

Turn the "undo" element into an InlineCommentView so we can scaffold it.

Then, scaffold it with the same code as everything else.

Test Plan: Hit "Undo", swapped from 1up to 2up, hit "undo" again, swapped back, tried left/right, everything rendered with proper scaffolding.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D12019
2015-03-09 10:26:53 -07:00
epriestley
355142fcbf Reduce code duplication on comment editing UI
Summary: Ref T2009. This has two more copies of the scaffolding.

Test Plan: Created, edited, deleted, replied to inline comments.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D12018
2015-03-09 10:26:51 -07:00
epriestley
082b7f95e6 Explicitly track inline comment reply threading
Summary:
Ref T1460. Track and store which comments are threaded replies to other comments, vs merely appearing on the same lines.

This doesn't actually write `hasReplies` yet, since that needs to happen when we un-draft comments on submission.

Test Plan: Made inline comments in Differential and Diffusion, including replies. Replies were marked as "Is Reply".

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T1460

Differential Revision: https://secure.phabricator.com/D12017
2015-03-09 10:26:50 -07:00
epriestley
1df321bf00 Fix left/right detection of inline comments in unified view
Summary:
Ref T2009. Currently, the code figures out if a comment is on the left or right by looking at the `<th />` preceeding the enclosing `<td />`.

This gets the right result in 2-up, but in 1-up rows are always `<th />`, `<th />`, `<td />`, so it always detects every inline as being in the new file.

Because "old" and "new" cells aren't inherently distingushable in the 1up view, we can't use a DOM test for this at all. Instead, just track this state explicitly.

Test Plan:
  - Made left/right comments in 1up view and 2up view.
  - Viewed them in 1up and 2up views.
  - Hovered in 1up and 2up views.
  - Diff-of-diff'd and reviewed old/new comments, then made some more.

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D12011
2015-03-07 14:37:57 -08:00
epriestley
f9cb366f00 Remove duplicate inline scaffold in 2up renderer
Summary: Ref T2009. Remove the 4 (!!) copies of this code.

Test Plan:
  - Added, edited, and removed inline comments in 2up view.
  - Stacked a bunch of comments on the same line and saw the JS place them correctly.
  - Created an image diff and added, edited and removed inlines on it.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D12000
2015-03-06 15:00:43 -08:00
epriestley
1352be827e Begin separating inline comment scaffolding from other renderers
Summary:
Ref T2009. Inline comments have "scaffolding", which is basically some empty table cells/rows around them to get the layout correct.

The scaffolding depends on the renderer, since the cells are different for side-by-side vs unified diffs.

This is currently duplicated all over the place:

  - Edit view has 1up/2up.
  - Detail view has 1up/2up.
  - 1up renderer has 1up.
  - 2up renderer has four separate copies of the 2up logic.

These all have subtle differences, which are mostly bugs. Start making the scaffolding more composable so we can get rid of that mess.

Test Plan: Added, edited, and removed inline comments on unified and side-by-side diffs.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D11997
2015-03-06 15:00:33 -08:00
epriestley
1088d34e58 Rename inline comment views to "PHUIDiff" and give them a base class
Summary:
Ref T2009. These classes are "Differential" now, but are used elsewhere in diff infrastructure (e.g., Diffusion).

  - Rename them to "PHUIDiff".
  - Move them to "src/infrastructure/".
  - Give them a base class.

Test Plan: Interacted with inlines in unified and side-by-side views.

Reviewers: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D11996
2015-03-06 15:00:14 -08:00
Chad Little
85f4bdc2ac Update Conpherence Main UI
Summary: Uses standard sidenav width, more spacing in labels, added background around textarea, make background work in Firefox.

Test Plan:
Test Desktop, Mobile, and Tablet break points. Test Firefox and Chrome.

{F331201}

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11993
2015-03-05 17:09:07 -08:00
epriestley
9564b0a40e Improve behavior of inline rendering with unified views
Summary:
Ref T2009. This reduces how buggy inlines are. They're still buggy.

Specifically, the inline endpoint didn't know how to scaffold inlines before, so some of them ended up rendering in the wrong rows or breaking layouts.

This passes the current renderer through to the inline editor endpoint, so it can at least get the layout correct.

Test Plan: Interacted with inlines in unified and side-by-side views.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D11988
2015-03-05 14:11:51 -08:00
epriestley
ad3c94dd45 Make "Show Context" persist rendering, whitespace, encoding, etc
Summary:
Ref T2009. Currently, we do not persist view parameters when making context rendering requests.

The big one is the renderer (1up vs 2up). This makes context on unified diffs come in with too many columns.

However, it impacts other parameters too. For example, at HEAD, if you change highlighting to "rainbow" and then load more context, the context uses the original highlighter instead of the rainbow highlighter.

This moves context loads into ChangesetViewManager, which maintains view parameters and can provide them correctly.

  - This removes "ref"; it is no longer required, as the ChangesetViewManager tracks it.
  - This removes URI management from `behavior-show-more`; it is no longer required, since the ChangesetViewManager knows how to render.
  - This removes "whitespace" since this is handled properly by the view manager.

Test Plan:
  - Used "Show Top" / "Show All" / "Show Bottom" in 1-up and 2-up views.
  - Changed file highlighting to rainbow, loaded stuff, saw rainbow stick.
  - Used "Show Entire File" in 1-up and 2-up views.
    - Saw loading chrome.
    - No loading chrome normally.
  - Made inlines, verified `copyRows()` code runs.
  - Poked around Diffusion -- it is missing some parameter handling, but works OK.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D11977
2015-03-05 14:03:00 -08:00
epriestley
06df75ebbd Render "Show Context" blocks in unified view
Summary:
Ref T2009. This basically copy/pastes them for now. Plans is:

  - Make this actually work all the way.
  - Add test coverage after D11970.
  - Move 2-up here after test coverage.

Clicking the links does not work yet, because they use the 2-up renderer. I'll fix this in the next diff.

Test Plan: Viewed diffs in unified, saw links to show more.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: chad, epriestley

Maniphest Tasks: T2009

Differential Revision: https://secure.phabricator.com/D11976
2015-03-05 14:02:29 -08:00
epriestley
e1d09fd035 Show change details for "Remarkup" standard custom field edits
Summary: Fixes T7436.

Test Plan: {F328222}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7436

Differential Revision: https://secure.phabricator.com/D11952
2015-03-03 10:39:32 -08:00
epriestley
a65667443b Fix quickstart.sql for old MySQL
Summary:
Fixes T7422. After the recent fix for "sort" columns, we can end up with invalid SQL in some cases when running quickstart.

In particular, we do "COLLATE binary CHARACTER SET utf8_general_ci" (which is invalid).

Preprocess these so we get "COLLATE utf8 CHARACTER SET utf8_general_ci" (which is valid and correct).

Test Plan: Ran `bin/storage upgrade -f --namespace blahblhbaba` with and without `--disable-utf8mb4`.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7422

Differential Revision: https://secure.phabricator.com/D11929
2015-03-02 09:57:38 -08:00
epriestley
9e82cfcc21 Use utf8_general_ci for "sort" columns in old MySQL
Summary:
Fixes T7287. This trades off 4-byte character support for case insensitivity in these columns, which is a much better trade on the balance.

Also adds more warnings about old MySQL. Note that we already issue a warning when you run "storage adjust" (which I've made stronger) and already "strongly recommend" MySQL 5.5 or newer in the install documentation.

Test Plan:
  - Ran `storage adjust --disable-utf8mb4` to go to old definitions, then ran `storage adjust` to get back to the new ones. Everything seemed OK in both cases.
  - Verified that utf8mb4 data can be migrated out of these colums with `--unsafe` (which will truncate).
  - Verified that manual explains this.
  - Faked my way into the setup warning.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7287

Differential Revision: https://secure.phabricator.com/D11893
2015-02-26 10:18:54 -08:00
epriestley
38636a39cf Allow modern phd stop to stop old daemons cleanly
Summary:
Ref T7352. Make sure modern `phd stop` can still read the old PID file format and stop the daemons, at least for now.

Without this, `stop` still detects them and tells you to `stop --force`, which works, but this makes things a good deal cleaner.

Test Plan: Ran `phd stop` from master, then `phd stop` from this revision. Saw old daemons stop cleanly.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11873
2015-02-24 14:50:40 -08:00
epriestley
a3518e19a5 Merge GC daemon into Trigger daemon
Summary:
Fixes T7352. This reduces the memory footprint for instances by combining these two similar daemons into one daemon which handles the responsibilities of both.

The fit isn't 100% perfect here but it's pretty close, and the GC daemon is fairly trivial.

Test Plan:
  - Adjusted all the numbers to small numbers (5 second sleep, 120 second GC length).
  - Added a ton of logging.
  - Started trigger daemon.
    - Saw it run a GC cycle.
    - Saw it reschedule another cycle after 120 seconds (adjusted down from 4 hours).
  - Reverted all the logging/small numbers.
  - Ran `bin/phd start`, saw stable trigger daemon running.
  - Grepped for removed daemon class name.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11872
2015-02-24 14:50:39 -08:00
epriestley
af303f458b Convert taskmasters to use an autoscale pool
Summary: Ref T7352. This is pretty straightforward. I renamed `phd.start-taskmasters` to `phd.taskmasters` for clarity.

Test Plan:
  - Ran `phd start`, `phd start --autoscale-reserve 0.25`, `phd restart --autoscale-reserve 0.25`, etc.
  - Examined PID file to see options were passed.
  - I'm defaulting this off (0 reserve) and making it a flag rather than an option because it's a very advanced feature which is probably not useful outside of instancing.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11871
2015-02-24 14:50:38 -08:00
epriestley
a354e5fa6b Track daemon unique IDs in Phabricator daemon logs
Summary:
Ref T7352. We were previously identifying things by `<daemonClass, overseerPID, startTime>` but that's not unique in a world where one overseer can run multiple daemons.

We already have an internal "daemonID", it just doesn't get written into the DB right now.

Start writing it, then use it to clean up `phd status`.

Test Plan: Ran `phd status`, got more accurate/useful output than previously.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11865
2015-02-24 14:50:37 -08:00
epriestley
48fc3126a1 Support autoscaling daemons in phd
Summary: Ref T7352. This supports passing autoscaling configuration to daemons, and adds `debug --autoscale`.

Test Plan: See D11711.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11860
2015-02-24 14:50:34 -08:00
epriestley
c2d66f29cd Make phd more aware of multiple daemons under a single overseer
Summary: Ref T7352. This makes `phd stop` and `phd status` produce more reasonable output with the new PID file format.

Test Plan: Ran `phd stop`, `phd status`, etc.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11856
2015-02-24 14:50:32 -08:00
epriestley
09f3d0bb7e Pass overseer configuration over stdin
Summary:
Ref T7352. This changes `phd` to pass configuration to overseers over stdin. We still run one overseer per daemon.

The "status" stuff needs some cleanup, but it's mostly just UI/cosmetic.

Test Plan:
  - Ran `phd debug`, `phd launch`, `phd start`, `phd status`, `phd stop`, etc.
  - Verified PID files write in a reasonable format.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7352

Differential Revision: https://secure.phabricator.com/D11855
2015-02-24 14:50:30 -08:00
epriestley
59ae91a5ce Disable caching of remarkup previews
Summary:
We currently cache previews, but the vast majority of previews are never rendered again (e.g., they're a preview of someone partway through typing a comment).

Especially when editing large documents (Legalpad, Phriction), this can bloat the markup cache with data that will never be read and won't get purged for 30 days.

In particular, most of the data on `admin.phacility.com` is currently 1,000 previews of legalpad documents as I made minor edits to them over the course of several hours.

This isn't a big concern, but it's a very easy fix.

Test Plan:
  - Previewed a legalpad document, verified that cache rows were not written as I mashed the keyboard.
  - Saved the document, verified a new cache row was written.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11832
2015-02-22 05:39:25 -08:00
epriestley
29fd3f136b Allow columns to be marked as nonmutable (so save() will not change them)
Summary:
Ref T6840. This feels a little dirty; open to alternate suggestions.

We currently have a race condition where multiple daemons may load a commit and then save it at the same time, when processing "reverts X" text. Prior to this feature, two daemons would never load a commit at the same time.

The "reverts X" load/save has no effect (doesn't change any object properties), but it will set the state back to the loaded state on save(). This overwrites any flag updates made to the commit in the meantime, and can produce the race in T6840.

In other cases (triggers, harbormaster, repositories) we deal with this kind of problem with "append-only-updates + single-consumer", or a bunch of locking. There isn't really a good place to add a single consumer for commits, since a lot of daemons need to access them. We could move the flags column to a separate table, but this feels pretty complicated. And locking is messy, also mostly because we have so many consumers.

Just exempting this column (which has unusual behavior) from `save()` feels OK-ish? I don't know if we'll have other use cases for this, and I like it even less if we never do, but this patch is pretty small and feels fairly understandable (that said, I also don't like that it can make some properties just silently not update if you aren't on the lookout).

So, this is //a// fix, and feels simplest/least-bad for the moment to me, I thiiink.

Test Plan: Added and executed unit tests.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6840

Differential Revision: https://secure.phabricator.com/D11822
2015-02-19 10:37:17 -08:00
Bob Trahan
a77127ab63 Projects - fix translation strings in watcher edge class
Summary: Fixes T7319. These need a "%s" for the count where they had a "%d"

Test Plan: plan in D11812 is no longer a lie! (watcher added / removed strings render correctly)

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7319

Differential Revision: https://secure.phabricator.com/D11813
2015-02-18 15:44:54 -08:00
Bob Trahan
7ef5c52934 Projects - add translation for watcher strings
Summary: Fixes T7319. ...except I can't get this working in my sandbox? Changes to the translation file don't seem to show up. TEST PLAN IS A LIE

Test Plan: became a watcher, un became a watcher - saw sensical translated strings

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7319

Differential Revision: https://secure.phabricator.com/D11812
2015-02-18 15:33:52 -08:00
epriestley
2cd77b5b58 Improve taskmaster behavior on empty queues
Summary:
Right now, taskmasters on empty queues sleep for 30 seconds. With a default setup (4 taskmasters), this averages out to 7.5 seconds between the time you do anything that queues something and the time that the taskmasters start work on it.

On instances, which currently launch a smaller number of taskmasters, this wait is even longer.

Instead, sleep for the number of seconds that there are taskmasters, with a random offset. This makes the average wait to start a task from an empty queue 1 second, and the average maximum load of an empty queue also one query per second.

On loaded instances this doesn't matter, but this should dramatically improve behavior for less-loaded instances without any real tradeoffs.

Test Plan: Started several taskmasters, saw them jitter out of sync and then use short sleeps to give an empty queue about a 1s delay.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11772
2015-02-16 11:30:49 -08:00
epriestley
36494d4e2e Add a "did verify email" event to Phabricator
Summary: Ref T7152. Gives us an event hook so we can go make users a member of any instance they've been invited to as soon as they verify an email address.

Test Plan:
  - Used `bin/auth verify` to trigger the event.
  - Build out the invite flow in rSERVICES.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7152

Differential Revision: https://secure.phabricator.com/D11752
2015-02-11 14:39:06 -08:00
epriestley
d4680a7e4e Update Phabricator to work with more modular translations
Summary:
Ref T7152. Ref T1139. This updates Phabricator so third-party libraries can translate their own stuff. Also:

  - Hide "All Caps" when not in development mode, since some users have found this a little confusing.
  - With other changes, adds a "Raw Strings" mode (development mode only).
  - Add an example silly translation to make sure the serious business flag works.
  - Add a basic British English translation.
  - Simplify handling of translation overrides.

Test Plan:
  - Flipped serious business / development on and off and saw silly/development translations drop off.
  - Switched to "All Caps" and saw all caps.
  - Switched to Very English, Wow!
  - Switched to British english and saw "colour".

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7152, T1139

Differential Revision: https://secure.phabricator.com/D11747
2015-02-11 13:02:35 -08:00
epriestley
6f90fbdef8 Send emails for email invites
Summary:
Ref T7152. Ref T3554.

  - When an administrator clicks "send invites", queue tasks to send the invites.
  - Then, actually send the invites.
  - Make the links in the invites work properly.
  - Also provide `bin/worker execute` to make debugging one-off workers like this easier.
  - Clean up some UI, too.

Test Plan:
We now get as far as the exception which is a placeholder for a registration workflow.

{F291213}

{F291214}

{F291215}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T3554, T7152

Differential Revision: https://secure.phabricator.com/D11736
2015-02-11 06:06:09 -08:00
Joshua Spence
84b0c8e6db Fix a pht method call
Summary: Ref T7046. I missed this in D11680.

Test Plan: `arc lint`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7046

Differential Revision: https://secure.phabricator.com/D11731
2015-02-11 06:54:10 +11:00
Joshua Spence
aaf8d73ec7 Fix pht method calls
Summary: Ref T7046. This is mainly a proof-of-concept for D11661.

Test Plan: `arc lint`

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7046

Differential Revision: https://secure.phabricator.com/D11680
2015-02-10 18:57:45 +11:00
Chad Little
cdd8dcbf17 Update InlineCommentSummary UI
Summary: Minor spring cleaning, improve the visual feel of the comments table, more consistent structure.

Test Plan:
Test multiple comments, long comments, short comments, and multiple lines.

{F282462}

Reviewers: btrahan, epriestley

Reviewed By: epriestley

Subscribers: hach-que, Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11666
2015-02-09 08:38:51 -08:00
epriestley
8c568d88d7 Reduce severity of auth provider warning
Summary:
Ref T7208. Now that we have approvals (new installs are safe by default), take those into account when generating this warning.

Try to soften the warning to cover the case discussed in T7208, hopefully without requiring additional measures.

Test Plan:
{F286014}

{F286015}

Reviewers: btrahan, chad

Reviewed By: chad

Subscribers: epriestley

Maniphest Tasks: T7208

Differential Revision: https://secure.phabricator.com/D11708
2015-02-07 14:45:27 -08:00
Eric
28b23fd789 Use --hex-blob flag in bin/storage dump
Summary: mysqldump output can end up having weird encoding issues when raw BLOBs are in the output, preventing the backup restoration from succeeding. This hex-encodes blobs in the dump from the backup workflow causing the output file to only contain ASCII and ensure imports are successful.

Test Plan: Had issues restoring a backup from the original `mysqldump` command issued by this workflow. Ran the same command with this flag added and I was able to restore the backup.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11704
2015-02-06 12:56:23 -08:00
epriestley
74ea59235a Make the "daemons and web have different config" warning more specific
Summary:
I'm hitting this in the cluster and couldn't figure it out after staring at it for a couple minutes. Produce a better error.

This dumps a hash of each configuration key value which is set to a non-default value into the daemon log. This is much more compact than the full config, and doesn't spread secrets around, so it seems like a good balance between providing information and going crazy with it.

Test Plan: {F284139}

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11699
2015-02-05 14:07:35 -08:00
Bob Trahan
137b0ebc53 Phabot / Conduit - stop using deprecated conduit api paste.info
Summary: Fixes T7111. Also nails out the TODO to show the username

Test Plan: fired up the ole phabot, chatted "P123" and saw "P123: https://secure.phabricator.com/P123 - Masterwork From Distant Lands by epriestley"...!

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin, epriestley

Maniphest Tasks: T7111

Differential Revision: https://secure.phabricator.com/D11664
2015-02-03 13:44:20 -08:00
Joshua Spence
7982b23eb4 Use PhutilXHPASTBinary methods
Summary: Use `PhutilXHPASTBinary` methods instead of `xhpast_parse` functions. Depends on D11517.

Test Plan: N/A, this is a direct swap.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: Korvin, epriestley

Differential Revision: https://secure.phabricator.com/D11612
2015-02-03 06:59:16 +11:00
epriestley
f6015dbb56 Improve the usability of Phortune
Summary:
Ref T6881.

  - Fix dead links.
  - Let implementations provide more information.
  - Provide more information to implementations.

Test Plan: Links work, invoices show billing periods, fewer "Subscription 6" crumbs, all is well in the world.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6881

Differential Revision: https://secure.phabricator.com/D11601
2015-02-01 12:32:48 -08:00
epriestley
d804598f17 Add some of a billing daemon skeleton
Summary:
Ref T6881. This adds the worker, and a script to make it easier to test. It doesn't actually invoice anything.

I'm intentionally allowing the script to double-bill since it makes testing way easier (by letting you bill the same period over and over again), and provides a tool for recovery if billing screws up.

(This diff isn't very interesting, just trying to avoid a 5K-line diff at the end.)

Test Plan: Used `bin/phortune invoice ...` to get the worker to print out some date ranges which it would theoretically invoice.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T6881

Differential Revision: https://secure.phabricator.com/D11577
2015-01-30 11:29:05 -08:00
Bob Trahan
2fc43598b5 Differential - add ability to setup "create" addresses for revisions
Summary: Fixes T1476. The body of the email should be just the output of some diff command.

Test Plan:
git diff master > text.txt; ./bin/mail receive-test --to <configured-diff-create-address> < text.txt; a diff was successfully created...! email generated had a working link to the diff.

./bin/mail receive-test --to <configured-diff-create-address> < README.md; a diff was not created as expected...! email generated had a sensical error message, telling me that the mail body should have been generated via a diff command

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: johnny-bit, Korvin, epriestley

Maniphest Tasks: T1476

Differential Revision: https://secure.phabricator.com/D11574
2015-01-30 10:31:39 -08:00
epriestley
c2efa9065c Raise a setup warning for an unparseable VCS binary version
Summary:
Hit this locally, with an error like:

> Version <empty string> is older than 1.9, the minimum supported version.

(Where `<empty string>` was just the empty string, not literally the text `<empty string>`.)

Be more careful about parsing versions, and parse the newer string.

Test Plan: Got "unknown version" with intentionally-broken test data, then clean readout.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Differential Revision: https://secure.phabricator.com/D11558
2015-01-29 14:28:49 -08:00
Joshua Spence
1ecfa0313c Add a ./bin/storage shell command
Summary: Fixes T7078. Adds a `./bin/storage shell` command which passes through to a MySQL shell. This is slightly more convenient than running `mysql` manually.

Test Plan: Ran `./bin/storage shell` and got a MySQL shell.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: Korvin, epriestley

Maniphest Tasks: T7078

Differential Revision: https://secure.phabricator.com/D11548
2015-01-30 07:15:27 +11:00
epriestley
8798083ad9 Proxy VCS SSH requests
Summary: Fixes T7034. Like HTTP, proxy requests to the correct host if a repository has an Almanac service host.

Test Plan: Ran VCS requests through the proxy.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7034

Differential Revision: https://secure.phabricator.com/D11543
2015-01-28 14:41:24 -08:00
epriestley
9b359affe7 Prepare SSH connections for proxying
Summary:
Ref T7034.

In a cluster environment, when a user connects with a VCS request over SSH (like `git pull`), the receiving server may need to proxy it to a server which can actually satisfy the request.

In order to proxy the request, we need to know which repository the user is interested in accessing.

Split the SSH workflow into two steps:

  # First, identify the repository.
  # Then, execute the operation.

In the future, this will allow us to put a possible "proxy the whole thing somewhere else" step in the middle, mirroring the behavior of Conduit.

This is trivially easy in `git` and `hg`. Both identify the repository on the commmand line.

This is fiendishly complex in `svn`, for the same reasons that hosting SVN was hard in the first place. Specifically:

  - The client doesn't tell us what it's after.
  - To get it to tell us, we have to send it a server capabilities string //first//.
  - We can't just start an `svnserve` process and read the repository out after a little while, because we may need to proxy the request once we figure out the repository.
  - We can't consume the client protocol frame that tells us what the client wants, because when we start the real server request it won't know what the client is after if it never receives that frame.
  - On the other hand, we must consume the second copy of the server protocol frame that would be sent to the client, or they'll get two "HELLO" messages and not know what to do.

The approach here is straightforward, but the implementation is not trivial. Roughly:

  - Start `svnserve`, read the "hello" frame from it.
  - Kill `svnserve`.
  - Send the "hello" to the client.
  - Wait for the client to send us "I want repository X".
  - Save the message it sent us in the "peekBuffer".
  - Return "this is a request for repository X", so we can proxy it.

Then, to continue the request:

  - Start the real `svnserve`.
  - Read the "hello" frame from it and throw it away.
  - Write the data in the "peekBuffer" to it, as though we'd just received it from the client.
  - State of the world is normal again, so we can continue.

Also fixed some other issues:

  - SVN could choke if `repository.default-local-path` contained extra slashes.
  - PHP might emit some complaints when executing the commit hook; silence those.

Test Plan: Pushed and pulled repositories in SVN, Mercurial and Git.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7034

Differential Revision: https://secure.phabricator.com/D11541
2015-01-28 10:18:07 -08:00
epriestley
d8550c114d Promote instance identity to the upstream and pass it to commit hooks
Summary:
Fixes T7019. In a cluster environment, pushes currently fail because the commit hook can't identify the instance.

For web processes, the hostname identifies the instance -- but we don't have a hostname in the hook.

For CLI processes, the environment identifies the instance -- but we don't have an environment in the hook under SVN.

Promote the instance identifier into the upstream and pack/unpack it explicitly for hooks. This is probably not useful for anyone but us, but the amount of special-purpose code we're introducing is very small.

I poked at trying to do this in a more general way, but:

  - We MUST know this BEFORE we run code, so the normal subclassing stuff is useless.
  - I couldn't come up with any other parameter which might ever be useful to pass in.

Test Plan: Used `git push` to push code through proxied HTTP, got a clean push.

Reviewers: btrahan

Reviewed By: btrahan

Subscribers: epriestley

Maniphest Tasks: T7019

Differential Revision: https://secure.phabricator.com/D11495
2015-01-27 14:51:48 -08:00