1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-12-22 05:20:56 +01:00
Commit graph

11249 commits

Author SHA1 Message Date
epriestley
99e4472447 Soften checks on a very old Maniphest transactionmigration
Summary:
Ref T9464. If an ancient transaction doesn't have array values for whatever reason, we fail here.

Instead, just recover as gracefully as we can. We may get the transaction "wrong" in some sense, but this only impacts what is rendered in the transaction log.

Test Plan: This is nearly a year old and there's no real way to test it.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9464

Differential Revision: https://secure.phabricator.com/D14149
2015-09-23 19:18:44 -07:00
epriestley
3379904237 Allow Drydock leases to expire after a time limit
Summary: Ref T6569. If a lease is activated with an expiration date, schedule a task to try to clean it up after that time.

Test Plan:
  - Used `bin/drydock lease ... --until ...` to activate a lease in the near future.
  - Waited for a bit.
  - Saw it expire and get destroyed at the scheduled time.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T6569

Differential Revision: https://secure.phabricator.com/D14148
2015-09-23 13:54:27 -07:00
epriestley
fcb6d1e2fa Strip some obsolete code out of Drydock
Summary:
Ref T9252. This simplifies some Drydock code.

Most of this code relates to the old notion of Drydock being able to enumerate all the tasks it needs to complete in order to acquire a lease. The code has stepped back from this, since it's unnecessary, the queue is more powerful than it used to be, and it would be a lot of work to keep track of.

The ~only thing that should ever wait for leases in modern code is `bin/drydock lease`, and it's fine for it to just sit there sleeping, so this just does that.

This reduces the granularity of logging, but I'll address that separately in future logging-focused changes.

Test Plan: Used `bin/drydock lease` to acquire a lease, saw it acquire cleanly.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14147
2015-09-23 13:21:41 -07:00
Chad Little
be9cc235b2 Add Application Routes to Phame AppSearch queries
Summary: Fixes T9388, lays in basic ApplicationSearch.

Test Plan: Build a dashboard with Posts and Blogs, click on search icon, get sent to correct page.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9388

Differential Revision: https://secure.phabricator.com/D14146
2015-09-23 12:48:19 -07:00
epriestley
1f311d64c6 Give Drydock resources and leases a real "destroy" lifecycle phase
Summary: Ref T9252. Some leases or resources may need to remove data, tear down VMs, etc., during cleanup. After they are released, queue a "destroy" phase for performing teardown.

Test Plan:
  - Used `bin/drydock lease ...` to create a working copy lease.
  - Used `bin/drydock release-lease` and `bin/drydock release-resource` to release the lease and then the working copy and host.
  - Saw working copy and host get destroyed and cleaned up properly.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T6569, T9252

Differential Revision: https://secure.phabricator.com/D14144
2015-09-23 11:20:20 -07:00
epriestley
0a37145072 [drydock/core] Show blueprints / resources as links in Drydock view controllers
Summary: Ref T2015. This updates the blueprint / resource references in the Drydock view controllers to render as handles.

Test Plan: Viewed the controllers, saw links.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: devurandom, joshuaspence, Korvin, epriestley

Maniphest Tasks: T2015

Differential Revision: https://secure.phabricator.com/D10873
2015-09-23 10:52:23 -07:00
epriestley
789df89c84 Add a command queue to Drydock to manage lease/resource release
Summary:
Ref T9252. Broadly, Drydock currently races on releasing objects from the "active" state. To reproduce this:

  - Scatter some sleep()s pretty much anywhere in the release code.
  - Release several times from web UI or CLI in quick succession.

Resources or leases will execute some release code twice or otherwise do inconsistent things.

(I didn't chase down a detailed reproduction scenario for this since inspection of the code makes it clear that there are no meaningful locks or mechanisms preventing this.)

Instead, add a Harbormaster-style command queue to resources and leases. When something wants to do a release, it adds a command to the queue and schedules a worker. The workers acquire a lock, then try to consume commands from the queue.

This guarantees that only one process is responsible for writes to active resource/leases.

This is the last major step to giving resources and leases a single writer during all states:

  - Resource, Unsaved: AllocatorWorker
  - Resource, Pending: ResourceWorker (Possible rename to "Allocated?")
  - Resource, Open: This diff, ResourceUpdateWorker. (Likely rename to "Active").
  - Resource, Closed/Broken: Future destruction worker. (Likely rename to "Released" / "Broken"; maybe remove "Broken").
  - Resource, Destroyed: No writes.
  - Lease, Unsaved: Whatever wants the lease.
  - Lease, Pending: AllocatorWorker
  - Lease, Acquired: LeaseWorker
  - Lease, Active: This diff, LeaseUpdateWorker.
  - Lease, Released/Broken: Future destruction worker (Maybe remove "Broken"?)
  - Lease, Expired: No writes. (Likely rename to "Destroyed").

In most phases, we can already guarantee that there is a single writer without doing any extra work. This is more complicated in the "Active" case because the release buttons on the web UI, the release tools on the CLI, the lease requestor itself, the garbage collector, and any other release process cleaning up related objects may try to effect a release. All of these could race one another (and, in many cases, race other processes from other phases because all of these get to act immediately) as this code is currently written. Using a queue here lets us make sure there's only a single writer in this phase.

One thing which is notable is that whatever acquires a lease **can not write to it**! It is never the writer once it queues the lease for activation. It can not write to any resources, either. And, likewise, Blueprints can not write to resources while acquiring or releasing leases.

We may need to provide a mechinism so that blueprints and/or resource/lease holders get to attach some storage to resources/leases for bookkeeping. For example, a blueprint might need to keep some kind of cache on a resource to help it manage state. But I think we can cross that bridge when we come to it, and nothing else would need to write to this storage so it's technically straightforward to introduce such a mechanism if we need one.

Test Plan:
  - Viewed buttons in web UI, checked enabled/disabled states.
  - Clicked the buttons.
  - Saw commands show up in the command queue.
  - Saw some daemon stuff get scheduled.
  - Ran CLI tools, saw commands get consumed and resources/leases release.

Reviewers: hach-que, chad

Reviewed By: chad

Maniphest Tasks: T9252

Differential Revision: https://secure.phabricator.com/D14143
2015-09-23 07:42:08 -07:00
epriestley
8ded0927aa Recover gracefully from Conduit failure when building "Tags" field in commit mail
Summary:
Ref T9458. This is basically the same as D13319, but the "Tags" field didn't get covered in that change.

Specifically, the issue is:

  - We try to generate mail to a disabled user (later, we'll drop it without delivering it, but that filtering doesn't happen yet).
  - The disabled user doesn't have permission to use Conduit (or any other Conduit-related problem occurs).
  - We fail here, then retry generating the mail again later.

Instead, just degrade to not building the field and showing what went wrong.

Test Plan:
  - Pushed some commits, saw mail generate.
  - Added a fake exception to the field, saw the mail generate with an error message.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9458

Differential Revision: https://secure.phabricator.com/D14142
2015-09-22 13:03:29 -07:00
Chad Little
799d86dc65 Show file UI box as collapsed in Diffusion
Summary: This UI should have a Collapsed PHUIObjectBoxView.

Test Plan: review a file in sandbox

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14139
2015-09-21 12:29:17 -07:00
Chad Little
b8567f1764 Use setTable on File Transforms tables
Summary: Minor, but nicer.

Test Plan: Review File Transforms page, see new UI.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14140
2015-09-21 12:29:05 -07:00
June Rhodes
d5dc4588fc [harbormaster/abort-builds] Support aborting builds in Harbormaster
Summary: Ref T1049.  This adds support for aborting builds in Harbormaster.

Test Plan: Tested it in production.

Reviewers: epriestley, #blessed_reviewers

Reviewed By: epriestley, #blessed_reviewers

Subscribers: yelirekim, traviscline, joshuaspence, Korvin, epriestley

Projects: #harbormaster

Maniphest Tasks: T1049

Differential Revision: https://secure.phabricator.com/D11870
2015-09-21 12:07:38 -07:00
Chad Little
26552a588b Tweakify the yellow in code embeds
Summary: This runs 50% of our stock 'lightyellow'.

Test Plan: staring intensifies

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14138
2015-09-21 11:52:04 -07:00
epriestley
f1119ffcf5 Support working copies and separate allocate + activate steps for resources/leases in Drydock
Summary:
Ref T9253. For resources and leases that need to do something which takes a lot of time or requires waiting, allow them to allocate/acquire first and then activate later.

When we allocate a resource or acquire a lease, the blueprint can either activate it immediately (if all the work can happen quickly/inline) or activate it later. If the blueprint activates it later, we queue a worker to handle activating it.

Rebuild the "working copy" blueprint to work with this model: it allocates/acquires and activates in a separate step, once it is able to acquire a host.

Test Plan: With some power of imagination, brought up a bunch of working copies with `bin/drydock lease --type working-copy ...`

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14127
2015-09-21 04:46:24 -07:00
epriestley
6a0eb9d84b Allow AlmanacHost blueprints to build a meaningful CommandInterface
Summary: Ref T9253. Provide a meaningful command interface for Almanac hosts.

Test Plan:
Configued and leased a real host (`sbuild001.phacility.net`) and ran a command on it.

```
$ ./bin/drydock command --lease 90 -- ls /
bin
boot
core
dev
etc
home
initrd.img
lib
lib64
lost+found
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var
vmlinuz
```

Reviewers: chad, hach-que

Reviewed By: chad, hach-que

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14126
2015-09-21 04:46:02 -07:00
epriestley
9a270efe8a Tidy up some Drydock UI
Summary: Ref T9253. We had some un-modern use of UI elements, clean that up. Add a tab for showing slot locks so you don't have to fish around in the database.

Test Plan: Looked at blueprints, resources and leases. Looked at slot locks.

Reviewers: chad, hach-que

Reviewed By: chad, hach-que

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14119
2015-09-21 04:45:43 -07:00
epriestley
3ac99006bf Implement optimistic "slot locks" in Drydock
Summary:
See discussion in D10304. There's a lot of context there, but the general idea is:

  - Blueprints should manage locks in a granular way during the actual allocation/acquisition phase.
  - Optimistic "slot locks" might a pretty good primitive to make that easy to implement and reason about in most cases.

The way these locks work is that you just pick some name for the lock (like the PHID of a resource) and say that it needs to be acquired for the allocation/acquisition to work:

```
...
->needSlotLock("mylock(PHID-XYZQ-...)")
...
```

When you fire off the acquisition or allocation, it fails unless it could acquire the slot with that name. This is really simple (no explicit lock management) and a pretty good fit for most of the locking that blueprints and leases need to do.

If you need to do limit-based locks (e.g., maximum of 3 locks) you could acquire a lock like this:

```
mylock(whatever).slot(2)
```

Blueprints generally only contend with themselves, so it's normally OK for them to pick whatever strategy works best for them in naming locks.

This may not work as well if you have a huge number of slots (e.g., 100TB you want to give out in 1MB chunks), or other complex needs for locks (like you have to synchronize access to some external resource), but slot locks don't need to be the only mechanism that blueprints use. If they run into a problem that slot locks aren't a good fit for, they can use something else instead. For now, slot locks seem like a good fit for the problems we currently face and most of the problems I anticipate facing.

(The release workflows have other race issues which I'm not addressing here. They work fine if nothing races, but aren't race-safe.)

Test Plan:
To create a race where the same binding is allocated as a resource twice:

  - Add `sleep(10)` near the beginning of `allocateResource()`, after the free bindings are loaded but before resources are allocated.
  - (Comment out slot lock acquisition if you have this patch.)
  - Run `bin/drydock lease ...` in two windows, within 10 seconds of one another.

This will reliably double-allocate the binding because both blueprints see a view of the world where the binding is free.

To verify the lock works, un-comment it (or apply this patch) and run the same test again. Now, the lock fails in one process and only one resource is allocated.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Differential Revision: https://secure.phabricator.com/D14118
2015-09-21 04:45:25 -07:00
epriestley
6e03419593 Implement a rough AlmanacService blueprint in Drydock
Summary:
Ref T9253. Broadly, this realigns Allocator behavior to be more consistent and straightforward and amenable to intended future changes.

This attempts to make language more consistent: resources are "allocated" and leases are "acquired".

This prepares for (but does not implement) optimistic "slot locking", as discussed in D10304. Although I suspect some blueprints will need to perform other locking eventually, this does feel like a good fit for most of the locking blueprints need to do.

In particular, I've made the blueprint operations on `$resource` and `$lease` objects more purposeful: they need to invoke an activator on the appropriate object to be implemented correctly. Before they invoke this activator method, they configure the object. In a future diff, this configuration will include specifying slot locks that the lease or resource must acquire. So the API will be something like:

  $lease
    ->setActivateWhenAcquired(true)
    ->needSlotLock('x')
    ->needSlotLock('y')
    ->acquireOnResource($resource);

In the common case where slot locks are a good fit, I think this should make correct blueprint implementation very straightforward.

This prepares for (but does not implement) resources and leases which need significant setup steps. I've basically carved out two modes:

  - The "activate immediately" mode, as here, immediately opens the resource or activates the lease. This is appropriate if little or no setup is required. I expect many leases to operate in this mode, although I expect many resources will operate in the other mode.
  - The "allocate now, activate later" mode, which is not fully implemented yet. This will queue setup workers when the allocator exits. Overall, this will work very similarly to Harbormaster.
  - This new structure makes it acceptable for blueprints to sleep as long as they want during resource allocation and lease acquisition, so long as they are not waiting on anything which needs to be completed by the queue. Putting a `sleep(15 * 60)` in your EC2Blueprint to wait for EC2 to bring a machine up will perform worse than using delayed activation, but won't deadlock the queue or block any locks.

Overall, this flow is more similar to Harbormaster's flow. Having consistency between Harbormaster's model and Drydock's model is good, and I think Harbormaster's model is also simply much better than Drydock's (what exists today in Drydock was implemented a long time ago, and we had more support and infrastructure by the time Harbormaster was implemented, as well as a more clearly defined problem).

The particular strength of Harbormaster is that objects always (or almost always, at least) have a single, clearly defined writer. Ensuring objects have only one writer prevents races and makes reasoning about everything easier.

Drydock does not currently have a clearly defined single writer, but this moves us in that direction. We'll probably need more primitives eventually to flesh this out, like Harbormaster's command queue for messaging objects which you can't write to.

This blueprint was originally implemented in D13843. This makes a few changes to the blueprint itself:

  - A bunch of code from that (e.g., interfaces) doesn't exist yet.
  - I let the blueprint have multiple services. This simplifies the code a little and seems like it costs us nothing.

This also removes `bin/drydock create-resource`, which no longer makes sense to expose. It won't get locking, leasing, etc., correct, and can not be made correct.

NOTE: This technically works but doesn't do anything useful yet.

Test Plan: Used `bin/drydock lease --type host` to acquire leases against these blueprints.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Subscribers: Mnkras

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14117
2015-09-21 04:43:53 -07:00
epriestley
bb28f94f9b Reduce garbage-level of Drydock Allocator implementation
Summary:
Ref T9253. The Drydock allocator is very pseudocodey right now. Particularly, it was written before Blueprints were concrete.

Reorganize it to make its responsibilities and error handling behaviors more clear.

In particular, the Allocator does not manage locks. It's primarily trying to reject allocations which can not possibly work. Blueprints are responsible for locks. See some discussion in D10304.

NOTE: This code probably doesn't work as written, see future diffs.

Test Plan: See future diffs.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Subscribers: cburroughs

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14114
2015-09-21 04:43:25 -07:00
epriestley
5362d3366c Modernize Drydock Query + Attach code
Summary:
Ref T9253. Some of the Drydock code is pretty old. This applies standard modernizations to it:

  - Modernize Query classes to use stuff like `buildWhereClauseParts()` and `loadStandardPage()`.
  - Modernize all the getX() / attachX() stuff. In particular:
    - Require and attach implementations to Blueprints.
    - Require and attach Blueprints to Resources.
    - BlueprintImplementations are now always unique per-Blueprint so they can store/cache state if they want without running over one another.
    - BlueprintImplementations are now passed a `$blueprint`, like other similar APIs (this could go various ways but I generally like this as a balance of concerns).

NOTE: This probably doesn't run on its own, I'm just trying to split the next diff (core allocator stuff) up a bit and these pieces are all pretty standard.

Test Plan:
  - Not much; see next revision or two.
  - Clicked around Resource and Blueprint lists.

Reviewers: chad, hach-que

Reviewed By: chad, hach-que

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14113
2015-09-21 04:42:04 -07:00
epriestley
635e9c6075 Provide a generic "Datasource" StandardCustomField
Summary:
Ref T9253. See discussion in D13843.

I want to let Drydock blueprints for Almanac services choose those services from a typeahead, but only list appropriate services in the typeahead. To do this:

  - Provide a StandardCustomField for an arbitrary datasource.
  - Adjust the AlmanacServiceDatasource to allow filtering by service class.

This implementation is substantially the same as the one in D13843, with some adjustments:

  - I lifted most of the code in the `Users` standard custom field into a new `Tokenizer` standard custom field.
  - The `Users` and `Datasource` custom fields now extend the `Tokenizer` custom field and can share most of the code it uses.
  - I exposed this field fully as a configurable field. I don't think anyone will ever use it, but this generality costs us nearly nothing and improves consistency.
  - The code in D13843 didn't actually pass the parameters over the wire, since the object that responds to the request is not the same object that renders the field. Use the "parameters" mechanism in datasources to get things passed over the wire.

Test Plan:
  - Created a custom "users" field in Maniphest and made sure it still wokred.
  - Created a custom "almanc services" field in Maniphest and selected some services for a task.
  - With additional changes from D13843, selected an appropriate Almanac service in a new Drydock blueprint.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14111
2015-09-21 04:41:52 -07:00
epriestley
c44f9d80de Remove DrydockPreallocatedHostBlueprintImplementation
Summary:
Ref T9253. This comes from a time before Almanac. Now that we have Almanac, it makes much more sense to put this logic there than to try to put it in Drydock itself.

Remove the preallocated host blueprint, a relic of a bygone time.

Test Plan: Grepped for callsites.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14110
2015-09-21 04:41:40 -07:00
epriestley
d6514321b1 Add an Almanac service type for Drydock to lease against
Summary: Ref T9253. See D13843 for some discussion. This is very bare-bones for now since I believe that almost all interesting configuration (e.g., credentials) should live in Drydock, although I imagine it getting some configuration eventually.

Test Plan: Used {nav Almanac > Services > Create Service} to create a new service of this type.

Reviewers: hach-que, chad

Reviewed By: hach-que, chad

Maniphest Tasks: T9253

Differential Revision: https://secure.phabricator.com/D14109
2015-09-21 04:41:23 -07:00
epriestley
a0ed843d47 Don't allow welcome mail to be sent to users who can't login
Summary:
Fixes T9446. We allow administrators to send "Welcome" mail to bots and mailing lists.

This is harmless (these links do not function), but confusing.

Instead, disable this option in the UI and explain why it is disabled when it is clicked. Also prevent generation of this mail lower in the stack.

Test Plan:
  - Viewed a bot page, saw action disabled, clicked it, got explanation.
  - Viewed a normal user page, saw action enabled, clicked it, sent welcome email.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9446

Differential Revision: https://secure.phabricator.com/D14134
2015-09-20 04:28:33 -07:00
Chad Little
9ea6249a18 Shrink aphront table headers
Summary: These already have a larger font, the extra height isn't needed. Make them the same size as `td`

Test Plan: Look at a bunch of tables.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14135
2015-09-19 19:42:19 -07:00
Chad Little
666f19e504 Make icon setting in Section Headers easier/consistent
Summary: You can already pass other icons, but this makes it a bit simpler.

Test Plan: Test Maniphest, Badges

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14131
2015-09-19 11:29:01 -07:00
Chad Little
9c43853815 Restore the delicate balance of the universe 2015-09-18 13:59:11 -07:00
Chad Little
36c8df3ef9 Add pageObjects to Macro
Summary: Fixes T9442

Test Plan: Like a macro, log out, switch to notchad, clear notification by visiting page

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9442

Differential Revision: https://secure.phabricator.com/D14129
2015-09-18 13:58:07 -07:00
Chad Little
ce7aa92c67 Normalize Paste, Remarkup code embed colors
Summary: These currently use different yellows, and more importantly when pygmentized, is hard to read. This lightens up code embeds and pastes.

Test Plan:
Review Diviner docs, Pastes, and normal code embeds.

{F820471}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: tycho.tatitscheff, Korvin

Differential Revision: https://secure.phabricator.com/D14123
2015-09-17 08:23:22 -07:00
Chad Little
f899762364 Better Paste layout on mobile
Summary: Builds a container of paste, makes it smaller on mobile.

Test Plan: View on desktop, tablet, mobile.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14122
2015-09-17 08:22:40 -07:00
Chad Little
2e0cbaa366 Usability updates to ActionPanel
Summary: Wraps entire element in the anchor tag, gives a hover state, makes icons bounce.

Test Plan: Hover and click.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14124
2015-09-17 08:22:23 -07:00
Chad Little
23b2653f52 More ActionPanel colors, hardening
Summary: Adds full ROYGBIVP color spectrum, adds basic overflow, collapse protection.

Test Plan: Review small and large panels are various breakpoints.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14120
2015-09-16 09:22:31 -07:00
Chad Little
a62337dcd8 Update PHUIActionPanelView
Summary: Making these a little more fun, a little more flexible and better looking. Will have an update for rSAAS in a bit.

Test Plan:
Make lots of them. Click.

{F815658}

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Differential Revision: https://secure.phabricator.com/D14115
2015-09-15 18:32:01 -07:00
epriestley
0449a07f53 Add bin/auth unlimit and clean up a TODO
Summary:
I stumbled across this TODO and was worried that there was a glaring hole in MFA that I'd somehow forgotten about, but the TODO is just out of date.

These actions are rate limited properly by `PhabricatorAuthTryFactorAction`, which permits a maximum of 10 actions per hour.

  - Remove the TODO.
  - Add `bin/auth unlimit` to make it easier to reset rate limits if someone needs to do that for whatever reason.

Test Plan:
  - Tried to brute force through MFA.
  - Got rate limited properly after 10 failures.
  - Reset rate limit with `bin/auth unlimit`.
  - Saw the expected number of actions clear.

{F805288}

Reviewers: chad

Reviewed By: chad

Subscribers: joshuaspence

Differential Revision: https://secure.phabricator.com/D14105
2015-09-14 07:03:39 -07:00
epriestley
6bd8ee861c Use PEAR Text_Figlet to render figlet fonts
Summary:
Ref T7785. Makes Figlet available without installing the `figlet` package.

The PEAR Text_Figlet code is really sketchy and includes this API, which is quite marvelous:

```
    function loadFont($filename, $loadgerman = true)
```

At some point, this should probably be rewritten into a modern style, but it's not trivial since the figlet file format and rendering engine are somewhat complicated. I made some adjustments:

  - Broke the dependency on the PEAR core.
  - Prevented it from doing any wrong HTML escaping.
  - Looked through it for any glaring security or correctness problems.

This code isn't very pretty or modern, but as far as I can tell it's safe and does render Figlet fonts in a reasonable way.

Test Plan: {F803268}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9408, T7785

Differential Revision: https://secure.phabricator.com/D14102
2015-09-13 12:31:07 -07:00
epriestley
935ced1edd Include "Figlet" and PEAR "Text_Figlet" in externals
Summary:
Ref T7785. This prepares for (but does not yet use) a pure PHP implementation of Figlet parsing and rendering.

Figlet is somewhat complex, but a parser already exists in PEAR. I'll make sure it's suitable and hook it up in the next diff.

Test Plan: N/A, code not reachable

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9408, T7785

Differential Revision: https://secure.phabricator.com/D14101
2015-09-13 12:30:48 -07:00
epriestley
c705c8011e Use PHP implementation of Cowsay for cowsay rule
Summary:
Ref T7785. Convert the Cowsay Remarkup rule to use a PHP implementation so we don't have to execute an external `cowsay` binary.

I removed some of the default ".cow" files that come with Cowsay because they:

  - include Perl code which we can not interpret; or
  - are primarily in-jokes or standalone visual puns or artwork rather than usable actors on the grand stage of cowsay; or
  - offended my delicate sensibilities.

Users can add new cows to `resources/cows/custom/` if they want to make new cows available.

I have included a majestic original artwork depicting the "Companion Cube" character from //Portal//.

Test Plan: {F802535}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9408, T7785

Differential Revision: https://secure.phabricator.com/D14100
2015-09-13 12:27:30 -07:00
epriestley
c02f750267 Remove dot/Graphviz Remarkup rule
Summary: Ref T9408. This rule is unsafe in principle, and a practical vulnerability has been found by a security researcher.

Test Plan: `grep`

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9408

Differential Revision: https://secure.phabricator.com/D14103
2015-09-13 12:27:23 -07:00
Chad Little
d199560a6b Add a box around the pager in Diffusion
Summary: Fixes T9392, adds some sweet sweet margin to the pager.

Test Plan: See pager with new padding, test different pages, breakpoints.

Reviewers: epriestley

Reviewed By: epriestley

Subscribers: Korvin

Maniphest Tasks: T9392

Differential Revision: https://secure.phabricator.com/D14098
2015-09-11 08:59:56 -07:00
epriestley
1c45a7d8e2 Revert "Allow search results to be snippeted, roughly"
Summary:
This reverts commit 1583738842.

See T8646 for discussion. This version of the feature feels terrible on real data.

Test Plan: Strict revert.

Reviewers: chad

Reviewed By: chad

Differential Revision: https://secure.phabricator.com/D14097
2015-09-10 20:57:26 -07:00
epriestley
6679a85ee1 Leafy vegetables from out-of-order lands. 2015-09-10 19:52:36 -07:00
epriestley
4e181a5611 Clean up browse/history links in Diffusion
Summary:
Fixes T9126. In particular:

  - Add "Browse" links to all history views.
  - Use icons to show "Browse" and "History" links, instead of text.
  - Use FontAwesome.
  - Generally standardize handling of these elements.

This might need a little design attention, but I think it's an improvement overall.

Test Plan:
  - Viewed repository history.
  - Viewed branch history.
  - Viewed file history.
  - Viewed table of contents on a commit.
  - Viewed merged changes on a merge commit.
  - Viewed a directory containing an external.
  - Viewed a deleted file.

{F788419}

{F788420}

{F788421}

{F788422}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9126

Differential Revision: https://secure.phabricator.com/D14096
2015-09-10 19:28:49 -07:00
epriestley
1583738842 Allow search results to be snippeted, roughly
Summary:
Ref T8646. This is fairly rough:

This interface is very niche, and not really flexible enough to accommodate other result customization (but I don't think we have any plans here)?

I'm just //summarizing// the content of documents, basically showing the first paragraph of their content, summary, etc. This isn't what Google does: it shows snippets surrounding the actual search terms. However, this is more involved and might be less useful in structured data: for example, I'd imagine that the first line of most phriciton documents, maniphest tasks and Differential revisions really might be the best machine-generatable summary of them. The actual contextual snippeting in Google doesn't often seem hugely useful to me. But this might also not be very useful.

There's not much design, not sure if you had any ideas.

I only implemented this for tasks, revisions and the wiki since those seem most useful.

I'm generally on the fence about this, but it's not a ton of work to swap out for something else later. Maybe we can see how it feels? But happy to toss it or rethink the approach.

Test Plan: {F788026}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8646

Differential Revision: https://secure.phabricator.com/D14095
2015-09-10 19:06:36 -07:00
Mihir Kedia
ae0348aac9 Add dialog to purge opcode/data caches
Summary: Reachable via the cache config page, restricted to admins only. This makes it convenient to hotfix phabricator without requiring a restart.

Test Plan:
  - Local dev machine doesn't have apc, so I get the not installed message.
  - Faked the name and isEnabled parameters, verified dialog shows up as expected.
  - Didn't test clear code

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: tycho.tatitscheff, joshuaspence, Korvin

Differential Revision: https://secure.phabricator.com/D14064
2015-09-10 14:19:02 -07:00
epriestley
f8080ce931 Add CustomField support to Owners
Summary: Fixes T9351. This is straightforward since this application is now relatively modern and doesn't have any bizarre craziness.

Test Plan:
{F787981}

{F787982}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9351

Differential Revision: https://secure.phabricator.com/D14093
2015-09-10 13:32:31 -07:00
epriestley
093a625698 Show users what's wrong when they try to edit an inline with an editor already open
Summary: Fixes T8572. Ideally we would probably just permit this, but clean up the behavior until the day arrives when inline code is actually rewritten.

Test Plan:
  - Tried to launch editors in Differential and Diffusion while comments were already open.
  - Verified that "Jump to inline" works in both cases.

{F788008}

{F788009}

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T8572

Differential Revision: https://secure.phabricator.com/D14094
2015-09-10 11:36:38 -07:00
epriestley
738cb1fa78 Fix Mercurial unable to authenticate with HTTP when pulling
Summary: As described in T7959, it looks like Diffusion does not provide Mercurial the required HTTP credentials when pulling from an external repository.

Test Plan: Add an external Mercurial repository to Diffusion, that requires HTTP authentication. A private BitBucket repository for example.

Reviewers: #blessed_reviewers, epriestley

Reviewed By: #blessed_reviewers, epriestley

Subscribers: epriestley

Projects: #mercurial, #diffusion

Differential Revision: https://secure.phabricator.com/D14092
2015-09-10 05:40:14 -07:00
epriestley
25786df3a1 Give "Arcanist Client Results" a more clear name
Summary: Fixes T9380. See that task for discussion. This doesn't feel awesome but is maybe the least-bad fix? I think this name is clearer.

Test Plan: Looked at autoplan in Harbormaster, saw new name.

Reviewers: meitros, chad

Reviewed By: chad

Maniphest Tasks: T9380

Differential Revision: https://secure.phabricator.com/D14088
2015-09-09 19:18:08 -07:00
epriestley
de01f3e2e0 Add Maniphest Task email creator to CCs
Summary: Fixes T9369.

Test Plan:
  - Sent a mail with Mail.app to `bugs@local.phacility.com`.
  - Used "View Raw Mail", copy-pasted it into `mail.txt` on disk.
  - Ran `cat mail.txt | ./scripts/mail/manage_mail.php --process-duplicates`.
  - Saw task get created and me get added as CC.
  - Changed "To" to include another user, ran command again, saw task get created and other user get added as CC.

Reviewers: chad

Reviewed By: chad

Subscribers: Korvin

Maniphest Tasks: T9369

Differential Revision: https://secure.phabricator.com/D14086
2015-09-09 14:07:07 -07:00
epriestley
345cc66f0f Fix some missing "@" in doclinks
Summary: Fixes T9368.

Test Plan: Grepped for "Adding New Classes".

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9368

Differential Revision: https://secure.phabricator.com/D14087
2015-09-09 14:06:49 -07:00
epriestley
d2374c468f Fix missing willUpload hook for drag-and-drop chunked upload
Summary:
Ref T9324. When you upload a normal file, we call `willUpload` and then `didUpload`.

When you upload a chunked file, we never call `willUpload`. This can get things out of sync. Make sure we invoke this event for both chunked and non-chunked uploads.

Test Plan: Got cleaner behavior (redirect after all uploads finish) in new Phacility file upload area.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9324

Differential Revision: https://secure.phabricator.com/D14083
2015-09-08 16:20:58 -07:00