Summary:
Fixes T9446. We allow administrators to send "Welcome" mail to bots and mailing lists.
This is harmless (these links do not function), but confusing.
Instead, disable this option in the UI and explain why it is disabled when it is clicked. Also prevent generation of this mail lower in the stack.
Test Plan:
- Viewed a bot page, saw action disabled, clicked it, got explanation.
- Viewed a normal user page, saw action enabled, clicked it, sent welcome email.
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T9446
Differential Revision: https://secure.phabricator.com/D14134
Summary: See D14025. In all cases where we compare hashes, use strict, constant-time comparisons.
Test Plan: Logged in, logged out, added TOTP, ran Conduit, terminated sessions, submitted forms, changed password. Tweaked CSRF token, got rejected.
Reviewers: chad
Reviewed By: chad
Subscribers: chenxiruanhai
Differential Revision: https://secure.phabricator.com/D14026
Summary: Shows badges on profile if you have them. Check if app is installed, show badges.
Test Plan:
Gave myself a liberal selection of badge. Gave notchad one badge. Gave chadtwo absolutely nothing.
{F651069}
Reviewers: btrahan, lpriestley, epriestley
Reviewed By: epriestley
Subscribers: johnny-bit, epriestley, Korvin
Differential Revision: https://secure.phabricator.com/D13692
Summary: Ref T8888, Makes People Flaggable (and makes me wonder if we should rename Flags->Bookmarks).
Test Plan: Flag myself. Get excited.
Reviewers: btrahan, epriestley
Reviewed By: epriestley
Subscribers: joshuaspence, epriestley, Korvin
Maniphest Tasks: T8888
Differential Revision: https://secure.phabricator.com/D13654
Summary:
Ref T8631. The query plan for feed stories is really bad right now, because we miss caches we should be hitting:
- The workspace cache is stored at each query, so adjacent queries can't benefit from the cache (only subqueries). Feed has primarily sibling queries.
- There is no technical reason to do this. Store the workspace cache on the root query, so sibling queries can hit it.
- In `ObjectQuery`, we check the workspace once, then load all the PHIDs. When the PHIDs are a mixture of transactions and objects, we always miss the workspace and load the objects twice.
- Instead, check the workspace after loading each type of object.
- `HandleQuery` does not set itself as the parent query for `ObjectQuery`, so handles never hit the workspace cache.
- Pass it, so they can hit the workspace cache.
- Feed's weird `PhabricatorFeedStory::loadAllFromRows()` method does not specify a parent query on its object/handle queries.
- Just declare the object query to be the "root" query until this eventually gets cleaned up.
Test Plan: Saw queries for each object drop from 4-6x to 1x in `/feed/`.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8631
Differential Revision: https://secure.phabricator.com/D13479
Summary:
Fixes T8601. To reproduce the problem:
- Set your time preference to `""` (the empty string). This isn't possible from the modern UI, but can be done with "Right Click > Inspect Element", or users may have carried it forward from an older setting (this is the case with me and @hach-que on this install).
- Load Calendar with some events.
- This parses an epoch, which sets `valueTime` to `""` (since there are no format characters in the preference) and then `getEpoch()` fails because `strlen($time)` is 0.
- Since `getEpoch()` failed, `getDateTime()` also fails.
To fix this:
- Only permit the date and time preferences to have valid values.
Test Plan:
- Loaded page before patch, saw fatal.
- Applied patch.
- No more fatal.
- Viewed tooltips, dates/times, dates/times in other apps.
- Changed my preferences, saw them respected.
Reviewers: lpriestley
Reviewed By: lpriestley
Subscribers: epriestley, hach-que
Maniphest Tasks: T8601
Differential Revision: https://secure.phabricator.com/D13346
Summary:
Ref T8377. This adds a standard disable/enable feature to Spaces, with a couple of twists:
- You can't create new stuff in an archived space, and you can't move stuff into an archived space.
- We don't show results from an archived space by default in ApplicationSearch queries. You can still find these objects if you explicitly search for "Spaces: <the archived space>".
So this is a "put it in a box in the attic" sort of operation, but that seems fairly nice/reasonable.
Test Plan:
- Archived and activated spaces.
- Used ApplicationSearch, which omitted archived objects by default but allowed searches for them, specifically, to succeed.
- Tried to create objects into an archived space (this is not allowed).
- Edited objects in an archived space (this is OK).
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8377
Differential Revision: https://secure.phabricator.com/D13238
Summary:
Ref T8449. Try out some more subtle behaviors:
- Make the "Space" control part of the policy control, so the UI shows "Visible To: [Space][Policy]". I think this helps make the role of spaces more clear. It also makes them easier to implement.
- Don't show the default space in headers: instead, show nothing.
- If the user has access to only one space, pretend spaces don't exist (no edit controls, no header stuff).
This might be confusing, but I think most of the time it will all align fairly well with user expectation.
Test Plan:
- Viewed a list of pastes (saw Space with non-default space, no space with default space, no space with user in only one space).
- Viewed a paste (saw Space with non-default space, saw no space with default space, saw no space with user in only one space).
- Edited spaces on objects (control as privileged user, no control as locked user).
- Created a new paste in a space (got space select as privileged user, no select as locked user).
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8449
Differential Revision: https://secure.phabricator.com/D13229
Summary: Ref T8496. In D13123, the condition for establishing a web session was made too strict: we need to let non-activated users establish web sessions in order to see "you are a bad disabled person" or "your account needs approval" messages. The previous behavior let them in, the new behavior incorrectly locks them out.
Test Plan: Enabled login approvals and registered a new account with username/password auth.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: joshuaspence, epriestley
Maniphest Tasks: T8496
Differential Revision: https://secure.phabricator.com/D13239
Summary:
We can end up here with a stack trace like this, while rendering an embedded Slowvote trying to publish a Feed story:
```
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] [2015-06-08 22:49:57] EXCEPTION: (PhutilProxyException) Error while executing Task ID 830591. {>} (PhabricatorDataNotAttachedException) Attempting to access attached data on PhabricatorUser (via getAlternateCSRFString()), but the data is not actually attached. Before accessing attachable data on an object, you must load and attach it.
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] Data is normally attached by calling the corresponding needX() method on the Query class when the object is loaded. You can also call the corresponding attachX() method explicitly. at [<phabricator>/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php:166]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] arcanist(head=master, ref.master=7d15b85a1bc0), phabricator(head=master, ref.master=929f5f22acef), phutil(head=master, ref.master=92882eb9404d)
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #0 <#2> PhabricatorLiskDAO::assertAttached(string) called at [<phabricator>/src/applications/people/storage/PhabricatorUser.php:556]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #1 <#2> PhabricatorUser::getAlternateCSRFString() called at [<phabricator>/src/applications/people/storage/PhabricatorUser.php:432]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #2 <#2> PhabricatorUser::generateToken(integer, integer, string, integer) called at [<phabricator>/src/applications/people/storage/PhabricatorUser.php:344]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #3 <#2> PhabricatorUser::getRawCSRFToken() called at [<phabricator>/src/applications/people/storage/PhabricatorUser.php:357]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #4 <#2> PhabricatorUser::getCSRFToken() called at [<phabricator>/src/infrastructure/javelin/markup.php:91]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #5 <#2> phabricator_form(PhabricatorUser, array, array) called at [<phabricator>/src/applications/slowvote/view/SlowvoteEmbedView.php:169]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #6 <#2> SlowvoteEmbedView::render() called at [<phabricator>/src/view/AphrontView.php:175]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #7 <#2> AphrontView::producePhutilSafeHTML() called at [<phutil>/src/markup/render.php:133]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #8 <#2> phutil_escape_html(SlowvoteEmbedView)
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #9 <#2> array_map(string, array) called at [<phutil>/src/markup/engine/remarkup/PhutilRemarkupBlockStorage.php:56]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #10 <#2> PhutilRemarkupBlockStorage::restore(PhutilSafeHTML, integer) called at [<phutil>/src/markup/engine/PhutilRemarkupEngine.php:299]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #11 <#2> PhutilRemarkupEngine::restoreText(PhutilSafeHTML, integer) called at [<phutil>/src/markup/engine/PhutilRemarkupEngine.php:295]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #12 <#2> PhutilRemarkupEngine::postprocessText(array) called at [<phabricator>/src/infrastructure/markup/PhabricatorMarkupEngine.php:138]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #13 <#2> PhabricatorMarkupEngine::process() called at [<phabricator>/src/applications/feed/story/PhabricatorFeedStory.php:167]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #14 <#2> PhabricatorFeedStory::loadAllFromRows(array, PhabricatorUser) called at [<phabricator>/src/applications/feed/query/PhabricatorFeedQuery.php:37]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #15 <#2> PhabricatorFeedQuery::willFilterPage(array) called at [<phabricator>/src/infrastructure/query/policy/PhabricatorPolicyAwareQuery.php:237]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #16 <#2> PhabricatorPolicyAwareQuery::execute() called at [<phabricator>/src/infrastructure/query/policy/PhabricatorPolicyAwareQuery.php:168]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #17 <#2> PhabricatorPolicyAwareQuery::executeOne() called at [<phabricator>/src/applications/feed/worker/FeedPushWorker.php:12]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #18 <#2> FeedPushWorker::loadFeedStory() called at [<phabricator>/src/applications/feed/worker/FeedPublisherWorker.php:6]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #19 <#2> FeedPublisherWorker::doWork() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorWorker.php:91]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #20 <#2> PhabricatorWorker::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:162]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #21 <#2> PhabricatorWorkerActiveTask::executeTask() called at [<phabricator>/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php:22]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #22 PhabricatorTaskmasterDaemon::run() called at [<phutil>/src/daemon/PhutilDaemon.php:185]
Daemon 43450 STDE [Mon, 08 Jun 2015 22:49:57 +0000] #23 PhutilDaemon::execute() called at [<phutil>/scripts/daemon/exec/exec_daemon.php:125]
```
Just return `null`.
Test Plan: Will check that tasks clear in production.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D13218
Summary: Right now, "Publish" workers for user profile edits (title / blub) can get gummed up in the daemons. Implement the interfaces and provide a Query so they can go through.
Test Plan:
- Made a profile "Title" edit.
- Used `bin/worker execute --id <id>` to see task fail.
- Applied patch.
- Saw task work.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D13213
Summary:
Ref T8424. This adds crude integration with Paste's edit/view workflows: you can change the space a Paste appears in, see transactions, and get a policy callout.
Lots of rough edges and non-obviousness but it pretty much works.
Test Plan:
- Created and updated Pastes.
- Moved them between spaces, saw policy effects.
- Read transactions.
- Looked at feed.
- Faked query to return no spaces, saw control and other stuff vanish.
- Faked query to return no spaces, created pastes.
- Tried to submit bad values and got errors.
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8424
Differential Revision: https://secure.phabricator.com/D13159
Test Plan: Submitted a form - saw nothing out of ordinary.
Reviewers: #blessed_reviewers
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D13167
Summary:
Fixes T8326. This removes calls to PhabricatorStartup from places that daemons may access.
This salt doesn't need to be global; it's embedded in the token we return. It's fine if we use a different salt every time. In practice, we always use the same viewer, so this change causes little or no behavioral change.
Ref T8424. For Spaces, I need a per-request cache for all spaces, because they have unusual access patterns and require repeated access, in some cases by multiple viewers.
We don't currently have a per-request in-process cache that we, e.g., clear in the daemons.
We do have a weak/theoretical/forward-looking attempt at this in `PhabricatorStartup::getGlobal()` but I'm going to throw that away (it's kind of junky, partly because of T8326) and replace it with a more formal mechanism.
Test Plan:
- Submitted some forms.
- Grepped for `csrf.salt`.
- Viewed page source, saw nice CSRF tokens with salt.
- All the salts are still the same on every page I checked, but it doesn't matter if this isn't true everywhere.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8326, T8424
Differential Revision: https://secure.phabricator.com/D13151
Summary:
Ref T8387. Adds new mailing list users.
This doesn't migrate anything yet. I also need to update the "Email Addresses" panel to let administrators change the list address.
Test Plan:
- Created and edited a mailing list user.
- Viewed profile.
- Viewed People list.
- Searched for lists / nonlists.
- Grepped for all uses of `getIsDisabled()` / `getIsSystemAgent()` and added relevant corresponding behaviors.
- Hit the web/api/ssh session blocks.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: eadler, tycho.tatitscheff, epriestley
Maniphest Tasks: T8387
Differential Revision: https://secure.phabricator.com/D13123
Summary: Fixes T8242. D12833 removed the title as well as the blurb from people hovercards. When re-adding the title don't bother throwing things through pht since that seems like not something you translate exactly and also lose the flavor text which most users end up having since title is rarely set (at least on this install).
Test Plan: viewed hovercards and saw title and blurb again as appropos relative to the data being set
Reviewers: chad, epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T8242
Differential Revision: https://secure.phabricator.com/D12915
Summary: Fix some method signatures so that arguments with default values are at the end of the argument list (see D12418).
Test Plan: Eyeballed the callsites.
Reviewers: epriestley, #blessed_reviewers, hach-que
Reviewed By: epriestley, #blessed_reviewers, hach-que
Subscribers: hach-que, Korvin, epriestley
Differential Revision: https://secure.phabricator.com/D12782
Summary: Ref T7707. Caches availability on users to reduce the cost of loading handles. This cache is very slightly tricky to dirty properly.
Test Plan:
- Use DarkConsole to examine queries; saw cache hits, miss+fill, dirty.
- Saw availability change correctly after canceling, joining, declining events.
- Saw no queries to Calendar for pages with only availability data.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7707
Differential Revision: https://secure.phabricator.com/D12838
Summary:
Ref T7707. Ref T8183.
- Currently, user status is derived by looking at events they //created//. Instead, look at non-cancelled invites they are attending.
- Prepare for on-user caching.
- Mostly remove "Sporradic" as a status, although I left room for adding more information later.
Test Plan:
- Called user.query.
- Viewed profile.
- Viewed hovercard.
- Used mentions.
- Saw status immediately update when attending/leaving/cancelling a current event.
- Created an event ending at 6 PM and an event from 6:10PM - 7PM, saw "Away until 7PM".
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T8183, T7707
Differential Revision: https://secure.phabricator.com/D12833
Summary:
Ref T7707. The general form of this can probably be refined somewhat over time as we have more use cases.
I put this cache on the user object itself because we essentially always need this data and it's trivial to invalidate the cache (we can do it implicilty during reads).
Also fix an issue with short, wide images not thumbnailing properly after recent changes.
Test Plan:
- Loaded some pages; saw caches write; saw good pictures.
- Reloaded; saw cache reads; saw good pictures.
- Changed profile picture; saw immediate update.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7707
Differential Revision: https://secure.phabricator.com/D12826
Summary:
Ref T8021.
- When "All Day" events are loaded, convert them into the viewer's time.
- When "All Day" events are saved, convert them into a +24 hour range.
Test Plan:
- Created and updated "All Day" events.
- Created and updated normal events.
- Changed timezones, edited and viewed "All Day" events and normal events.
- In all cases, "All Day" events appeared to be 12:00AM - 11:59:59PM to the viewer, on the correct day.
- Normal events shifted around properly according to timezones.
Reviewers: lpriestley
Reviewed By: lpriestley
Subscribers: epriestley
Maniphest Tasks: T8021
Differential Revision: https://secure.phabricator.com/D12765
Summary:
Ref T7689, which discusses some of the motivation here. Briefly, these methods are awkward:
- Controller->loadHandles()
- Controller->loadViewerHandles()
- Controller->renderHandlesForPHIDs()
This moves us toward better semantics, less awkwardness, and a more reasonable attack on T7688 which won't double-fetch a bunch of data.
Test Plan:
- Added unit tests.
- Converted one controller to the new stuff.
- Viewed countdown lists, saw handles render.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7689
Differential Revision: https://secure.phabricator.com/D12202
Summary:
Currently, PhortuneAccounts have a very open default policy to allow merchants to see and interact with them.
This has the undesirable side effect of leaking their names in too many places, because all users are allowed to load the handles for the accounts. Although this information is not super sensitive, we shouldn't expose it.
I went through about 5 really messy diffs trying to fix this. It's very complicated because there are a lot of objects and many of them are related to PhortuneAccounts, but PhortuneAccounts are not bound to a specific merchant. This lead to a lot of threading viewers and merchants all over the place through the call stack and some really sketchy diffs with OmnipotentUsers that weren't going anywhere good.
This is the cleanest approach I came up with, by far:
- Introduce the concept of an "Authority", which gives a user more powers as a viewer. For now, since we only have one use case, this is pretty open-ended.
- When a viewer is acting as a merchant, grant them authority through the merchant.
- Have Accounts check if the viewer is acting with merchant authority. This lets us easily implement the rule "merchants can see this stuff" without being too broad.
Then update the Subscription view to respect Merchant Authority.
I partially updated the Cart views to respect it. I'll finish this up in a separate diff, but this seemed like a good checkpoint that introduced the concept without too much extra baggage.
This feels pretty good/clean to me, overall, even ignoring the series of horrible messes I made on my way here.
Test Plan:
- Verified I can see everything I need to as a merchant (modulo un-updated Cart UIs).
- Verified I can see nothing when acting as a normal user.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D11945
Summary: Fixes T7159.
Test Plan:
Created a legalpad document that needed a signature and I was required to sign it no matter what page I hit. Signed it and things worked! Added a new legalpad document and I had to sign again!
Ran unit tests and they passed!
Logged out as a user who was roadblocked into signing a bunch of stuff and it worked!
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7159
Differential Revision: https://secure.phabricator.com/D11759
Summary:
Ref T7152. Ref T1139. This updates Phabricator so third-party libraries can translate their own stuff. Also:
- Hide "All Caps" when not in development mode, since some users have found this a little confusing.
- With other changes, adds a "Raw Strings" mode (development mode only).
- Add an example silly translation to make sure the serious business flag works.
- Add a basic British English translation.
- Simplify handling of translation overrides.
Test Plan:
- Flipped serious business / development on and off and saw silly/development translations drop off.
- Switched to "All Caps" and saw all caps.
- Switched to Very English, Wow!
- Switched to British english and saw "colour".
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7152, T1139
Differential Revision: https://secure.phabricator.com/D11747
Summary:
Ref T7152. This builds the core of email invites and implements all the hard logic for them, covering it with a pile of tests.
There's no UI to create these yet, so users can't actually get invites (and administrators can't send them).
This stuff is a complicated mess because there are so many interactions between accounts, email addresses, email verification, email primary-ness, and user verification. However, I think I got it right and got test coverage everwhere.
The degree to which this is exception-driven is a little icky, but I think it's a reasonable way to get the testability we want while still making it hard for callers to get the flow wrong. In particular, I expect there to be at least two callers (one invite flow in the upstream, and one derived invite flow in Instances) so I believe there is merit in burying as much of this logic inside the Engine as is reasonably possible.
Test Plan: Unit tests only.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T7152
Differential Revision: https://secure.phabricator.com/D11723
Summary: Ref T7094. We already had and were mostly using "needProfileImage" on the people query class. Only real trick in this diff is deleting a conduit end point that has been marked deprecated for the better part of 3 years.
Test Plan: clicked around the people action and profiles and calendars loaded nicely.
Reviewers: epriestley
Reviewed By: epriestley
Subscribers: Korvin, epriestley
Maniphest Tasks: T7094
Differential Revision: https://secure.phabricator.com/D11630
Summary:
Ref T2783. Ref T6706.
- Add `cluster.addresses`. This is a whitelist of CIDR blocks which define cluster hosts.
- When we recieve a request that has a cluster-based authentication token, require the cluster to be configured and require the remote address to be a cluster member before we accept it.
- This provides a general layer of security for these mechanisms.
- In particular, it means they do not work by default on unconfigured hosts.
- When cluster addresses are configured, and we receive a request //to// an address not on the list, reject it.
- This provides a general layer of security for getting the Ops side of cluster configuration correct.
- If cluster nodes have public IPs and are listening on them, we'll reject requests.
- Basically, this means that any requests which bypass the LB get rejected.
Test Plan:
- With addresses not configured, tried to make requests; rejected for using a cluster auth mechanism.
- With addresses configred wrong, tried to make requests; rejected for sending from (or to) an address outside of the cluster.
- With addresses configured correctly, made valid requests.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T6706, T2783
Differential Revision: https://secure.phabricator.com/D11159
Summary:
Ref T6240. Some discussion in that task. In instance/cluster environments, daemons need to make Conduit calls that bypass policy checks.
We can't just let anyone add SSH keys with this capability to the web directly, because then an adminstrator could just add a key they own and start signing requests with it, bypassing policy checks.
Add a `bin/almanac trust-key --id <x>` workflow for trusting keys. Only trusted keys can sign requests.
Test Plan:
- Generated a user key.
- Generated a device key.
- Trusted a device key.
- Untrusted a device key.
- Hit the various errors on trust/untrust.
- Tried to edit a trusted key.
{F236010}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T6240
Differential Revision: https://secure.phabricator.com/D10878
Summary:
Ref T5833. I want to add SSH keys to Almanac devices, but the edit workflows for them are currently bound tightly to users.
Instead, decouple key management from users and the settings panel.
Test Plan:
- Uploaded, generated, edited and deleted SSH keys.
- Hit missing name, missing key, bad key format, duplicate key errors.
- Edited/generated/deleted/etc keys for a bot user as an administrator.
- Got HiSec'd on everything.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5833
Differential Revision: https://secure.phabricator.com/D10824
Summary:
Ref T5833. This fixes a few weird things with this table:
- A bunch of columns were nullable for no reason.
- We stored an MD5 hash of the key (unusual) but never used it and callers were responsible for manually populating it.
- We didn't perform known-key-text lookups by using an index.
Test Plan:
- Ran migrations.
- Faked duplicate keys, saw them clean up correctly.
- Added new keys.
- Generated new keys.
- Used `bin/auth-ssh` and `bin/auth-ssh-key`.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5833
Differential Revision: https://secure.phabricator.com/D10805
Summary: Ref T1191. Same deal as D10786. These were previously case-insensitive, but changed to a case-sensitive column type.
Test Plan: Ran `bin/storage adjust` and got and adjustment.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: webframp, epriestley
Maniphest Tasks: T1191
Differential Revision: https://secure.phabricator.com/D10806
Summary: Ref T5833. Since these will no longer be bound specifically to users, bring them to a more central location.
Test Plan:
- Edited SSH keys.
- Ran `bin/ssh-auth` and `bin/ssh-auth-key`.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5833
Differential Revision: https://secure.phabricator.com/D10791
Summary: Ref T1191. After adjustment, usernames currently end up case-sensitive, which means `alincoln` and `Alincoln` are different users. Make them case-sensitive so these names collie.
Test Plan: Ran `bin/storage adjust`.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T1191
Differential Revision: https://secure.phabricator.com/D10786
Summary:
Ref T1191. Notable stuff:
- Adds `--disable-utf8mb4` to `bin/storage` to make it easier to test what things will (approximately) do on old MySQL. This isn't 100% perfect but should catch all the major stuff. It basically makes us pretend the server is an old server.
- Require utf8mb4 to dump a quickstart.
- Fix some issues with quickstart generation, notably special casing the FULLTEXT handling.
- Add an `--unsafe` flag to `bin/storage adjust` to let it truncate data to fix schemata.
- Fix some old patches which don't work if the default table charset is utf8mb4.
Test Plan:
- Dumped a quickstart.
- Loaded the quickstart with utf8mb4.
- Loaded the quickstart with `--disable-utf8mb4` (verified that we get binary columns, etc).
- Adjusted schema with `--disable-utf8mb4` (got a long adjustment with binary columns, some truncation stuff with weird edge case test data).
- Adjusted schema with `--disable-utf8mb4 --unsafe` (got truncations and clean adjust).
- Adjusted schema back without `--disable-utf8mb4` (got a long adjustment with utf8mb4 columns, some invalid data on truncated utf8).
- Adjusted schema without `--disable-utf8mb4`, but with `--unsafe` (got truncations on the invalid data).
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T1191
Differential Revision: https://secure.phabricator.com/D10757
Summary:
Ref T1191. Now that the whole database is covered, we don't need to do as much work to build expected schemata. Doing them database-by-database was helpful in converting, but is just reudndant work now.
Instead of requiring every application to build its Lisk objects, just build all Lisk objects.
I removed `harbormaster.lisk_counter` because it is unused.
It would be nice to autogenerate edge schemata, too, but that's a little trickier.
Test Plan: Database setup issues are all green.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley, hach-que
Maniphest Tasks: T1191
Differential Revision: https://secure.phabricator.com/D10620
Summary:
Ref T1191. Some notes here:
- Drops the old LDAP and OAuth info tables. These were migrated to the ExternalAccount table a very long time ago.
- Separates surplus/missing keys from other types of surplus/missing things. In the long run, my plan is to have only two notice levels:
- Error: something we can't fix (missing database, table, or column; overlong key).
- Warning: something we can fix (surplus anything, missing key, bad column type, bad key columns, bad uniqueness, bad collation or charset).
- For now, retaining three levels is helpful in generating all the expected scheamta.
Test Plan:
- Saw ~200 issues resolve, leaving ~1,300.
- Grepped for removed tables.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T1191
Differential Revision: https://secure.phabricator.com/D10580
Summary: Ref T5861. Adds an option to opt out of all notification email. We'll still send you password resets, email verifications, etc.
Test Plan:
{F189484}
- Added unit tests.
- With preference set to different things, tried to send myself mail. Mail respected preferences.
- Sent password reset email, which got through the preference.
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: rush898, epriestley
Maniphest Tasks: T5861
Differential Revision: https://secure.phabricator.com/D10237
Summary: Fixes T5510. This purely reduces false positives from HackerOne: we currently rotate CSRF tokens, but do not bind them explicitly to specific sessions. Doing so has no real security benefit and may make some session rotation changes more difficult down the line, but researchers routinely report it. Just conform to expectations since the expected behavior isn't bad and this is less work for us than dealing with false positives.
Test Plan:
- With two browsers logged in under the same user, verified I was issued different CSRF tokens.
- Verified the token from one browser did not work in the other browser's session.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5510
Differential Revision: https://secure.phabricator.com/D10136
Summary: Ref T5655. Rename `PhabricatorPHIDType` subclasses for clarity (see discussion in D9839). I'm not too keen on some of the resulting class names, so feel free to suggest alternatives.
Test Plan: Ran unit tests.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin, hach-que
Maniphest Tasks: T5655
Differential Revision: https://secure.phabricator.com/D9986
Summary: Ref T5655. The `PhabricatorDestructibleInterface` interface is misspelled as `PhabricatorDestructableInterface`. Fix the spelling mistake.
Test Plan: `grep`. Seeing as this interface is fairly recent, I don't expect that this would cause any widespread breakages.
Reviewers: epriestley, #blessed_reviewers
Reviewed By: epriestley, #blessed_reviewers
Subscribers: epriestley, Korvin
Maniphest Tasks: T5655
Differential Revision: https://secure.phabricator.com/D9988
Summary:
Fixes T5614. Ref T4420. Other than the "users" datasource and a couple of others, many datasources ignore what the user typed and just return all results, then rely on the client to filter them.
This works fine for rarely used ("legalpad documents") or always small ("task priorities", "applications") datasets, but is something we should graudally move away from as datasets get larger.
Add a token table to projects, populate it, and use it to drive the datasource query. Additionally, expose it on the applicationsearch UI.
Test Plan:
- Ran migration.
- Manually checked the table.
- Searched for projects by name from ApplicationSearch.
- Searched for projects by name from typeahead.
- Manually checked the typeahead response.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5614, T4420
Differential Revision: https://secure.phabricator.com/D9896
Summary:
Ref T4420. This was a performance hack introduced long ago to make typeaheads for users a little cheaper. The idea was that you could load some of an object's columns and skip other ones.
We now always load users on demand, so the cost of loading the whole objects is very small. No other use cases ever arose for this, and it seems unlikely that they will in the future. Remove it all.
Test Plan:
- Grepped for `CONFIG_PARTIAL_OBJECTS`.
- Grepped for `dirtyFields`.
- Grepped for `missingFields`.
- Grepped for `resetDirtyFields`.
- Grepped for `loadColumns`.
- Grepped for `loadColumnsWhere`.
- Grepped for `loadRawDataWhere`.
- Loaded and saved some lisk objects.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4420
Differential Revision: https://secure.phabricator.com/D9895
Summary:
Fixes T3732. Ref T1205. Ref T3116.
External accounts (like emails used as identities, Facebook accounts, LDAP accounts, etc.) are stored in "ExternalAccount" objects.
Currently, we have a very restrictive `CAN_VIEW` policy for ExternalAccounts, to add an extra layer of protection to make sure users can't use them in unintended ways. For example, it would be bad if a user could link their Phabricator account to a Facebook account without proper authentication. All of the controllers which do sensitive things have checks anyway, but a restrictive CAN_VIEW provided an extra layer of protection. Se T3116 for some discussion.
However, this means that when grey/external users take actions (via email, or via applications like Legalpad) other users can't load the account handles and can't see anything about the actor (they just see "Restricted External Account" or similar).
Balancing these concerns is mostly about not making a huge mess while doing it. This seems like a reasonable approach:
- Add `CAN_EDIT` on these objects.
- Make that very restricted, but open up `CAN_VIEW`.
- Require `CAN_EDIT` any time we're going to do something authentication/identity related.
This is slightly easier to get wrong (forget CAN_EDIT) than other approaches, but pretty simple, and we always have extra checks in place anyway -- this is just a safety net.
I'm not quite sure how we should identify external accounts, so for now we're just rendering "Email User" or similar -- clearly not a bug, but not identifying. We can figure out what to render in the long term elsewhere.
Test Plan:
- Viewed external accounts.
- Linked an external account.
- Refreshed an external account.
- Edited profile picture.
- Viewed sessions panel.
- Published a bunch of stuff to Asana/JIRA.
- Legalpad signature page now shows external accounts.
{F171595}
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T3732, T1205, T3116
Differential Revision: https://secure.phabricator.com/D9767
Summary: Ran `arc lint --apply-patches --everything` over rP, mainly to change double quotes to single quotes where appropriate. These changes also validate that the `ArcanistXHPASTLinter::LINT_DOUBLE_QUOTE` rule is working as expected.
Test Plan: Eyeballed it.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin, hach-que
Differential Revision: https://secure.phabricator.com/D9431
Summary: Ref T5089. Adds a `security.require-multi-factor-auth` which forces all users to enroll in MFA before they can use their accounts.
Test Plan:
Config:
{F159750}
Roadblock:
{F159748}
After configuration:
{F159749}
- Required MFA, got roadblocked, added MFA, got unblocked.
- Removed MFA, got blocked again.
- Used `bin/auth strip` to strip MFA, got blocked.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5089
Differential Revision: https://secure.phabricator.com/D9285
Summary: Both email verify and welcome links now verify email, centralize them and record them in the user activity log.
Test Plan:
- Followed a "verify email" link and got verified.
- Followed a "welcome" (verifying) link.
- Followed a "reset" (non-verifying) link.
- Looked in the activity log for the verifications.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D9284
Summary:
Fixes T5143. Currently, if your allowed domain is "example.com", we reject signups from "@Example.com".
Instead, lowercase both parts before performing the check.
Test Plan:
- Before patch:
- Set allowed domains to "yghe.net".
- Tried "x@yghe.net", no error.
- Tried "x@xxxy.net", error.
- Tried "x@yghE.net", incorrectly results in an error.
- After patch:
- Set allowed domains to "yghe.net".
- Tried "x@yghe.net", no error.
- Tried "x@xxxy.net", error.
- Tried "x@yghE.net", this correctly no longer produces an error.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T5143
Differential Revision: https://secure.phabricator.com/D9261
Summary:
Ref T4398. This code hadn't been touched in a while and had a few crufty bits.
**One Time Resets**: Currently, password reset (and similar links) are valid for about 48 hours, but we always use one token to generate them (it's bound to the account). This isn't horrible, but it could be better, and it produces a lot of false positives on HackerOne.
Instead, use TemporaryTokens to make each link one-time only and good for no more than 24 hours.
**Coupling of Email Verification and One-Time Login**: Currently, one-time login links ("password reset links") are tightly bound to an email address, and using a link verifies that email address.
This is convenient for "Welcome" emails, so the user doesn't need to go through two rounds of checking email in order to login, then very their email, then actually get access to Phabricator.
However, for other types of these links (like those generated by `bin/auth recover`) there's no need to do any email verification.
Instead, make the email verification part optional, and use it on welcome links but not other types of links.
**Message Customization**: These links can come out of several workflows: welcome, password reset, username change, or `bin/auth recover`. Add a hint to the URI so the text on the page can be customized a bit to help users through the workflow.
**Reset Emails Going to Main Account Email**: Previously, we would send password reset email to the user's primary account email. However, since we verify email coming from reset links this isn't correct and could allow a user to verify an email without actually controlling it.
Since the user needs a real account in the first place this does not seem useful on its own, but might be a component in some other attack. The user might also no longer have access to their primary account, in which case this wouldn't be wrong, but would not be very useful.
Mitigate this in two ways:
- First, send to the actual email address the user entered, not the primary account email address.
- Second, don't let these links verify emails: they're just login links. This primarily makes it more difficult for an attacker to add someone else's email to their account, send them a reset link, get them to login and implicitly verify the email by not reading very carefully, and then figure out something interesting to do (there's currently no followup attack here, but allowing this does seem undesirable).
**Password Reset Without Old Password**: After a user logs in via email, we send them to the password settings panel (if passwords are enabled) with a code that lets them set a new password without knowing the old one.
Previously, this code was static and based on the email address. Instead, issue a one-time code.
**Jump Into Hisec**: Normally, when a user who has multi-factor auth on their account logs in, we prompt them for factors but don't put them in high security. You usually don't want to go do high-security stuff immediately after login, and it would be confusing and annoying if normal logins gave you a "YOU ARE IN HIGH SECURITY" alert bubble.
However, if we're taking you to the password reset screen, we //do// want to put the user in high security, since that screen requires high security. If we don't do this, the user gets two factor prompts in a row.
To accomplish this, we set a cookie when we know we're sending the user into a high security workflow. This cookie makes login finalization upgrade all the way from "partial" to "high security", instead of stopping halfway at "normal". This is safe because the user has just passed a factor check; the only reason we don't normally do this is to reduce annoyance.
**Some UI Cleanup**: Some of this was using really old UI. Modernize it a bit.
Test Plan:
- **One Time Resets**
- Used a reset link.
- Tried to reuse a reset link, got denied.
- Verified each link is different.
- **Coupling of Email Verification and One-Time Login**
- Verified that `bin/auth`, password reset, and username change links do not have an email verifying URI component.
- Tried to tack one on, got denied.
- Used the welcome email link to login + verify.
- Tried to mutate the URI to not verify, or verify something else: got denied.
- **Message Customization**
- Viewed messages on the different workflows. They seemed OK.
- **Reset Emails Going to Main Account Email**
- Sent password reset email to non-primary email.
- Received email at specified address.
- Verified it does not verify the address.
- **Password Reset Without Old Password**
- Reset password without knowledge of old one after email reset.
- Tried to do that without a key, got denied.
- Tried to reuse a key, got denied.
- **Jump Into Hisec**
- Logged in with MFA user, got factor'd, jumped directly into hisec.
- Logged in with non-MFA user, no factors, normal password reset.
- **Some UI Cleanup**
- Viewed new UI.
- **Misc**
- Created accounts, logged in with welcome link, got verified.
- Changed a username, used link to log back in.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4398
Differential Revision: https://secure.phabricator.com/D9252
Summary: Fixes T4728, first pass, Make real name optional on user accounts
Test Plan: Default real name config should be false (not required). Create new user, real name should not be required. Toggle config, real name should be required. Users with no real name should be always listed by their usernames.
Reviewers: #blessed_reviewers, epriestley
Reviewed By: #blessed_reviewers, epriestley
Subscribers: epriestley, Korvin
Maniphest Tasks: T4728
Differential Revision: https://secure.phabricator.com/D9027
Summary:
Ref T4749. Ref T3265. Ref T4909. Several goals here:
- Move user destruction to the CLI to limit the power of rogue admins.
- Start consolidating all "destroy named object" scripts into a single UI, to make it easier to know how to destroy things.
- Structure object destruction so we can do a better and more automatic job of cleaning up transactions, edges, search indexes, etc.
- Log when we destroy objects so there's a record if data goes missing.
Test Plan: Used `bin/remove destroy` to destroy several users.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T3265, T4749, T4909
Differential Revision: https://secure.phabricator.com/D8940
Summary:
Ref T4398. This prompts users for multi-factor auth on login.
Roughly, this introduces the idea of "partial" sessions, which we haven't finished constructing yet. In practice, this means the session has made it through primary auth but not through multi-factor auth. Add a workflow for bringing a partial session up to a full one.
Test Plan:
- Used Conduit.
- Logged in as multi-factor user.
- Logged in as no-factor user.
- Tried to do non-login-things with a partial session.
- Reviewed account activity logs.
{F149295}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4398
Differential Revision: https://secure.phabricator.com/D8922
Summary:
Ref T4398. Allows auth factors to render and validate when prompted to take a hi-sec action.
This has a whole lot of rough edges still (see D8875) but does fundamentally work correctly.
Test Plan:
- Added two different TOTP factors to my account for EXTRA SECURITY.
- Took hisec actions with no auth factors, and with attached auth factors.
- Hit all the error/failure states of the hisec entry process.
- Verified hisec failures appear in activity logs.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4398
Differential Revision: https://secure.phabricator.com/D8886
Summary:
Ref T4398. This is still pretty rough and isn't exposed in the UI yet, but basically works. Some missing features / areas for improvement:
- Rate limiting attempts (see TODO).
- Marking tokens used after they're used once (see TODO), maybe. I can't think of ways an attacker could capture a token without also capturing a session, offhand.
- Actually turning this on (see TODO).
- This workflow is pretty wordy. It would be nice to calm it down a bit.
- But also add more help/context to help users figure out what's going on here, I think it's not very obvious if you don't already know what "TOTP" is.
- Add admin tool to strip auth factors off an account ("Help, I lost my phone and can't log in!").
- Add admin tool to show users who don't have multi-factor auth? (so you can pester them)
- Generate QR codes to make the transfer process easier (they're fairly complicated).
- Make the "entering hi-sec" workflow actually check for auth factors and use them correctly.
- Turn this on so users can use it.
- Adding SMS as an option would be nice eventually.
- Adding "password" as an option, maybe? TOTP feels fairly good to me.
I'll post a couple of screens...
Test Plan:
- Added TOTP token with Google Authenticator.
- Added TOTP token with Authy.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4398
Differential Revision: https://secure.phabricator.com/D8875
Summary:
Ref T4398. This adds a settings panel for account activity so users can review activity on their own account. Some goals are:
- Make it easier for us to develop and support auth and credential information, see T4398. This is the primary driver.
- Make it easier for users to understand and review auth and credential information (see T4842 for an example -- this isn't there yet, but builds toward it).
- Improve user confidence in security by making logging more apparent and accessible.
Minor corresponding changes:
- Entering and exiting hisec mode is now logged.
- This, sessions, and OAuth authorizations have moved to a new "Sessions and Logs" area, since "Authentication" was getting huge.
Test Plan:
- Viewed new panel.
- Viewed old UI.
- Entered/exited hisec and got prompted.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4398
Differential Revision: https://secure.phabricator.com/D8871
Summary:
Ref T4398. Ref T4842. I want to let users review their own account activity, partly as a general security measure and partly to make some of the multi-factor stuff easier to build and debug.
To support this, implement modern policies and application search.
I also removed the "old" and "new" columns from this output, since they had limited utility and revealed email addresses to administrators for some actions. We don't let administrators access email addresses from other UIs, and the value of doing so here seems very small.
Test Plan: Used interface to issue a bunch of queries against user logs, got reasonable/expected results.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: keir, epriestley
Maniphest Tasks: T4842, T4398
Differential Revision: https://secure.phabricator.com/D8856
Summary:
Ref T4398. This is roughly a "sudo" mode, like GitHub has for accessing SSH keys, or Facebook has for managing credit cards. GitHub actually calls theirs "sudo" mode, but I think that's too technical for big parts of our audience. I've gone with "high security mode".
This doesn't actually get exposed in the UI yet (and we don't have any meaningful auth factors to prompt the user for) but the workflow works overall. I'll go through it in a comment, since I need to arrange some screenshots.
Test Plan: See guided walkthrough.
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4398
Differential Revision: https://secure.phabricator.com/D8851
Summary:
When we generate account tokens for CSRF keys and email verification, one of the inputs we use is the user's password hash. Users won't always have a password hash, so this is a weak input to key generation. This also couples CSRF weirdly with auth concerns.
Instead, give users a dedicated secret for use in token generation which is used only for this purpose.
Test Plan:
- Ran upgrade scripts.
- Verified all users got new secrets.
- Created a new user.
- Verified they got a secret.
- Submitted CSRF'd forms, they worked.
- Adjusted the CSRF token and submitted CSRF'd forms, verified they don't work.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D8748
Summary:
Ref T4371. Ref T4699. Fixes T3994.
Currently, we're very conservative about sending errors back to users. A concern I had about this was that mistakes could lead to email loops, massive amounts of email spam, etc. Because of this, I was pretty hesitant about replying to email with more email when I wrote this stuff.
However, this was a long time ago. We now have Message-ID deduplication, "X-Phabricator-Sent-This-Mail", generally better mail infrastructure, and rate limiting. Together, these mechanisms should reasonably prevent anything crazy (primarily, infinite email loops) from happening.
Thus:
- When we hit any processing error after receiving a mail, try to send the author a reply with details about what went wrong. These are limited to 6 per hour per address.
- Rewrite most of the errors to be more detailed and informative.
- Rewrite most of the errors in a user-facing voice ("You sent this mail..." instead of "This mail was sent..").
- Remove the redundant, less sophisticated code which does something similar in Differential.
Test Plan:
- Using `scripts/mail/mail_receiver.php`, artificially received a pile of mail.
- Hit a bunch of different errors.
- Saw reasonable error mail get sent to me.
- Saw other reasonable error mail get rate limited.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T3994, T4371, T4699
Differential Revision: https://secure.phabricator.com/D8692
Summary:
Ref T4065. Currently, we have this super copy/pasted "edit profile picture" UI for system agents.
Instead, give administrators direct access from profiles, so they can use the same code pages do.
Test Plan: Edited my profile picture and profile details. Edited an agent's. Was unable to edit a non-agent user.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Maniphest Tasks: T4065
Differential Revision: https://secure.phabricator.com/D8664
Summary: Fixes T4665. The "attachable" logic was a little off after a recent change.
Test Plan: With and without a profile image, viewed a page.
Reviewers: btrahan, chad
Reviewed By: chad
Subscribers: epriestley
Maniphest Tasks: T4665
Differential Revision: https://secure.phabricator.com/D8594
Summary: Ref T4400. Same deal as projects. Tweaked the CSS a touch to make it look better in these views.
Test Plan: Viewed /people/.
Reviewers: chad, btrahan
Reviewed By: btrahan
Subscribers: epriestley, chad
Maniphest Tasks: T4400
Differential Revision: https://secure.phabricator.com/D8571
Summary:
This is the other half of D8548. Specifically, the attack here was to set your own editor link to `javascript\n:...` and then you could XSS yourself. This isn't a hugely damaging attack, but we can be more certain by adding a whitelist here.
We already whitelist linkable protocols in remarkup (`uri.allowed-protocols`) in general.
Test Plan:
Tried to set and use valid/invalid editor URIs.
{F130883}
{F130884}
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: epriestley
Differential Revision: https://secure.phabricator.com/D8551
Summary:
Via HackerOne. In regular expressions, "$" matches "end of input, or before terminating newline". This means that the expression `/^A$/` matches two strings: `"A"`, and `"A\n"`.
When we care about this, use `\z` instead, which matches "end of input" only.
This allowed registration of `"username\n"` and similar.
Test Plan:
- Grepped codebase for all calls to `preg_match()` / `preg_match_all()`.
- Fixed the ones where this seemed like it could have an impact.
- Added and executed unit tests.
Reviewers: btrahan
Reviewed By: btrahan
Subscribers: aran, epriestley
Differential Revision: https://secure.phabricator.com/D8516
Summary: Put a very rough filter on what we'll accept as an email address. We can expand this if anyone is actually using local delivery or other weird things. This is mostly to avoid a theoretical case where some input is parsed differently by `PhutilAddressParser` and the actual mail adapter, in some subtle hypothetical way. This should give us only "reasonable" email addresses which parsers would be hard-pressed to trip up on.
Test Plan: Added and executed unit tests. Tried to add silly emails. Added valid emails.
Reviewers: btrahan, arice
Reviewed By: arice
CC: arice, chad, aran
Differential Revision: https://secure.phabricator.com/D8320
Summary:
Via HackerOne. An attacker can bypass `auth.email-domains` by registering with an email like:
aaaaa...aaaaa@evil.com@company.com
We'll validate the full string, then insert it into the database where it will be truncated, removing the `@company.com` part. Then we'll send an email to `@evil.com`.
Instead, reject email addresses which won't fit in the table.
`STRICT_ALL_TABLES` stops this attack, I'm going to add a setup warning encouraging it.
Test Plan:
- Set `auth.email-domains` to `@company.com`.
- Registered with `aaa...aaa@evil.com@company.com`. Previously this worked, now it is rejected.
- Did a valid registration.
- Tried to add `aaa...aaaa@evil.com@company.com` as an email address. Previously this worked, now it is rejected.
- Did a valid email add.
- Added and executed unit tests.
Reviewers: btrahan, arice
Reviewed By: arice
CC: aran, chad
Differential Revision: https://secure.phabricator.com/D8308
Summary:
Ref T4443. Make hashing algorithms pluggable and extensible so we can deal with the attendant complexities more easily.
This moves "Iterated MD5" to a modular implementation, and adds a tiny bit of hack-glue so we don't need to migrate the DB in this patch. I'll migrate in the next patch, then add bcrypt.
Test Plan:
- Verified that the same stuff gets stored in the DB (i.e., no functional changes):
- Logged into an old password account.
- Changed password.
- Registered a new account.
- Changed password.
- Switched back to master.
- Logged in / out, changed password.
- Switched back, logged in.
- Ran unit tests (they aren't super extensive, but cover some of the basics).
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, kofalt
Maniphest Tasks: T4443
Differential Revision: https://secure.phabricator.com/D8268
Summary: Ref T4375. We never join this table, so this is a pretty straight find/replace.
Test Plan: Browsed around Calendar, verified that nothing seemed broken. Saw my red dot in other apps.
Reviewers: btrahan, chad
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4375
Differential Revision: https://secure.phabricator.com/D8145
Summary: Fixes T4339. If you're anonymous, we use a digest of your session key to generate a CSRF token. Otherwise, everything works normally.
Test Plan: Logged out, logged in, tweaked CSRF in forms -- I'll add some inlines.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4339
Differential Revision: https://secure.phabricator.com/D8046
Summary:
Ref T4339. Ref T4310. Currently, sessions look like `"afad85d675fda87a4fadd54"`, and are only issued for logged-in users. To support logged-out CSRF and (eventually) external user sessions, I made two small changes:
- First, sessions now have a "kind", which is indicated by a prefix, like `"A/ab987asdcas7dca"`. This mostly allows us to issue session queries more efficiently: we don't have to issue a query at all for anonymous sessions, and can join the correct table for user and external sessions and save a query. Generally, this gives us more debugging information and more opportunity to recover from issues in a user-friendly way, as with the "invalid session" error in this diff.
- Secondly, if you load a page and don't have a session, we give you an anonymous session. This is just a secret with no special significance.
This does not implement CSRF yet, but gives us a client secret we can use to implement it.
Test Plan:
- Logged in.
- Logged out.
- Browsed around.
- Logged in again.
- Went through link/register.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4310, T4339
Differential Revision: https://secure.phabricator.com/D8043
Summary: Ref T4339. We have more magical cookie names than we should, move them all to a central location.
Test Plan: Registered, logged in, linked account, logged out. See inlines.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4339
Differential Revision: https://secure.phabricator.com/D8041
Summary: Fixes T3857. Earlier work made this trivial and just left product questions, which I've answered by requiring the daemons to run on reasonable installs.
Test Plan: Ran `bin/search index` and `bin/search index --background`. Observed indexes write in the former case and tasks queue in the latter case. Commented with a unique string on a revision and searched for it a moment later, got exactly one result (that revision), verifying that reindexing works correctly.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3857
Differential Revision: https://secure.phabricator.com/D7966
Summary: Ref T4310. Ref T3720. Session operations are currently part of PhabricatorUser. This is more tightly coupled than needbe, and makes it difficult to establish login sessions for non-users. Move all the session management code to a `SessionEngine`.
Test Plan:
- Viewed sessions.
- Regenerated Conduit certificate.
- Verified Conduit sessions were destroyed.
- Logged out.
- Logged in.
- Ran conduit commands.
- Viewed sessions again.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4310, T3720
Differential Revision: https://secure.phabricator.com/D7962
Summary:
Ref T4310. This is a small step toward separating out the session code so we can establish sessions for `ExternalAccount` and not just `User`.
Also fix an issue with strict MySQL and un-admin / un-disable from web UI.
Test Plan: Logged in, logged out, admined/de-admin'd user, added email address, checked user log for all those events.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4310
Differential Revision: https://secure.phabricator.com/D7953
Summary:
Fixes T4132. If you run "bin/auth recover" before setting the base URI, it throws when trying to generate a production URI.
Instead, just show the path. We can't figure out the domain, and I think this is less confusing than showing "your.phabricator.example.com", etc.
Test Plan: Ran `bin/auth recover <user>` for valid and missing base-uri.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T4132
Differential Revision: https://secure.phabricator.com/D7615
Summary:
- Add an option for the queue.
- By default, enable it.
- Dump new users into the queue.
- Send admins an email to approve them.
Test Plan:
- Registered new accounts with queue on and off.
- As an admin, approved accounts and disabled the queue from email.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7576
Summary:
Nothing fancy here, just:
- UI to show users needing approval.
- "Approve" and "Disable" actions.
- Send "Approved" email on approve.
- "Approve" edit + log operations.
- "Wait for Approval" state for users who need approval.
There's still no natural way for users to end up not-approved -- you have to write directly to the database.
Test Plan: See screenshots.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D7573
Summary:
Small step forward which improves existing stuff or lays groudwork for future stuff:
- Currently, to check for email verification, we have to single-query the email address on every page. Instead, denoramlize it into the user object.
- Migrate all the existing users.
- When the user verifies an email, mark them as `isEmailVerified` if the email is their primary email.
- Just make the checks look at the `isEmailVerified` field.
- Add a new check, `isUserActivated()`, to cover email-verified plus disabled. Currently, a non-verified-but-not-disabled user could theoretically use Conduit over SSH, if anyone deployed it. Tighten that up.
- Add an `isApproved` flag, which is always true for now. In a future diff, I want to add a default-on admin approval queue for new accounts, to prevent configuration mistakes. The way it will work is:
- When the queue is enabled, registering users are created with `isApproved = false`.
- Admins are sent an email, "[Phabricator] New User Approval (alincoln)", telling them that a new user is waiting for approval.
- They go to the web UI and approve the user.
- Manually-created accounts are auto-approved.
- The email will have instructions for disabling the queue.
I think this queue will be helpful for new installs and give them peace of mind, and when you go to disable it we have a better opportunity to warn you about exactly what that means.
Generally, I want to improve the default safety of registration, since if you just blindly coast through the path of least resistance right now your install ends up pretty open, and realistically few installs are on VPNs.
Test Plan:
- Ran migration, verified `isEmailVerified` populated correctly.
- Created a new user, checked DB for verified (not verified).
- Verified, checked DB (now verified).
- Used Conduit, People, Diffusion.
Reviewers: btrahan
Reviewed By: btrahan
CC: chad, aran
Differential Revision: https://secure.phabricator.com/D7572
Summary:
Ref T603. This cleans up an existing callsite in the policy filter, and opens up some stuff in the future.
Some policy objects don't have real PHIDs:
PhabricatorTokenGiven
PhabricatorSavedQuery
PhabricatorNamedQuery
PhrequentUserTime
PhabricatorFlag
PhabricatorDaemonLog
PhabricatorConduitMethodCallLog
ConduitAPIMethod
PhabricatorChatLogEvent
PhabricatorChatLogChannel
Although it would be reasonable to add real PHIDs to some of these (like `ChatLogChannel`), it probably doesn't make much sense for others (`DaemonLog`, `MethodCallLog`). Just let them return `null`.
Also remove some duplicate `$id` and `$phid` properties. These are declared on `PhabricatorLiskDAO` and do not need to be redeclared.
Test Plan: Ran the `testEverythingImplemented` unit test, which verifies that all classes conform to the interface.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D7306
Summary: Ref T603. Swaps out most `PhabricatorFile` loads for `PhabricatorFileQuery`.
Test Plan:
- Viewed Differential changesets.
- Used `file.info`.
- Used `file.download`.
- Viewed a file.
- Deleted a file.
- Used `/Fnnnn` to access a file.
- Uploaded an image, verified a thumbnail generated.
- Created and edited a macro.
- Added a meme.
- Did old-school attach-a-file-to-a-task.
- Viewed a paste.
- Viewed a mock.
- Embedded a mock.
- Profiled a page.
- Parsed a commit with image files linked to a revision with image files.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D7178
Summary:
Ref T603. Adds clarifying text which expands on policies and explains exceptions and rules. The goal is to provide an easy way for users to learn about special policy rules, like "task owners can always see a task".
This presentation might be a little aggressive. That's probably OK as we introduce policies, but something a little more tempered might be better down the road.
Test Plan: See screenshot.
Reviewers: btrahan, chad
Reviewed By: chad
CC: aran
Maniphest Tasks: T603
Differential Revision: https://secure.phabricator.com/D7150
Summary:
Ref T2625. Ref T3794. Ref T418. Ref T1703.
This is a more general version of D5278. It expands CustomField support to include real integration with ApplicationSearch.
Broadly, custom fields may elect to:
- build indicies when objects are updated;
- populate ApplicationSearch forms with new controls;
- read inputs entered into those controls out of the request; and
- apply constraints to search queries.
Some utility/helper stuff is provided to make this easier. This part could be cleaner, but seems reasonable for a first cut. In particular, the Query and SearchEngine must manually call all the hooks right now instead of everything happening magically. I think that's fine for the moment; they're pretty easy to get right.
Test Plan:
I added a new searchable "Company" field to People:
{F58229}
This also cleaned up the disable/reorder view a little bit:
{F58230}
As it did before, this field appears on the edit screen:
{F58231}
However, because it has `search`, it also appears on the search screen:
{F58232}
When queried, it returns the expected results:
{F58233}
And the actually good bit of all this is that the query can take advantage of indexes:
mysql> explain SELECT * FROM `user` user JOIN `user_customfieldstringindex` `appsearch_0` ON `appsearch_0`.objectPHID = user.phid AND `appsearch_0`.indexKey = 'mk3Ndy476ge6' AND `appsearch_0`.indexValue IN ('phacility') ORDER BY user.id DESC LIMIT 101;
+----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
| 1 | SIMPLE | appsearch_0 | ref | key_join,key_find | key_find | 232 | const,const | 1 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | user | eq_ref | phid | phid | 194 | phabricator2_user.appsearch_0.objectPHID | 1 | |
+----+-------------+-------------+--------+-------------------+----------+---------+------------------------------------------+------+----------------------------------------------+
2 rows in set (0.00 sec)
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T418, T1703, T2625, T3794
Differential Revision: https://secure.phabricator.com/D6992
Summary:
Ref T3599
Go through everything, grep a bit, replace some bits.
Test Plan: Navigate around a bit
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Maniphest Tasks: T3599
Differential Revision: https://secure.phabricator.com/D6871
Summary: Also, don't try to load prefs for non-users.
Test Plan: toggle, save, look at something with a time. arc unit.
Reviewers: epriestley
Reviewed By: epriestley
CC: Korvin, aran
Differential Revision: https://secure.phabricator.com/D6796
Summary:
We can get this out of PHIDType reasonably in all cases and simplify implementation here.
None of these translate correctly anyway so they're basically debugging/development strings.
Test Plan: `grep`, browsed some transactions
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D6786
Summary:
Ref T1703. Ref T3718. This introduces `PhabricatorCustomFieldAttachment`, which is just a fancy `array()`. The goal here is to simplify `PhabricatorCustomFieldInterface` as much as possible.
In particular, it can now use common infrastructure (`assertAttached()`) and is more difficult to get wrong.
Test Plan: Edited custom fields on profile.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1703, T3718
Differential Revision: https://secure.phabricator.com/D6752
Summary:
Ref T1703. Ref T3718. The `PhabricatorCustomFieldList` seems like a pretty good idea. Move more code into it to make it harder to get wrong.
Also the sequencing on old/new values for these transactions was a bit off; fix that up.
Test Plan: Edited standard and custom profile fields.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1703, T3718
Differential Revision: https://secure.phabricator.com/D6751
Summary:
Ref T1702. Ref T3718. There are a couple of things going on here:
**PhabricatorCustomFieldList**: I added `PhabricatorCustomFieldList`, which is just a convenience class for dealing with lists of fields. Often, current field code does something like this inline in a Controller:
foreach ($fields as $field) {
// do some junk
}
Often, that junk has some slightly subtle implications. Move all of it to `$list->doSomeJunk()` methods (like `appendFieldsToForm()`, `loadFieldsFromStorage()`) to reduce code duplication and prevent errors. This additionally moves an existing list-convenience method there, out of `PhabricatorPropertyListView`.
**PhabricatorUserConfiguredCustomFieldStorage**: Adds `PhabricatorUserConfiguredCustomFieldStorage` for storing custom field data (like "ICQ Handle", "Phone Number", "Desk", "Favorite Flower", etc).
**Configuration-Driven Custom Fields**: Previously, I was thinking about doing these with interfaces, but as I thought about it more I started to dislike that approach. Instead, I built proxies into `PhabricatorCustomField`. Basically, this means that fields (like a custom, configuration-driven "Favorite Flower" field) can just use some other Field to actually provide their implementation (like a "standard" field which knows how to render text areas). The previous approach would have involed subclasssing the "standard" field and implementing an interface, but that would mean that every application would have at least two "base" fields and generally just seemed bleh as I worked through it.
The cost of this approach is that we need a bunch of `proxy` junk in the base class, but that's a one-time cost and I think it simplifies all the implementations and makes them a lot less magical (e.g., all of the custom fields now extend the right base field classes).
**Fixed Some Bugs**: Some of this code hadn't really been run yet and had minor bugs.
Test Plan:
{F54240}
{F54241}
{F54242}
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T1702, T1703, T3718
Differential Revision: https://secure.phabricator.com/D6749
Summary: Fixes T2348. We should probably do some of this more broadly, but can tackle them one at a time as they arise, since many fields have no effective length limit.
Test Plan: {F54126}
Reviewers: btrahan, asherkin
Reviewed By: asherkin
CC: aran
Maniphest Tasks: T2348
Differential Revision: https://secure.phabricator.com/D6744
Summary: Ref T3684 for discussion. This could be cleaned up a bit (it would be nice to draw entropy once per request, for instance, and maybe respect CSRF_TOKEN_LENGTH more closely) but should effectively mitigate BREACH.
Test Plan: Submitted forms; submitted forms after mucking with CSRF and observed CSRF error. Verified that source now has "B@..." tokens.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T3684
Differential Revision: https://secure.phabricator.com/D6686
Summary: Ref T2715. Had to start loading status information in the query class. Debated trying to clean up some of the attach / load stuff but decided to just add status under the new paradigm for now.
Test Plan: phid.query also made a status and checked that out. also played in conpherence.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, Korvin
Maniphest Tasks: T2715
Differential Revision: https://secure.phabricator.com/D6585
