Summary:
We have a lot of cases where we store object relationships, but it's all kind of messy and custom. Some particular problems:
- We go to great lengths to enforce order stability in Differential revisions, but the implementation is complex and inelegant.
- Some relationships are stored on-object, so we can't pull the inverses easily. For example, Maniphest shows child tasks but not parent tasks.
- I want to add more of these and don't want to continue building custom stuff.
- UIs like the "attach stuff to other stuff" UI need custom branches for each object type.
- Stuff like "allow commits to close tasks" is notrivial because of nonstandard metadata storage.
Provide an association-like "edge" framework to fix these problems. This is nearly identical to associations, with a few differences:
- I put edge metadata in a separate table and don't load it by default, to keep edge rows small and allow large metadata if necessary. The on-edge metadata seemed to get abused a lot at Facebook.
- I put a 'seq' column on the edges to ensure they have an explicit, stable ordering within a source and type.
This isn't actually used anywhere yet, but my first target is attaching commits to tasks for T904.
Test Plan: Made a mock page that used Editor and Query. Verified adding and removing edges, overwriting edges, writing and loading edge data, sequence number generation.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, 20after4
Differential Revision: https://secure.phabricator.com/D2088
Summary: NOTE: This is not produced by a script so there might be errors. Please review carefully.
Test Plan: Browse around Differential.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran
Differential Revision: https://secure.phabricator.com/D2103
Summary:
Update the Herald documentation with information about global vs personal rules.
NOTE: Some of the features mentioned here don't exist yet (flags, delete rules), I'll add them before I land this.
Test Plan: Read documentation
Reviewers: jungejason, btrahan
Reviewed By: btrahan
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D2031
Summary: This header allows recipients to distinguish between CCs generated by Herald and CCs generated by humans.
Test Plan: Created a Herald rule to add a bunch of CC's to every revision. Created a revision. Added some CCs manually. Verified that only manual CCs appeared in the "Explicit" header.
Reviewers: btrahan, vrana
Reviewed By: btrahan
CC: aran, epriestley
Maniphest Tasks: T808
Differential Revision: https://secure.phabricator.com/D2018
Summary:
I noticed that documentation said it is possible to have 'ctags' symbol import, so I hacked a quick version. I tested it on Python based project and successfuly imported symbols.
It is limited to classes right now, as the importer script complained about not-unique method names (there are a lot of 'get' & 'post' methods accross classes in my project).
If you would have any feedback about this, I would definetly try to wrap it up for possibly merging into main repository.
Test Plan:
Required 'ctags' tool (ctags.sourceforge.net/) Tested to work with version 5.8+ and didn't work with 3.x.
1. `find . -type f '*.py' | ./generate_ctags_symbols.php > /tmp/symbols`
2. `./import_project_symbols.php` < /tmp/symbols
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: seporaitis, aran, epriestley
Maniphest Tasks: T1034
Differential Revision: https://secure.phabricator.com/D1995
Summary:
We render a huge picture of a PDF for PDFs right now, etc. This is hella dumb.
Also allow users to force this rendering style, and change the link name.
Test Plan: Uploaded image and non-image files, used layout=link and name=....
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, epriestley
Maniphest Tasks: T1040
Differential Revision: https://secure.phabricator.com/D2006
Summary:
See <https://github.com/facebook/arcanist/issues/21>, <https://github.com/facebook/arcanist/issues/22>.
- Provide more detailed install instructions.
- Provide advice on managing team installs.
- Provide information on libphutil path resolution.
- Provide OSX-specific documentation.
- Update some of the documentation to reflect evolution of the tools.
Test Plan: Generated and read documentation.
Reviewers: btrahan
Reviewed By: btrahan
CC: jmhsieh, aran, epriestley
Differential Revision: https://secure.phabricator.com/D1990
Summary: Try to explain how this stuff works a little better. Let me know what's unclear / missing / not good.
Test Plan: Generated documentation, read documentation.
Reviewers: btrahan, gschmidt
Reviewed By: btrahan
CC: aran, epriestley, davidreuss
Maniphest Tasks: T904
Differential Revision: https://secure.phabricator.com/D1969
Summary: Add @20after4's new rule to the Phabricator engine.
Test Plan: Wrote some ~~deleted~~ text.
Reviewers: 20after4, btrahan
Reviewed By: btrahan
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1972
Summary: @gschmidt noticed this in IRC.
Test Plan: Verified correct URI is "/file/", eyeballed it.
Reviewers: btrahan, gschmidt
Reviewed By: gschmidt
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1963
Summary: Split from D1921 via D1742. The "B" flag prevents excessive unescaping, especially of "+" into " ".
Test Plan: Added "B" to server config, var_dump()'d __path__ with "+" in it, got "+" instead of " ".
Reviewers: nh, vrana, btrahan
Reviewed By: nh
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1922
Summary:
The link to libphutil Libraries User Guide was broken on Arcanist User
Guide: Customizing Lint, Unit Tests and Workflows.
Test Plan: ran diviner
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1890
Summary:
1. The setup flow complains if you haven't updated your schema, so that section
should be moved above the setup flow.
2. The setup flow tells you to lower your timeout, but it doesn't tell you how
low will make it stop complaining.
Test Plan:
Didn't test the setup.
Regenerated the docs and saw the change.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1888
Summary:
This will allow sending mail to be done by task workers. See T750.
Task ID: #
Blame Rev:
Test Plan:
- started taskmaster daemon in test env
- used "send new test message" feature in MetMTA (with send now unchecked)
- confirmed receipt of 1 email
- repeated 2 & 3 with send now checked
Revert Plan:
Tags:
Reviewers: epriestley, jungejason
Reviewed By: epriestley
CC: aran, epriestley
Maniphest Tasks: T388, T750
Differential Revision: https://secure.phabricator.com/D1723
Summary: add a big ole HELP tab and make "scope" link to the specific
sub-section about scope
Test Plan:
read my doc a few times, it basically english
verified links looked correct and should work right once this is all in
production
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, epriestley
Maniphest Tasks: T910
Differential Revision: https://secure.phabricator.com/D1752
Summary:
- We have a lot of headers now; document them.
- Remove the one random protip from like 3 years ago from all Differential
mail.
Test Plan: generated; read documentation
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, epriestley
Maniphest Tasks: T931
Differential Revision: https://secure.phabricator.com/D1748
Summary:
This is so freaking cool that I will try to implement it also on Facebook.
Idea is from
http://strd6.com/2011/09/html5-javascript-pasting-image-data-in-chrome/.
I don't know how to properly detect support but lying about it is not a big
deal.
Test Plan:
Go to revision comment textarea.
Paste some text data - works as usual.
Paste some image data in Chrome - file is uploaded and a link to it is inserted.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1681
Summary: Document "--" list sytle and improve explicitness of list documentation
in general.
Test Plan: Generated, read documentation.
Reviewers: btrahan
Reviewed By: btrahan
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1661
create new tasks
Summary: see title
Test Plan: Tested jump nav and found the correct urls were being loaded. Old
functionality was not effected.
Reviewers: epriestley, btrahan
Reviewed By: epriestley
CC: ddfisher, allenjohnashton, kpark517, aran, epriestley
Differential Revision: https://secure.phabricator.com/D1642
Summary: Update Phabricator for Remarkup changes in D1638.
Test Plan: Looked at various sorts of nested, enumerated, and phantom-item
lists.
Reviewers: btrahan, jungejason
Reviewed By: jungejason
CC: aran, epriestley
Maniphest Tasks: T885
Differential Revision: https://secure.phabricator.com/D1639
Summary:
Some user feedback:
- Named link information not present in quick reference.
- Named link information buried in Phriction docs.
- Optional omission of trailing "=" in headers not documented.
Fix these things.
Test Plan: generated, read documentation
Reviewers: btrahan, paularmstrong, Josereyes
Reviewed By: btrahan
CC: aran, epriestley
Maniphest Tasks: T884
Differential Revision: https://secure.phabricator.com/D1637
Summary:
This is pretty simple and unpolished, but it's getting pretty big and it seems
like a reasonable starting point.
- Log chat in various "channels".
- Conduit record and query methods.
- IRCBot integration for IRC logging
Major TODO:
- Web UI is really unpolished and has no search, paging, anchor-linking, etc.
Basically all presentation stuff, though.
- I think the bot should have a map of channels to log with channel aliases?
- The "channels" should probably be in a separate table.
- The "authors" should probably be correlated to Phabricator accounts somehow,
where possible.
Test Plan: Used phabotlocal to log #phabricator.
Reviewers: kdeggelman, btrahan, Koolvin
Reviewed By: btrahan
CC: aran, epriestley
Maniphest Tasks: T837
Differential Revision: https://secure.phabricator.com/D1625
Summary:
- Restore quick methods for getting to common features (upload file, create
task, etc.)
- Provide a flexible cli-like navigation element similar to stuff used at
Facebook (bunny1 / lolbunny).
Test Plan: Used jump nav and nav buttons.
Reviewers: btrahan, fratrik
Reviewed By: btrahan
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1619
Summary:
As title.
Please help me to improve the wording!
Test Plan:
generate the documentation from the diviner file; read it; spell
check
Reviewers: epriestley, nh
Reviewed By: nh
CC: aran, dihde14, mpodobnik, prithvi, TomL, epriestley
Differential Revision: https://secure.phabricator.com/D1395
Summary: Provide explicit guidance in the documentation about liberal use of
"final".
Test Plan: Generated, read documentation.
Reviewers: btrahan, jungejason, aran, nh
Reviewed By: btrahan
CC: aran, epriestley
Maniphest Tasks: T795
Differential Revision: https://secure.phabricator.com/D1520
Summary:
Render coverage information in the right gutter, if available.
We could render some kind of summary report deal too but this seems like a good
start.
Test Plan:
- Looked at diffs with coverage.
- Looked at diffs without coverage.
- Used inline comments, diff-of-diff, "show more", "show entire file", "show
generated file", "undo". Nothing seemed disrupted by the addition of a 5th
column.
Reviewers: btrahan, tuomaspelkonen, jungejason
Reviewed By: btrahan
CC: zeeg, aran, epriestley
Maniphest Tasks: T140
Differential Revision: https://secure.phabricator.com/D1527
Summary: Provide some documentation for this feature since it's not super
obvious how it works.
Test Plan: Generated documentation, read documentation.
Reviewers: btrahan, vrana, jungejason, nh
Reviewed By: vrana
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1521
Summary: As per discussion with @johnduhart, improve documentation around
reusing and customizing linters.
Test Plan: Generated and read documentation.
Reviewers: btrahan, jungejason, johnduhart
Reviewed By: btrahan
CC: aran, epriestley
Maniphest Tasks: T795
Differential Revision: https://secure.phabricator.com/D1501
Summary:
Rough cut of Drydock. This is very basic and doesn't do much of use yet (it
//does// allocate EC2 machines as host resources and expose interfaces to them),
but I think the overall structure is more or less reasonable.
== Interfaces
Vision: Applications interact with Drydock resources through DrydockInterfaces,
like **command**, **filesystem** and **httpd** interfaces. Each interface allows
applications to perform some kind of operation on the resource, like executing
commands, reading/writing files, or configuring a web server. Interfaces have a
concrete, specific API:
// Filesystem Interface
$fs = $lease->getInterface('filesystem'); // Constants, some day?
$fs->writeFile('index.html', 'hello world!');
// Command Interface
$cmd = $lease->getInterface('command');
echo $cmd->execx('uptime');
// HTTPD Interface
$httpd = $lease->getInterface('httpd');
$httpd->restart();
Interfaces are mostly just stock, although installs might add new interfaces if
they expose different ways to interact with resources (for instance, a resource
might want to expose a new 'MongoDB' interface or whatever).
Currently: We have like part of a command interface.
== Leases
Vision: Leases keep track of which resources are in use, and what they're being
used for. They allow us to know when we need to allocate more resources (too
many sandcastles on the existing hosts, e.g.) and when we can release resources
(because they are no longer being used). They also give applications something
to hold while resources are being allocated.
// EXAMPLE: How this should work some day.
$allocator = new DrydockAllocator();
$allocator->setResourceType('sandcastle');
$allocator->setAttributes(
array(
'diffID' => $diff->getID(),
));
$lease = $allocator->allocate();
$diff->setSandcastleLeaseID($lease->getID());
// ...
if ($lease->getStatus() == DrydockLeaseStatus::STATUS_ACTIVE) {
$sandcastle_link = $lease->getInterface('httpd')->getURI('/');
} else {
$sandcastle_link = 'Still building your sandcastle...';
}
echo "Sandcastle for this diff: ".$sandcastle_link;
// EXAMPLE: How this actually works now.
$allocator = new DrydockAllocator();
$allocator->setResourceType('host');
// NOTE: Allocation is currently synchronous but will be task-driven soon.
$lease = $allocator->allocate();
Leases are completely stock, installs will not define new lease types.
Currently: Leases exist and work but are very very basic.
== Resources
Vision: Resources represent some actual thing we've put somewhere, whether it's
a host, a block of storage, a webroot, or whatever else. Applications interact
through resources by acquiring leases to them, and then getting interfaces
through these leases. The lease acquisition process has a side effect of
allocating new resources if a lease can't be acquired on existing resources
(e.g., the application wants storage but all storage resources are full) and
things are configured to autoscale.
Resources may themselves acquire leases in order to allocate. For instance, a
storage resource might first acquire a lease to a host resource. A 'test
scaffold' resource might lease a storage resource and a mysql resource.
Not all resources are auto-allocate: the entry-level version of Drydock is that
you manually allocate a couple boxes and configure them through the web console.
Then, e.g., 'storage' / 'webroot' resources allocate on top of them, but the
host pool itself does not autoscale.
Resources are completely stock, they are abstract shells representing any
arbitrary thing.
Currently: Resource exist ('host' only) but are very very basic.
== Blueprints
Vision: Blueprints contain instructions for building interfaces to, (possibly)
allocating, updating, managing, and destroying a specific type of resource in a
specific location. One way to think of them is that they are scripts for
creating and deleting resources. For example, the LocalHost, RemoteHost and
EC2Host blueprints can all manage 'host' resources.
Eventually, we will support more types of resources (storage, webroot,
sandcastle, test scaffold, phacility deployment) and more providers for resource
types, some of which will be in the Phabricator mainline and some of which will
be custom.
Blueprints are very custom and specific to application types, so installs will
define new blueprints if they are making significant use of Drydock.
Currently: They exist but have few capabilities. The stock blueprints do nearly
nothing useful. There is a technically functional blueprint for host allocation
in EC2.
== Allocator
This is just the actual code to execute the lease acquisition process.
Test Plan: Ran "drydock_control.php" script, it allocated a machine in EC2,
acquired a lease on it, interfaced with it, and then released the lease. Ran it
again, got a fresh lease on the existing resource.
Reviewers: btrahan, jungejason
Reviewed By: btrahan
CC: aran
Differential Revision: https://secure.phabricator.com/D1454
Summary:
See D1416. Add options to file-embed syntax, and document new code and
embed options.
Test Plan: Used new options in markup blocks.
Reviewers: davidreuss, btrahan, jungejason
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T336
Differential Revision: https://secure.phabricator.com/D1417
Summary: Make it more explicit that headers are block formatters, see T778.
Test Plan: Read docs.
Reviewers: davidreuss, btrahan, jungejason
Reviewed By: btrahan
CC: aran
Maniphest Tasks: T778
Differential Revision: https://secure.phabricator.com/D1420
Summary:
- Link to "importing a repository" from Config next steps, since it's not
obvious (and the article isn't obviously named).
- Some minor doc tweaks.
- Remove "Roadmap" document since it's super out of date and not very useful.
Test Plan: Regenerated and read documentation.
Reviewers: btrahan, jungejason
Reviewed By: jungejason
CC: aran, jungejason
Maniphest Tasks: T743
Differential Revision: https://secure.phabricator.com/D1384
Summary: XHProf install documentation went missing a month or two ago (see T725)
and doesn't work in the widely deployed versions of PEAR/PECL. Provide
build-from-source instructions inline.
Test Plan: Generated, read documentation.
Reviewers: btrahan, jungejason
Reviewed By: btrahan
CC: aran, btrahan
Maniphest Tasks: T725
Differential Revision: https://secure.phabricator.com/D1345
Summary: Simple notificaiton handler that reads the difx event timeline and
posts notifications to IRC.
Test Plan: Ran it in #phabricator.
Reviewers: btrahan, jungejason
Reviewed By: btrahan
CC: aran, btrahan
Differential Revision: https://secure.phabricator.com/D1337
Summary: Make it a little easier to create a bunch of accounts if your company
has more than like 5 employees.
Test Plan: Ran "add_user.php" to create new users. Created new users from the
web console.
Reviewers: btrahan, jungejason, rguerin
Reviewed By: btrahan
CC: aran, btrahan, rguerin
Differential Revision: https://secure.phabricator.com/D1336
Summary: This is a fairly common question but I think it's the right product
behavior, document it so I can reference the docs next time it comes up.
Test Plan: Generated and read documentation.
Reviewers: btrahan, jungejason
Reviewed By: jungejason
CC: aran, jungejason
Maniphest Tasks: T735
Differential Revision: https://secure.phabricator.com/D1310
Test Plan: none, not sure how to test this
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1299
Summary:
- On the edit view, this is represented as a checkbox.
- On the detail view, it renders with a user-selectable string.
Test Plan: Added a bool field to my local install, checked and unchecked it.
Reviewers: zeeg, jungejason, btrahan
Reviewed By: jungejason
CC: aran, jungejason
Differential Revision: https://secure.phabricator.com/D1277
Summary: Some day we might have a fancy daemon for this, but for now at least
provide some instructions on using the existing importers, etc., to index
project symbols.
Test Plan:
- Generated documentation, read over the result.
- Ran the example code.
Reviewers: btrahan, jungejason, davidreuss
Reviewed By: jungejason
CC: aran, jungejason
Maniphest Tasks: T315
Differential Revision: https://secure.phabricator.com/D1262
Summary: Write a little documentation about how to get the IRC bot running since
there's a reasonable process with some examples but no documentation.
Test Plan: Generated, read the documentation.
Reviewers: btrahan, jungejason
Reviewed By: jungejason
CC: iAladdin, aran, jungejason
Maniphest Tasks: T686
Differential Revision: 1244
Summary:
- For context, see T547. This is the last (maybe?) in a series of diffs that
moves us off raw sha1() calls in order to make it easier to audit the codebase
for correct use of hash functions.
- This breaks CSRF tokens. Any open forms will generate an error when
submitted, so maybe upgrade off-peak.
- We now generate HMAC mail keys but accept MAC or HMAC. In a few months, we
can remove the MAC version.
- The only remaining callsite is Conduit. We can't use HMAC since Arcanist
would need to know the key. {T550} provides a better solution to this, anyway.
Test Plan:
- Verified CSRF tokens generate properly.
- Manually changed CSRF to an incorrect value and got an error.
- Verified mail generates with a new mail hash.
- Verified Phabricator accepts both old and new mail hashes.
- Verified Phabricator rejects bad mail hashes.
- Checked user log, things look OK.
Reviewers: btrahan, jungejason, benmathews
Reviewed By: btrahan
CC: aran, epriestley, btrahan
Maniphest Tasks: T547
Differential Revision: 1237
Test Plan:
Created a listener that adds some patterns to $matches array, reloaded
Differential, some changesets were not shown as generated.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, epriestley, mareksapota
Differential Revision: 1200
Summary: There was some documentation for this but it was kind of buried in a
random, difficult-to-discover file. Separate it into its own file and link to it
from the previous location.
Test Plan: Regenerated documentation and read through it without catching
anything terrible.
Reviewers: btrahan
Reviewed By: btrahan
CC: zeeg, aran, btrahan
Maniphest Tasks: T643
Differential Revision: 1161
Summary:
- Update documentation for changes in D1148.
- Link to Remarkup documentation from Maniphest.
- Support "Note:" syntax in Phabricator (previously, it was only supported in
Diviner, but I've found it pretty good and useful).
Test Plan: Regenerated and perused documentation; made a "NOTE:".
Reviewers: btrahan, broofa, fugalh, jungejason, nh, aran
Reviewed By: btrahan
CC: aran, btrahan
Differential Revision: 1149
Summary: Allow tweaking Differential mail before sending.
Test Plan:
Wrote a listener renaming Differential attachments and it worked without
problems.
Reviewers: epriestley
Reviewed By: epriestley
CC: aran, epriestley, mareksapota, davidreuss
Differential Revision: 1091
