1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-18 12:52:42 +01:00

Merge branch 'docs2'

This commit is contained in:
epriestley 2011-06-25 20:34:05 -07:00
commit 0245b6cdf7
16 changed files with 198 additions and 148 deletions

View file

@ -25,7 +25,7 @@ development install for open tasks in the "Bootcamp" project which are owned by
"Up For Grabs". These are small-to-medium-sized bugs and projects intended to "Up For Grabs". These are small-to-medium-sized bugs and projects intended to
introduce new contributors to the codebase. introduce new contributors to the codebase.
= Required Reading = = Suggested Reading =
You should read the relevant coding convention documents before you submit a You should read the relevant coding convention documents before you submit a
change and make sure you're following the project guidelines: change and make sure you're following the project guidelines:
@ -33,17 +33,3 @@ change and make sure you're following the project guidelines:
- @{article:General Coding Standards} (for all languages) - @{article:General Coding Standards} (for all languages)
- @{article:PHP Coding Standards} (for PHP) - @{article:PHP Coding Standards} (for PHP)
- @{article:Javascript Coding Standards} (for Javascript) - @{article:Javascript Coding Standards} (for Javascript)
= Philosophy =
The Phabricator philosophy is simple tools that work well. Performance,
reliability, and ease of use are of paramount importance. This does not exclude
a rich featureset, but adding features shouldn't compromise these pillars.
One of Phabricator's main strengths is that it's pretty fast, but it got there
by accepting only patches which are also pretty fast and reverting out a bunch
of garbage that was really slow. Accepting a few slow patches undoes all this
work, so you need to be prepared to defend the performance of your patches. The
best way you can do this is to do your homework and come armed with concrete
performance measurements and information (XHProf profiles, EXPLAIN for query
plans, etc.)

View file

@ -1,6 +1,8 @@
@title Please Please Please @title Please Please Please
@group flavortext @group flavortext
Please read this document.
When you send a message that says "I got an error when I tried to do something": When you send a message that says "I got an error when I tried to do something":
= Please Please Please = = Please Please Please =

View file

@ -0,0 +1,60 @@
@title Phabricator Project History
@group flavortext
A riveting tale of adventure. In this document, I refer to worldly and
sophisticated engineer Evan Priestley as "I", which is only natural as I am he.
This document is mostly just paragraph after paragraph of self-aggrandizement.
= In The Beginning =
I wrote the original version of Differential in one night at a Facebook
Hackathon in April or May 2007, along with Luke Shepard. I joined the company in
April and code review was already an established and mostly-mandatory part of
the culture, but it happened over email and was inefficient and hard to keep
track of. I remember feeling like I was spending a lot of time waiting for code
review to happen, which was a major motivator for building the tool.
The original name of the tool was "Diffcamp". Some time earlier there had been
an attempt to create a project management tool that was a sort of hybrid between
Trac and Basecamp called "Traccamp". Since we were writing the code review tool
at the height of the brief popularity Traccamp enjoyed, we integrated and called
the new tool Diffcamp even though it had no relation to Basecamp. Traccamp fell
by the wayside shortly thereafter and was eventually removed.
However, Diffcamp didn't share its fate. We spent some more time working on it
and got good enough to win hearts and minds over emailing diffs around and was
soon the de facto method of code review at Facebook.
= The Long Bloat =
For the next two and a half years, Diffcamp grew mostly organically and gained a
number of features like inline commenting, CLI support and git support (Facebook
was 100% SVN in early 2007 but 90%+ of Engineers worked primarily in git with
SVN bridging by 2010). As these patches were contributed pretty much randomly,
it also gained a lot of performance problems, usability issues, and bugs.
Through 2007 and 2008 I worked mostly on frontend and support infrastructure;
among other things, I wrote a static resource management system called Haste. In
2009 I worked on the Facebook Lite site, where I built the Javelin Javascript
library and an MVC-flavored framework called Alite.
But by early 2010, Diffcamp was in pretty bad shape. Two years of having random
features grafted onto it without real direction had left it slow and difficult
to use. Internal feedback on the tool was pretty negative, with a lot of
complaints about performance and stability. The internal XTools team had made
inroads at fixing these problems in late 2009, but they were stretched thin and
the tool had become a sprawling landscape of architectural and implementation
problems.
= Differential =
I joined the new Dev Tools team around February 2010 and took over Diffcamp. I
renamed it to Differential, moved it to a new Alite-based infrastructure with
Javelin, and started making it somewhat less terrible. I eventually wrote
Diffusion and build Herald to replace a very difficult-to-use predecessor. These
tools were less negatively received than the older versions. By December 2010 I
started open sourcing them; Haste became //Celerity// and Alite became
//Aphront//. I wrote Maniphest to track open issues with the project in January
or February and we open sourced Phabricator in late April, shortly after I left
Facebook.

View file

@ -0,0 +1,123 @@
@title Things You Should Do Soon: Static Resources
@group flavortext
Over time, you'll write more JS and CSS and eventually need to put systems in
place to manage it.
= Manage Dependencies Automatically =
The naive way to add static resources to a page is to include them at the top
of the page, before rendering begins, by enumerating filenames. Facebook used to
work like that:
COUNTEREXAMPLE
<?php
require_js('js/base.js');
require_js('js/utils.js');
require_js('js/ajax.js');
require_js('js/dialog.js');
// ...
This was okay for a while but had become unmanageable by 2007. Because
dependencies were managed completely manually and you had to explicitly list
every file you needed in the right order, everyone copy-pasted a giant block
of this stuff into every page. The major problem this created was that each page
pulled in way too much JS, which slowed down frontend performance.
We moved to a system (called //Haste//) which declared JS dependencies in the
files using a docblock-like header:
/**
* @provides dialog
* @requires utils ajax base
*/
We annotated files manually, although theoretically you could use static
analysis instead (we couldn't realistically do that, our JS was pretty
unstructured). This allowed us to pull in the entire dependency chain of
component with one call:
require_static('dialog');
...instead of copy-pasting every dependency.
= Include When Used =
The other part of this problem was that all the resources were required at the
top of the page instead of when they were actually used. This meant two things:
- you needed to include every resource that //could ever// appear on a page;
- if you were adding something new to 2+ pages, you had a strong incentive to
put it in base.js.
So every page pulled in a bunch of silly stuff like the CAPTCHA code (because
there was one obscure workflow involving unverified users which could
theoretically show any user a CAPTCHA on any page) and every random thing anyone
had stuck in base.js.
We moved to a system where JS and CSS tags were output **after** page rendering
had run instead (they still appeared at the top of the page, they were just
prepended rather than appended before being output to the browser -- there are
some complexities here, but they are beyond the immediate scope), so
require_static() could appear anywhere in the code. Then we moved all the
require_static() calls to be proximate to their use sites (so dialog rendering
code would pull in dialog-related CSS and JS, for example, not any page which
might need a dialog), and split base.js into a bunch of smaller files.
= Packaging =
The biggest frontend performance killer in most cases is the raw number of HTTP
requests, and the biggest hammer for addressing it is to package related JS
and CSS into larger files, so you send down all the core JS code in one big file
instead of a lot of smaller ones. Once the other groundwork is in place, this is
a relatively easy change. We started with manual package definitions and
eventually moved to automatic generation based on production data.
= Caches and Serving Content =
In the simplest implementation of static resources, you write out a raw JS tag
with something like ##src="/js/base.js"##. This will break disastrously as you
scale, because clients will be running with stale versions of resources. There
are bunch of subtle problems (especially once you have a CDN), but the big one
is that if a user is browsing your site as you push/deploy, their client will
not make requests for the resources they already have in cache, so even if your
servers respond correctly to If-None-Match (ETags) and If-Modified-Since
(Expires) the site will appear completely broken to everyone who was using it
when you push a breaking change to static resources.
The best way to solve this problem is to version your resources in the URI,
so each version of a resource has a unique URI:
rsrc/af04d14/js/base.js
When you push, users will receive pages which reference the new URI so their
browsers will retrieve it.
**But**, there's a big problem, once you have a bunch of web frontends:
While you're pushing, a user may make a request which is handled by a server
running the new version of the code, which delivers a page with a new resource
URI. Their browser then makes a request for the new resource, but that request
is routed to a server which has not been pushed yet, which delivers an old
version of the resource. They now have a poisoned cache: old resource data for
a new resource URI.
You can do a lot of clever things to solve this, but the solution we chose at
Facebook was to serve resources out of a database instead of off disk. Before a
push begins, new resources are written to the database so that every server is
able to satisfy both old and new resource requests.
This also made it relatively easy to do processing steps (like stripping
comments and whitespace) in one place, and just insert a minified/processed
version of CSS and JS into the database.
= Reference Implementation: Celerity =
Some of the ideas discussed here are implemented in Phabricator's //Celerity//
system, which is essentially a simplified version of the //Haste// system used
by Facebook.

View file

@ -12,125 +12,6 @@ Then you can come back and read about these things. These are basically some
problems you'll probably eventually encounter when building a web application problems you'll probably eventually encounter when building a web application
that might be good to start thinking about now. that might be good to start thinking about now.
= Static Resources: JS and CSS = = Things You Should Do Soon =
Over time, you'll write more JS and CSS and eventually need to put systems in - @{article:Things You Should Do Soon: Static Resources}
place to manage it.
== Manage Dependencies Automatically ==
The naive way to add static resources to a page is to include them at the top
of the page, before rendering begins, by enumerating filenames. Facebook used to
work like that:
COUNTEREXAMPLE
<?php
require_js('js/base.js');
require_js('js/utils.js');
require_js('js/ajax.js');
require_js('js/dialog.js');
// ...
This was okay for a while but had become unmanageable by 2007. Because
dependencies were managed completely manually and you had to explicitly list
every file you needed in the right order, everyone copy-pasted a giant block
of this stuff into every page. The major problem this created was that each page
pulled in way too much JS, which slowed down frontend performance.
We moved to a system (called //Haste//) which declared JS dependencies in the
files using a docblock-like header:
/**
* @provides dialog
* @requires utils ajax base
*/
We annotated files manually, although theoretically you could use static
analysis instead (we couldn't realistically do that, our JS was pretty
unstructured). This allowed us to pull in the entire dependency chain of
component with one call:
require_static('dialog');
...instead of copy-pasting every dependency.
== Include When Used ==
The other part of this problem was that all the resources were required at the
top of the page instead of when they were actually used. This meant two things:
- you needed to include every resource that //could ever// appear on a page;
- if you were adding something new to 2+ pages, you had a strong incentive to
put it in base.js.
So every page pulled in a bunch of silly stuff like the CAPTCHA code (because
there was one obscure workflow involving unverified users which could
theoretically show any user a CAPTCHA on any page) and every random thing anyone
had stuck in base.js.
We moved to a system where JS and CSS tags were output **after** page rendering
had run instead (they still appeared at the top of the page, they were just
prepended rather than appended before being output to the browser -- there are
some complexities here, but they are beyond the immediate scope), so
require_static() could appear anywhere in the code. Then we moved all the
require_static() calls to be proximate to their use sites (so dialog rendering
code would pull in dialog-related CSS and JS, for example, not any page which
might need a dialog), and split base.js into a bunch of smaller files.
== Packaging ==
The biggest frontend performance killer in most cases is the raw number of HTTP
requests, and the biggest hammer for addressing it is to package related JS
and CSS into larger files, so you send down all the core JS code in one big file
instead of a lot of smaller ones. Once the other groundwork is in place, this is
a relatively easy change. We started with manual package definitions and
eventually moved to automatic generation based on production data.
== Caches and Serving Content ==
In the simplest implementation of static resources, you write out a raw JS tag
with something like ##src="/js/base.js"##. This will break disastrously as you
scale, because clients will be running with stale versions of resources. There
are bunch of subtle problems (especially once you have a CDN), but the big one
is that if a user is browsing your site as you push/deploy, their client will
not make requests for the resources they already have in cache, so even if your
servers respond correctly to If-None-Match (ETags) and If-Modified-Since
(Expires) the site will appear completely broken to everyone who was using it
when you push a breaking change to static resources.
The best way to solve this problem is to version your resources in the URI,
so each version of a resource has a unique URI:
rsrc/af04d14/js/base.js
When you push, users will receive pages which reference the new URI so their
browsers will retrieve it.
**But**, there's a big problem, once you have a bunch of web frontends:
While you're pushing, a user may make a request which is handled by a server
running the new version of the code, which delivers a page with a new resource
URI. Their browser then makes a request for the new resource, but that request
is routed to a server which has not been pushed yet, which delivers an old
version of the resource. They now have a poisoned cache: old resource data for
a new resource URI.
You can do a lot of clever things to solve this, but the solution we chose at
Facebook was to serve resources out of a database instead of off disk. Before a
push begins, new resources are written to the database so that every server is
able to satisfy both old and new resource requests.
This also made it relatively easy to do processing steps (like stripping
comments and whitespace) in one place, and just insert a minified/processed
version of CSS and JS into the database.
== Reference Implementation: Celerity ==
Some of the ideas discussed here are implemented in Phabricator's //Celerity//
system, which is essentially a simplified version of the //Haste// system used
by Facebook.

View file

@ -12,22 +12,19 @@ with teams. Phabricator is largely based on Facebook's internal tools.
The major components of Phabricator are: The major components of Phabricator are:
- **Differential**, a code review tool; and - **Differential**, a code review tool; and
- **Diffusion**, a repository browser. - **Diffusion**, a repository browser; and
- **Maniphest**, a bug tracker.
Phabricator also includes a number of smaller tools. Phabricator also includes a number of smaller tools.
= Why use Phabricator? = = Why use Phabricator? =
Phabricator gives you a box of reasonably decent tools for a comparatively small Phabricator gives you a box of solid tools for a comparatively small setup cost.
setup cost. Plus: The tools all work together and are richly integrated. The whole thing is free
and open source. You own all your data. Phabricator is extremely fast and proven
- It has some good project names. to scale both to large datasets (Facebook has 500,000+ commits across many
- Facebook integration makes it very hip indeed. repositories) and large organizations (Facebook has 500+ fulltime engineers).
- It's fairly snappy. Phabricator's tools are easy to learn, understand, and use.
- You only have to install and configure one thing instead of setting up
and customizing a bunch of different things.
- A couple of the core tools are actually pretty good.
- Maniphest allows you to close tasks "out of spite".
However, Phabricator may also not be a good solution for you: However, Phabricator may also not be a good solution for you:
@ -35,7 +32,8 @@ However, Phabricator may also not be a good solution for you:
with the toolsets you use lacking. with the toolsets you use lacking.
- If you don't use SVN or Git, you'll have to add support for your VCS before - If you don't use SVN or Git, you'll have to add support for your VCS before
you can get anywhere. you can get anywhere.
- If you loathe PHP, well, it's written in PHP. - If you loathe PHP, well, it's written in PHP. Sorry. It is a victim of
circumstances. We assert it is well-written PHP, at least.
= Next Steps = = Next Steps =