phorge-phorge/src/docs/user/userguide/diffusion.diviner

@title Diffusion User Guide
@group userguide

Guide to Diffusion, the Phabricator repository browser.

= Overview =

Diffusion is a repository browser which allows you to explore source code in a
Subversion, Git, or Mercurial repository. It is somewhat similar to software
like Trac and GitWeb.

Diffusion can either import a read-only copy of repositories hosted somewhere
else (for example, from GitHub, Bitbucket or existing hosting) or host
repositories within Phabricator. Hosted repositories support a variety of
triggers and access controls.

Diffusion is integrated with the other tools in the Phabricator suite. For
instance:

  - when you commit Differential revisions to a tracked repository, they are
    automatically updated and linked to the corresponding commits;
  - you can add Herald rules to notify you about commits that match certain
    rules;
  - for hosted repositories, Herald can enforce granular access control rules;
  - in all the tools, commit names are automatically linked.

= Adding Repositories =

Repository administration is accomplished through Diffusion. You can use the
web interface in Diffusion to import an external repository, or create a new
hosted repository.

  - For hosted repositories, make sure you go through the setup instructions
    in @{article:Diffusion User Guide: Repository Hosting} first.
  - For all repositories, you'll need to be running the daemons. If you have
    not set them up yet, see @{article:Managing Daemons with phd}.

By default, you must be an administrator to create a new repository.

= Repository Callsigns and Commit Names =

Each repository is identified by a "callsign", which is a short uppercase string
like "P" (for Phabricator) or "ARC" (for Arcanist).

Each repository must have a unique callsign. Callsigns must be unique within
an install but do not need to be globally unique, so you are free to use the
single-letter callsigns for brevity. For example, Facebook uses "E" for the
Engineering repository, "O" for the Ops repository, "Y" for a Yum package
repository, and so on, while Phabricator uses "P", "ARC", "PHU" for libphutil,
and "J" for Javelin. Keeping callsigns brief will make them easier to use, and
the use of one-character callsigns is recommended if they are reasonably
evocative and you have no more than 26 tracked repositories.

The primary goal of callsigns is to namespace commits to SVN repositories: if
you use multiple SVN repositories, each repository has a revision 1, revision 2,
etc., so referring to them by number alone is ambiguous. However, even for Git
they impart additional information to human readers and allow parsers to detect
that something is a commit name with high probability (and allow distinguishing
between multiple copies of a repository).

Diffusion uses this callsign and information about the commit itself to generate
a commit name, like "rE12345" or "rP28146171ce1278f2375e3646a1e1ea3fd56fc5a3".
The "r" stands for "revision". It is followed by the repository callsign, and
then a VCS-specific commit identifier (for SVN, the commit number; for Git and
Mercurial, the commit hash). When writing the name of a Git commit you may
abbreviate the hash, but note that hash collisions are probable for short prefix
lengths. See this post on the LKML for a historical explanation of Git's
occasional internal use of 7-character hashes:

  https://lkml.org/lkml/2010/10/28/287

Because 7-character hashes are likely to collide for even moderately large
repositories, Diffusion generally uses either a 16-character prefix (which makes
collisions very unlikely) or the full 40-character hash (which makes collisions
astronomically unlikely).

= Running Diffusion Daemons =

In most cases, it is sufficient to run:

  phabricator/bin/ $ ./phd start

...to start the daemons. For a more in-depth explanation of `phd` and daemons,
see @{article:Managing Daemons with phd}.

NOTE: If you have an unusually large install with multiple web frontends, see
notes in @{article:Managing Daemons with phd}.

You can use the repository detail screen and the Daemon Console to monitor the
daemons and their progress importing the repository. Small repositories should
import quickly, while larger repositories may take some time (it takes about 10
minutes to begin discovering commits in Facebook's 350,000-commit primary
repository, and about 18 hours to import it all with 64 taskmasters on modern
hardware). Commits should begin appearing in Diffusion within a few minutes for
all but the largest repositories.

== Tuning Daemons ==

By default, Phabricator launches one daemon to pull and discover all of the
tracked repositories. This works well for a small number of repositories or
a large number of relatively inactive repositories, but might benefit from
tuning in some cases. The daemon makes a rough effort to respect pull
frequencies defined in repository configuration, but may not be able to import
new commits very quickly if you have a large number of repositories (as it is
blocked waiting on I/O from other repositories).

If you want to provide lower commit import latency for some repositories, you
can either launch **more daemons** (which will generally lower latency for all
repositories) or launch additional **dedicated daemons** (which will give you
very fine-grained control over import latency).

=== More Daemons ===

The coarse approach to reducing import latency is to simply launch more daemons,
using `phd`:

  phabricator/bin $ ./phd launch RepositoryPullLocal

This will launch another copy of the daemon. The daemons acquire a global lock
before pulling a repository, so you can launch additional daemons without
causing contention or race conditions.

=== Dedicated Daemons ===

You can take a more fine-grained approach and launch dedicated daemons for
specific repositories or groups of repositories. For example, if you want low
latency on the repositories with callsigns `A` and `B`, but don't care about
latency for the other repositories, you could launch two daemons like this:

  phabricator/bin $ ./phd launch RepositoryPullLocal -- A B
  phabricator/bin $ ./phd launch RepositoryPullLocal -- --not A --not B

The first one will work only on `A` and `B`, and should be able to import
commits with low latency more reliably. The second one will work on all other
repositories.

= Next Steps =

  - Learn about creating a symbol index at
    @{article:Diffusion User Guide: Symbol Indexes}; or
  - set up repository hosting with
    @{article:Diffusion User Guide: Repository Hosting}; or
  - understand daemons in detail with @{article:Managing Daemons with phd}; or
  - give us feedback at @{article:Give Feedback! Get Support!}.