mirror of
https://we.phorge.it/source/phorge.git
synced 2024-11-27 01:02:42 +01:00
Update some Audit documentation
Summary: Ref T10978. - Generally refresh this documentation. - Use the word "publish", not the word "push", to distinguish between review and audit, echoing the language in the "Write, Review, Merge, Publish" document. - Mention the new "Needs Verification" state. Test Plan: Read documentation. Reviewers: chad Reviewed By: chad Maniphest Tasks: T10978 Differential Revision: https://secure.phabricator.com/D17253
This commit is contained in:
parent
97cac83e9b
commit
3b8e2739fc
2 changed files with 148 additions and 70 deletions
|
@ -1,71 +1,125 @@
|
||||||
@title Audit User Guide
|
@title Audit User Guide
|
||||||
@group userguide
|
@group userguide
|
||||||
|
|
||||||
Guide to the Audit (post-push code review) tool and workflow.
|
Guide to using Phabricator to audit published commits.
|
||||||
|
|
||||||
= Overview =
|
|
||||||
|
|
||||||
Phabricator supports two code review workflows, "review" (pre-push) and
|
Overview
|
||||||
"audit" (post-push). To understand the differences between the two, see
|
========
|
||||||
|
|
||||||
|
Phabricator supports two code review workflows, "review" (pre-publish) and
|
||||||
|
"audit" (post-publish). To understand the differences between the two, see
|
||||||
@{article:User Guide: Review vs Audit}.
|
@{article:User Guide: Review vs Audit}.
|
||||||
|
|
||||||
This document summarizes the post-push "audit" workflow implemented by the
|
|
||||||
creatively-named //Audit// tool.
|
|
||||||
|
|
||||||
= How Audit Works =
|
How Audit Works
|
||||||
|
===============
|
||||||
|
|
||||||
Using auditing allows you to push and deploy code without waiting for code
|
The audit workflow occurs after changes have been published. It provides ways
|
||||||
review, while still doing code review eventually. The Audit tool primarily keeps
|
to track, discuss, and resolve issues with commits that are discovered after
|
||||||
track of two things:
|
they go through whatever review process you have in place (if you have one).
|
||||||
|
|
||||||
|
Two examples of how you might use audit are:
|
||||||
|
|
||||||
|
**Fix Issues**: If a problem is discovered after a change has already been
|
||||||
|
published, users can find the commit which introduced the problem and raise a
|
||||||
|
concern on it. This notifies the author of the commit and prompts them to
|
||||||
|
remedy the issue.
|
||||||
|
|
||||||
|
**Watch Changes**: In some cases, you may want to passively look over changes
|
||||||
|
that satisfy some criteria as they are published. For example, you may want to
|
||||||
|
review all Javascript changes at the end of the week to keep an eye on things,
|
||||||
|
or make sure that code which impacts a subsystem is looked at by someone on
|
||||||
|
that team, eventually.
|
||||||
|
|
||||||
|
Developers may also want other developers to take a second look at things if
|
||||||
|
they realize they aren't sure about something after a change has been published,
|
||||||
|
or just want to provide a heads-up.
|
||||||
|
|
||||||
|
You can configure Herald rules and Owners packages to automatically trigger
|
||||||
|
audits of commits that satisfy particular criteria.
|
||||||
|
|
||||||
|
|
||||||
|
Audit States and Actions
|
||||||
|
========================
|
||||||
|
|
||||||
|
The audit workflow primarily keeps track of two things:
|
||||||
|
|
||||||
- **Commits** and their audit state (like "Not Audited", "Approved", or
|
- **Commits** and their audit state (like "Not Audited", "Approved", or
|
||||||
"Concern Raised").
|
"Concern Raised").
|
||||||
- **Audit Requests** which ask a user (or some other entity) to audit a
|
- **Audit Requests** which ask a user (or some other entity, like a project
|
||||||
commit. These can be triggered in a number of ways (see below).
|
or package) to audit a commit. These can be triggered in a number of ways
|
||||||
|
(see below).
|
||||||
|
|
||||||
In the Audit tool's home screen and on the homepage you can see commits and
|
Users interact with commits by leaving comments and applying actions, like
|
||||||
requests that require your action:
|
accepting the changes or raising a concern. These actions change the state of
|
||||||
|
their own audit and the overall audit state of the commit. Here's an example of
|
||||||
|
a typical audit workflow:
|
||||||
|
|
||||||
- **Required Audits** are open audit requests that require you, a project
|
- Alice publishes a commit containing some Javascript.
|
||||||
you are a member of, or a package you own to audit a commit. An audit
|
- This triggers an audit request to Bailey, the Javascript technical
|
||||||
request is closed when you approve the associated commit.
|
lead on the project (see below for a description of trigger mechanisms).
|
||||||
- **Problem Commits** are commits you authored which someone has raised a
|
- Later, Bailey logs into Phabrictor and sees the audit request. She ignores
|
||||||
concern about in audit. Problem commits go away when you satisfy all the
|
it for the moment, since it isn't blocking anything. At the end of the
|
||||||
auditors and get them to "Approve" the commit.
|
week she looks through her open requests to see what the team has been
|
||||||
|
up to.
|
||||||
|
- Bailey notices a few minor problems with Alice's commit. She leaves
|
||||||
|
comments describing improvements and uses "Raise Concern" to send the
|
||||||
|
commit back into Alice's queue.
|
||||||
|
- Later, Alice logs into Phabricator and sees that Bailey has raised a
|
||||||
|
concern (usually, Alice will also get an email). She resolves the issue
|
||||||
|
somehow, maybe by making a followup commit with fixes.
|
||||||
|
- After the issues have been dealt with, she uses "Request Verification" to
|
||||||
|
return the change to Bailey so Bailey can verify that the concerns have
|
||||||
|
been addressed.
|
||||||
|
- Bailey uses "Accept Commit" to close the audit.
|
||||||
|
|
||||||
For example:
|
In {nav Diffusion > Browse Commits}, you can review commits and query for
|
||||||
|
commits with certain audit states. The default "Active Audits" view shows
|
||||||
|
all of the commits which are relevant to you given their audit state, divided
|
||||||
|
into buckets:
|
||||||
|
|
||||||
- Evan creates commit `abcdef1234` and pushes it to the remote.
|
- **Needs Attention**: These are commits which you authored that another
|
||||||
- This triggers an audit request to Bob through some mechanism (see below for
|
user has raised a concern about: for example, maybe they believe they have
|
||||||
a description of trigger mechanisms).
|
found a bug or some other problem. You should address the concerns.
|
||||||
- Later, Bob logs into Phabricator and sees the audit request on his homepage.
|
- **Needs Verification**: These are commits which someone else authored
|
||||||
- Bob clicks through and examines the commit. He notices a problem, so he
|
that you previously raised a concern about. The author has indicated that
|
||||||
selects "Raise Concern" and describes the issue in a comment.
|
they believe the concern has been addressed. You should verify that the
|
||||||
- Evan receives an email that Bob has raised a concern about his commit. He
|
remedy is satisfactory and accept the change, or raise a further concern.
|
||||||
opts not to deal with it immediately.
|
- **Ready to Audit**: These are commits which someone else authored that you
|
||||||
- Later, Evan logs into Phabricator and sees the commit on his homepage
|
have been asked to audit, either by a user or by a system rule. You should
|
||||||
under "Problem Commits".
|
look over the changes and either accept them or raise concerns.
|
||||||
- Evan resolves the issue somehow (e.g., by discussing it with Bob, or fixing
|
- **Waiting on Authors**: These are commits which someone else authored that
|
||||||
it in another commit).
|
you previously raised a concern about. The author has not responded to the
|
||||||
- Now satisfied, Bob "Accepts" the original commit.
|
concern yet. You may want to follow up.
|
||||||
- This causes the request to disappear from Bob's queue, and the commit to
|
- **Waiting on Auditors**: These are commits which you authored that someone
|
||||||
disappear from Evan's queue.
|
else needs to audit.
|
||||||
|
|
||||||
= Audit Triggers =
|
You can use the query constraints to filter this list or find commits that
|
||||||
|
match certain criteria.
|
||||||
|
|
||||||
|
|
||||||
|
Audit Triggers
|
||||||
|
==============
|
||||||
|
|
||||||
Audit requests can be triggered in a number of ways:
|
Audit requests can be triggered in a number of ways:
|
||||||
|
|
||||||
|
- You can add auditors explicitly from the web UI, using either "Edit Commit"
|
||||||
|
or the "Change Auditors" action. You might do this if you realize you are
|
||||||
|
not sure about something that you recently published and want a second
|
||||||
|
opinion.
|
||||||
- If you put `Auditors: username1, username2` in your commit message, it will
|
- If you put `Auditors: username1, username2` in your commit message, it will
|
||||||
trigger an audit request to those users when you push it to a tracked
|
trigger an audit request to those users when you push it to a tracked
|
||||||
branch.
|
branch.
|
||||||
- You can create rules in Herald that trigger audits based on properties
|
- You can create rules in Herald that trigger audits based on properties
|
||||||
of the commit -- like the files it touches, the text of the change, the
|
of the commit -- like the files it touches, the text of the change, the
|
||||||
author, etc.
|
author, etc.
|
||||||
- You can create an audit request for yourself by commenting on any commit.
|
- You can create an Owners package and enable automatic auditing for the
|
||||||
- You can create an Owners package and select "Enable Auditing" (this is an
|
package.
|
||||||
advanced feature which is only likely to be useful for very large teams).
|
|
||||||
|
|
||||||
= Audits in Small Teams =
|
|
||||||
|
Audits in Small Teams
|
||||||
|
=====================
|
||||||
|
|
||||||
If you have a small team and don't need complicated trigger rules, you can set
|
If you have a small team and don't need complicated trigger rules, you can set
|
||||||
up a simple audit workflow like this:
|
up a simple audit workflow like this:
|
||||||
|
@ -84,7 +138,9 @@ commit should have //someone// look at it".
|
||||||
Once your team gets bigger, you can refine this ruleset so that developers see
|
Once your team gets bigger, you can refine this ruleset so that developers see
|
||||||
only changes that are relevant to them.
|
only changes that are relevant to them.
|
||||||
|
|
||||||
= Audit Tips =
|
|
||||||
|
Audit Tips
|
||||||
|
==========
|
||||||
|
|
||||||
- When viewing a commit, audit requests you are responsible for are
|
- When viewing a commit, audit requests you are responsible for are
|
||||||
highlighted. You are responsible for a request if it's a user request
|
highlighted. You are responsible for a request if it's a user request
|
||||||
|
@ -99,6 +155,8 @@ only changes that are relevant to them.
|
||||||
you submit a comment at the bottom of the page.
|
you submit a comment at the bottom of the page.
|
||||||
- Press "?" to view keyboard shortcuts.
|
- Press "?" to view keyboard shortcuts.
|
||||||
|
|
||||||
= Next Steps =
|
|
||||||
|
Next Steps
|
||||||
|
==========
|
||||||
|
|
||||||
- Learn more about Herald at @{article:Herald User Guide}.
|
- Learn more about Herald at @{article:Herald User Guide}.
|
||||||
|
|
|
@ -1,30 +1,45 @@
|
||||||
@title User Guide: Review vs Audit
|
@title User Guide: Review vs Audit
|
||||||
@group userguide
|
@group userguide
|
||||||
|
|
||||||
Discusses the differences between Review and Audit workflows.
|
Discusses the differences between "review" and "audit" workflows.
|
||||||
|
|
||||||
= Overview =
|
Overview
|
||||||
|
========
|
||||||
|
|
||||||
Phabricator supports two similar but separate code review workflows:
|
Phabricator supports two similar but separate code review workflows: "review"
|
||||||
|
and "audit".
|
||||||
|
|
||||||
- **Differential** is used for pre-push code review, called "reviews"
|
Review occurs in **Differential**, before changes are published. You can learn
|
||||||
elsewhere in the documentation. You can learn more in
|
more in @{article:Differential User Guide}.
|
||||||
@{article:Differential User Guide}.
|
|
||||||
- **Audit** is used for post-push code reviews, called "audits" elsewhere in
|
|
||||||
the documentation. You can learn more in @{article:Audit User Guide}.
|
|
||||||
|
|
||||||
(By "pre-push", this document means review which blocks deployment of changes,
|
Audit occurs in **Diffusion**, after changes are published. You can learn more
|
||||||
while "post-push" means review which happens after changes are deployed or
|
in @{article:Audit User Guide}.
|
||||||
en route to deployment.)
|
|
||||||
|
|
||||||
Both are lightweight, asynchronous web-based workflows where reviewers/auditors
|
When this documentation discusses "unpublished changes", it refers to changes
|
||||||
inspect code independently, from their own machines -- not synchronous review
|
which are still subject to being reworked in response to feedback. In many
|
||||||
sessions where authors and reviewers meet in person to discuss changes.
|
workflows, these changes will only exist locally on the developer's machine,
|
||||||
|
but some workflows push tentative or temporary changes into remotes. The step
|
||||||
|
that "publishes" changes might be either pushing or merging them, depending on
|
||||||
|
your workflow.
|
||||||
|
|
||||||
= Advantages of Review =
|
Both the audit and review workflows are lightweight, asynchronous web-based
|
||||||
|
workflows where reviewers or auditors inspect code independently, from their
|
||||||
|
own machines -- not synchronous review sessions where authors and reviewers
|
||||||
|
meet in person to discuss changes.
|
||||||
|
|
||||||
Pre-push review is significantly more powerful than post-push auditing. You
|
Broadly, review is normally a //blocking// workflow: in review workflows,
|
||||||
gain these advantages by requiring review //before// changes may be pushed:
|
authors usually can not publish changes until review completes and reviewers
|
||||||
|
are satisfied.
|
||||||
|
|
||||||
|
In contrast, audit is normally a //nonblocking// workflow: in audit workflows,
|
||||||
|
changes usually move forward by default.
|
||||||
|
|
||||||
|
Advantages of Review
|
||||||
|
====================
|
||||||
|
|
||||||
|
Pre-publish review is significantly more powerful than post-publish auditing.
|
||||||
|
You gain these advantages by requiring review //before// changes may be
|
||||||
|
published:
|
||||||
|
|
||||||
- Authors have a strong incentive to craft small, well-formed changes that
|
- Authors have a strong incentive to craft small, well-formed changes that
|
||||||
will be readily understood, to explain them adequately, and to provide
|
will be readily understood, to explain them adequately, and to provide
|
||||||
|
@ -32,11 +47,12 @@ gain these advantages by requiring review //before// changes may be pushed:
|
||||||
- Reviewers have a real opportunity to make significant suggestions about
|
- Reviewers have a real opportunity to make significant suggestions about
|
||||||
architecture or approach in review. These suggestions are less attractive
|
architecture or approach in review. These suggestions are less attractive
|
||||||
to adopt from audit, and may be much more difficult to adopt if significant
|
to adopt from audit, and may be much more difficult to adopt if significant
|
||||||
time has passed between push and audit.
|
time has passed between publish and audit.
|
||||||
- Authors have a strong incentive to fix problems and respond to feedback
|
- Authors have a strong incentive to fix problems and respond to feedback
|
||||||
received during review, because it blocks them. Authors have a much weaker
|
received during review because it blocks them. Authors have a much weaker
|
||||||
incentive to address problems raised during audit.
|
incentive to promptly address problems raised during audit.
|
||||||
- Authors can ask reviewers to apply and verify fixes before they are pushed.
|
- Authors can ask reviewers to apply and verify fixes before they are
|
||||||
|
published.
|
||||||
- Authors can easily pursue feedback early, and get course corrections on
|
- Authors can easily pursue feedback early, and get course corrections on
|
||||||
approach or direction.
|
approach or direction.
|
||||||
- Reviewers are better prepared to support a given change once it is in
|
- Reviewers are better prepared to support a given change once it is in
|
||||||
|
@ -54,7 +70,7 @@ a blocking step into the process and generally wastes developer time that could
|
||||||
be better spent developing. This is less true than it appears, because the costs
|
be better spent developing. This is less true than it appears, because the costs
|
||||||
are low and pay for themselves in other ways:
|
are low and pay for themselves in other ways:
|
||||||
|
|
||||||
- Differential is fast and provides a very lightweight process for submitting
|
- Differential is fast and provides a lightweight process for submitting
|
||||||
code for review and for performing review.
|
code for review and for performing review.
|
||||||
- Authors are free to pursue other changes while code is being reviewed. With
|
- Authors are free to pursue other changes while code is being reviewed. With
|
||||||
appropriate change management (like local branching in Git) they can even
|
appropriate change management (like local branching in Git) they can even
|
||||||
|
@ -87,13 +103,15 @@ are low and pay for themselves in other ways:
|
||||||
- With `arc patch`, it is roughly as easy to pull a change out of Differential
|
- With `arc patch`, it is roughly as easy to pull a change out of Differential
|
||||||
as it is to pull it out of the remote.
|
as it is to pull it out of the remote.
|
||||||
|
|
||||||
= Advantages of Audit =
|
Advantages of Audit
|
||||||
|
===================
|
||||||
|
|
||||||
Post-push review is significantly better than nothing. If you are unpersuaded
|
Post-publish audit is a less powerful workflow than pre-publish review, but can
|
||||||
|
supplement review and is better than nothing on its own. If you are unpersuaded
|
||||||
by the arguments above (or work on a team that is unswayed), audits provide
|
by the arguments above (or work on a team that is unswayed), audits provide
|
||||||
some of the benefits of review with less friction:
|
some of the benefits of review with less friction:
|
||||||
|
|
||||||
- Audits are driven entirely by Phabricator, users do not need to install
|
- Audits are driven entirely by Phabricator: users do not need to install
|
||||||
`arc`.
|
`arc`.
|
||||||
- Audits require little adjustment to existing workflows and little training.
|
- Audits require little adjustment to existing workflows and little training.
|
||||||
- Audits are completely nonblocking, and send fewer notifications than review.
|
- Audits are completely nonblocking, and send fewer notifications than review.
|
||||||
|
@ -101,7 +119,8 @@ some of the benefits of review with less friction:
|
||||||
on lower-importance changes or raise issues that are discovered after
|
on lower-importance changes or raise issues that are discovered after
|
||||||
review.
|
review.
|
||||||
|
|
||||||
= Recommendations =
|
Recommendations
|
||||||
|
===============
|
||||||
|
|
||||||
Here are super biased recommendations from developers of code review software:
|
Here are super biased recommendations from developers of code review software:
|
||||||
|
|
||||||
|
@ -117,7 +136,8 @@ Here are super biased recommendations from developers of code review software:
|
||||||
- If you aren't interested in review, just do audits. You can always
|
- If you aren't interested in review, just do audits. You can always
|
||||||
change your mind later. But consider review! It's really good, we promise!
|
change your mind later. But consider review! It's really good, we promise!
|
||||||
|
|
||||||
= Next Steps =
|
Next Steps
|
||||||
|
==========
|
||||||
|
|
||||||
- Learn more about reviews in @{article:Differential User Guide}; or
|
- Learn more about reviews in @{article:Differential User Guide}; or
|
||||||
- learn more about audits in @{article:Audit User Guide}.
|
- learn more about audits in @{article:Audit User Guide}.
|
||||||
|
|
Loading…
Reference in a new issue