mirror of
https://we.phorge.it/source/phorge.git
synced 2024-11-22 14:52:41 +01:00
Add basic documentation for using "bin/worker" to manage imports of large repositories
Summary: Ref T13591. Provide some guidance on the most common cases for wanting to interact with the worker queue. Test Plan: Read documentation. Maniphest Tasks: T13591 Differential Revision: https://secure.phabricator.com/D21536
This commit is contained in:
parent
c9ab363959
commit
365836cd31
1 changed files with 83 additions and 0 deletions
83
src/docs/user/field/worker_queue.diviner
Normal file
83
src/docs/user/field/worker_queue.diviner
Normal file
|
@ -0,0 +1,83 @@
|
|||
@title Managing the Worker Queue
|
||||
@group fieldmanual
|
||||
|
||||
Advanced guide to managing the background worker task queue.
|
||||
|
||||
Overview
|
||||
========
|
||||
|
||||
Phabricator uses daemonized worker processes to execute some tasks (like
|
||||
importing repositories and sending mail) in the background.
|
||||
|
||||
In most cases, this queue will automatically execute tasks in an appropriate
|
||||
order. However, in some cases you may want to exercise greater control over
|
||||
which tasks execute, when, and at what priority.
|
||||
|
||||
Reference: Priority Levels
|
||||
==========================
|
||||
|
||||
Tasks queued by Phabricator use these default priority levels:
|
||||
|
||||
| Priority | Name | Tasks |
|
||||
|---|---|---|
|
||||
| 1000 | `ALERTS` | Time-sensitive notifications and email. |
|
||||
| 2000 | `DEFAULT` | Normal publishing and processing. |
|
||||
| 2500 | `COMMIT` | Import of commits in existing repositories. |
|
||||
| 3000 | `BULK` | Edits applied via "Bulk Edit" interface. |
|
||||
| 3500 | `INDEX` | Search engine index updates. |
|
||||
| 4000 | `IMPORT` | Import of commits in new repositories. |
|
||||
|
||||
Tasks with smaller priority numbers execute before tasks with larger priority
|
||||
numbers (for example, a task with priority 1000 will execute before a task
|
||||
with priority 2000).
|
||||
|
||||
Any positive integer is a valid priority level, and if you adjust the priority
|
||||
of tasks with `bin/worker priority` you may select any level even if
|
||||
Phabricator would never naturally queue tasks at that level. For example, you
|
||||
may adjust tasks to priority `5678`, which will make them execute after all
|
||||
other types of natural tasks.
|
||||
|
||||
Although tasks usually execute in priority order, task execution order is not
|
||||
strictly a function of priority, and task priority does not guarantee execution
|
||||
order.
|
||||
|
||||
Large Repository Imports
|
||||
========================
|
||||
|
||||
The most common case where you may want to make an adjustment to the default
|
||||
behavior of the worker queue is when importing a very large repository like
|
||||
the Linux kernel.
|
||||
|
||||
Although Phabricator will automatically process imports of new repositories at
|
||||
a lower priority level than all other non-import tasks, you may still run into
|
||||
issues like these:
|
||||
|
||||
- You may also want to import one or more //other// new repositories, and
|
||||
would prefer they import at a higher priority.
|
||||
- You may find overall repository performance is impacted by the large
|
||||
repository import.
|
||||
|
||||
You can manually change the priority of tasks with `bin/worker priority`. For
|
||||
example, if your copy of the Linux repository is `R123` and you'd like it to
|
||||
import at a lower priority than all other tasks (including other imports of
|
||||
new repositories), you can run a command like this:
|
||||
|
||||
```
|
||||
phabricator/ $ ./bin/worker priority --priority 5000 --container R123
|
||||
```
|
||||
|
||||
This means: set all tasks associated with container `R123` (in this example,
|
||||
the Linux repository) to priority 5000 (which is lower than any natural
|
||||
priority).
|
||||
|
||||
You can delay tasks until later with `bin/worker delay`, which allows you to
|
||||
schedule tasks to execute at night or over the weekend. For example, to
|
||||
pause an import for 6 hours, run a command like this:
|
||||
|
||||
```
|
||||
phabricator/ $ ./bin/worker delay --until "6 hours" --container R123
|
||||
```
|
||||
|
||||
The selected tasks will not execute until 6 hours from the time this command
|
||||
is issued. You can also provide an explicit date, or "now" to let tasks begin
|
||||
execution immediately.
|
Loading…
Reference in a new issue