mirror of
https://we.phorge.it/source/phorge.git
synced 2024-11-25 00:02:41 +01:00
Disallow webcrawlers to index Diffusion commits
Summary: Phorge already sets `Disallow: /diffusion/` and `Disallow: /source/`. Thus consequently also disallow accessing specific commits via `/r*`. See https://secure.phabricator.com/T4610 for previous discussions. Closes T15670 Test Plan: Go to `/robots.txt` in the web browser. Cross fingers that more webcrawlers abide by RFC 9309. Reviewers: O1 Blessed Committers, speck Reviewed By: O1 Blessed Committers, speck Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno Maniphest Tasks: T15670 Differential Revision: https://we.phorge.it/D25474
This commit is contained in:
parent
76ed0c7ff7
commit
05f4d5071f
1 changed files with 2 additions and 0 deletions
|
@ -18,6 +18,8 @@ final class PhabricatorRobotsPlatformController
|
||||||
$out[] = 'User-Agent: *';
|
$out[] = 'User-Agent: *';
|
||||||
$out[] = 'Disallow: /diffusion/';
|
$out[] = 'Disallow: /diffusion/';
|
||||||
$out[] = 'Disallow: /source/';
|
$out[] = 'Disallow: /source/';
|
||||||
|
// See T15670. Also prevent directly accessing commits in Diffusion.
|
||||||
|
$out[] = 'Disallow: /r*';
|
||||||
|
|
||||||
// See T15662. Prevent indexing line anchor links in Pastes. Per RFC 9309
|
// See T15662. Prevent indexing line anchor links in Pastes. Per RFC 9309
|
||||||
// section 2.2.3, percentage-encode "$" to avoid interpretation as end of
|
// section 2.2.3, percentage-encode "$" to avoid interpretation as end of
|
||||||
|
|
Loading…
Reference in a new issue