Summary:
Per https://github.com/php-fig/fig-standards/blob/master/proposed/phpdoc.md the valid keyword is `$this`.
Thus replace `this` to make output of static code analysis slightly less noisy.
(See rARC4b7ee1985b3c15b4af81b5480522ae24c0afd6a2 for its Arcanist counterpart.)
Test Plan: Read the docs.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Differential Revision: https://we.phorge.it/D25819
Summary: `arc unit --everything` triggers a `FAIL PhutilRemarkupEngineTestCase::testEngine` due to adding the CSS class `remarkup-link-ext` in rP349f006904fabf1d4df31ff4840502af3ab379a7. Thus update it accordingly.
Test Plan: Run `arc unit`.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Differential Revision: https://we.phorge.it/D25820
Summary:
The question mark in `@param type? $foo Desc` is a custom notation not consistently applied across the codebase and not necessarily obvious to the reader (because custom and not mentioned in https://github.com/php-fig/fig-standards/blob/master/proposed/phpdoc.md ).
Instead, explicitly state "optional" in the parameter description for clarity.
Closes T15925.
Test Plan: Run PHPStan, see no `PHPDoc @param has invalid value (type? [...]` style output anymore.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15925
Differential Revision: https://we.phorge.it/D25797
Summary:
Add variable names (`$varname` in `@param type $varname explanation`) to PHPDoc method headers, for fun and profit.
Closes T15923
Test Plan:
* Read the method signatures and their corresponding PHPDoc headers at your fireplace
* Still run `./bin/diviner generate` without explosions (though it is very lenient anyway?)
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15923
Differential Revision: https://we.phorge.it/D25794
Summary:
`strlen()` was used in Phabricator to check if a generic value is a non-empty string.
This behavior is deprecated since PHP 8.1. Phorge adopts `phutil_nonempty_string()` as a replacement.
Note: this may highlight other absurd input values that might be worth correcting
instead of just ignoring. If phutil_nonempty_string() throws an exception in your
instance, report it to Phorge to evaluate and fix that specific corner case.
All lines changed in this patch had `Parameter #1 $string of function strlen expects string, string|null given` reported by PHPStan. Thus these should be safe to replace with `phutil_nonempty_string` as no calls care about the actual `strlen()` return value (length of a string).
Test Plan: Run static code analysis via `phpstan analyse -l 9`
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Differential Revision: https://we.phorge.it/D25778
Summary:
This is an attempt to improve the default behavior in Remarkup about
links. It does not change any behaviors manually specified in the engine
and it does not change any behaviors related to external domains.
As default, now these kind of links will open in the same tab:
- anchors
- relative URLs
- absolute URLs pointing to the base-URI domain
All the other cases are kept as before - so they open in another tab.
In short, assuming you are we.phorge.it, here the changes:
| |https://gnu.org|[[changelog/]]|[[#anchor|#anchor]]|https://we.phorge.it/|[[/config/|/config/]]|
|Before|external |internal |internal |external |external |
|After |external |internal |internal |**internal** |**internal** |
This situation can further improve but it already covers most of the
cases where most users do not expect to break their navigation into
several tabs. Moreover, if an user wants to open a link in another
window, no one prevents from using the middle mouse button,
or CTRL+click or any other nice really basic feature from their browser.
Also, this change introduces a new CSS class, allowing web designers
to style these external resources.
Example CSS rule to try:
```css
.remarkup-link-ext::before {
content: "[external] ";
}
```
Closes T15161
Closes T15182
Test Plan:
- Copy the example text from this Task: https://we.phorge.it/T15161
- Verify that "internal resources" are internal links as default now
- Verify that "external resources" are still external links as before
Reviewers: O1 Blessed Committers, Cigaryno, avivey, speck
Reviewed By: O1 Blessed Committers, Cigaryno, speck
Subscribers: avivey, speck, tobiaswiese, Matthew, Cigaryno
Maniphest Tasks: T15182, T15161
Differential Revision: https://we.phorge.it/D25118
Summary:
The Remarkup Help page is currently pointing to a HTML page on we.phorge.it.
Because of this it can not be customized/extended by custom extensions.
This diff allows you to implement a `RemarkupSyntaxDocumentationProvider` interface
to your custom class, which will implement a getDocumentation method.
This getDocumentation should return the custom help text in Remarkup format.
This diff also contains some extra help for the cowsay and figlet interpreters.
The local URLs for the local references are:
* /reference/remarkup/
* /reference/cowsay/
* /reference/figlet/
Closes T15401
Test Plan: Open the Phriction wiki editor and click on the Book (Remarkup Reference) icon in the toolbar and validate URL afterwards
Reviewers: O1 Blessed Committers, avivey
Reviewed By: O1 Blessed Committers, avivey
Subscribers: avivey, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15401
Differential Revision: https://we.phorge.it/D25698
Summary:
Check type before potentially mangling HTML in a PhutilSafeHTML element.
For now, do not try to apply this renderer when not dealing with a plain string.
Closes T15802
Test Plan: After deleting the corresponding MarkupCache via `DELETE FROM phabricator_cache.cache_markupcache WHERE cacheData LIKE "%whatever description on the page%";` check the description of a page, e.g. of `/config/edit/security.require-https/` or `/config/edit/storage.default-namespace/`.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15802
Differential Revision: https://we.phorge.it/D25605
Summary:
Implements a remarkup rule that formats the background color to match the value of inline hex color codes.
The format for this rule looks like `{#RRGGBB}` and it will be formatted the same as monospace text, so `#RRGGBB` with the background set to the specified color.
Test Plan:
* run `arc unit` and see the tests pass
* create some markup with {#ff0000} color codes for all of your favorite colors {#ee33ee}
Reviewers: O1 Blessed Committers, avivey
Reviewed By: O1 Blessed Committers, avivey
Subscribers: avivey, aklapper, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15744
Differential Revision: https://we.phorge.it/D25540
Summary:
The file name is a sufficient source of information, if available.
Closes T15729
Test Plan: The new unit test is green. Old unit tests are green.
Reviewers: O1 Blessed Committers, avivey
Reviewed By: O1 Blessed Committers, avivey
Subscribers: avivey, tobiaswiese, Matthew, Cigaryno
Maniphest Tasks: T15729
Differential Revision: https://we.phorge.it/D25560
Summary:
This is supposed to render in plain text instead of crashing or giving an error, after D25415:
a {{{a}}}
The related unit test is updated accordingly.
We have unnoticed this, because of the well-known bug T15500.
Ref T15372
See D25415
Test Plan:
This makes sense with the introduced functionality in D25415.
Unit tests are happy again.
Reviewers: O1 Blessed Committers, avivey
Reviewed By: O1 Blessed Committers, avivey
Subscribers: tobiaswiese, Matthew, Cigaryno
Maniphest Tasks: T15372
Differential Revision: https://we.phorge.it/D25559
Summary:
With this patch, the underlying exception described in T15372#8537 still remains. However, with this patch, the bug is more contained as it is not triggered when not calling an interpreter (`cowsay`, `figlet`), so Phorge does not crash rendering `noValidInterpreter {{{foo}}} bar` lines but renders them as is (for whatever reasons such lines may exist).
See T15372
Test Plan:
Enter strings into a comment:
* `invalid {{{saysay}}} foo` now renders as plain text instead of crashing
* `invalid (invalid) {{{saysay}}} foo` now renders as plain text instead of crashing
* `cowsay (invalid) {{{saysay}}} foo` will still crash as before
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Differential Revision: https://we.phorge.it/D25415
Summary:
We add support to code blocks with the language expressed as GitLab/GitHub/StackOverflow/...
"flavored markdown".
So we support this syntax: (to avoid confusion see it online on the Diff)
lang=text
```php
$asd = 1;
```
Before this change, this was the only supposed syntax in Remarkup, with an explicit "lang=":
lang=text
```lang=php
$asd = 1;
```
This change introduces a minor risk to eat legitimate Remarkup content, since Remarkup allows
to do a multi-line in this way:
lang=text
```$asd = 1;
$asd = 2;```
The above example still works, but, there is a chance that hardcore Remarkup people
have a problem when doing a code block to mention programming languages.
In short, this can be problematic since "cpp" will be eaten from this list:
COUNTEREXAMPLE
```cpp
php
python
```
Using the above example is not socially nice because it is not usable in GitLab, GitHub and Stack Overflow.
If your first line is eaten:
Just *add* a newline on the top to reach a valid raw Markdown list (suggested, valid in Remarkup + Markdown):
lang=text
```
cpp
php
python
```
Or, just add "text" to specify that as language (suggested, valid in Remarkup + Markdown):
lang=text
```text
cpp
php
python
```
Or, just *remove* a newline from the bottom to reach a valid raw Remarkup list (Remarkup-only):
lang=text
```cpp
php
python```
Or, just specify that you are writing in the language "text" (Remarkup-only):
lang=text
```lang=text
cpp
php
python```
To reduce impact and help you, the logic of this strict implementation is:
- must have backticks
- must not have any valid remarkup option, like lang=, counterexample, etc.
- must not have content in the same line of the last backticks
- must have a known language in our proposed subset
If everything is OK, we remove that language from the content since it would be otherwise displayed.
Interestingly, this could improve performance when rendering README files or snippets from external
websites, since - in case - we do not need to guess the language using our deep dark magic.
Closes T15481
Test Plan:
We added some nice unit tests. Ensure that this test passes:
PhutilRemarkupEngineTestCase::testEngine
Optionally, take vision of these, before and after:
https://we.phorge.it/P16
Change the test plan slightly every time, to make sure it is not in your cache.
Reviewers: O1 Blessed Committers, avivey
Reviewed By: O1 Blessed Committers, avivey
Subscribers: avivey, speck, tobiaswiese, Matthew, Cigaryno
Maniphest Tasks: T15481
Differential Revision: https://we.phorge.it/D25299
Summary:
Since PHP 8.1, passing a null string to `trim()` is deprecated.
Thus first check that `$content` is not null before trimming it.
Also since trim() returns a string and never null, we can also simplify the non-empty check,
in a more readable and efficient way, avoiding strlen() that was usually used for other "more wild" cases.
```
EXCEPTION: (RuntimeException) trim(): Passing null to parameter #1 ($string) of type string is deprecated at [<arcanist>/src/error/PhutilErrorHandler.php:261]
arcanist(head=master, ref.master=18554ea76ceb), phorge(head=master, ref.master=0d81da590923)
#0 <#2> PhutilErrorHandler::handleError(integer, string, string, integer) called at [<arcanist>/src/error/PhutilErrorHandler.php:261]
#1 <#2> trim(NULL) called at [<phorge>/src/infrastructure/markup/blockrule/PhutilRemarkupTableBlockRule.php:117]
```
Closes T15427
Test Plan: Applied this change, afterwards `/conduit/method/harbormaster.sendmessage/` correctly rendered in web browser.
Reviewers: O1 Blessed Committers, speck, valerio.bozzolan
Reviewed By: O1 Blessed Committers, speck, valerio.bozzolan
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15427
Differential Revision: https://we.phorge.it/D25259
Summary:
Removes the -khtml, -moz and -ms prefix, since most Browsers are natively supporting the user select directive.
The -webkit prefix is still kept or added for Safari, wich does not support user-select.
Ref: see https://we.phorge.it/D25024#815 for context
Test Plan: Removing the CSS should change nothing in modern browsers.
Reviewers: O1 Blessed Committers, speck, valerio.bozzolan, Matthew
Reviewed By: O1 Blessed Committers, speck, valerio.bozzolan
Subscribers: Cigaryno, Matthew, speck, tobiaswiese, valerio.bozzolan
Differential Revision: https://we.phorge.it/D25025
Summary:
Ref T13682. This is a marginal case, but if you quote someone else's text and it has a file reference embedded in it, that isn't good enough to generate "attachment intent".
Since you need both a reference and an explicit attachment, this should never actually affect any user-visible behavior today, but makes the ruleset more thorough.
Test Plan: Dragged and dropped a file, referencing it in either a quoted or unquoted block. Saw it attach only for a quoted block.
Maniphest Tasks: T13682
Differential Revision: https://secure.phabricator.com/D21833
Summary: Ref T13588. This allows "arc unit --everything" to more or less run cleanly on PHP 8.1.
Test Plan: Ran "arc unit --everything".
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21821
Summary: Ref T13678. When remarkup content embeds other remarkup content, detect and degrade if the references have nesting depth greater than 1. This is a coarse cycle detector, since rendering shallow (but technically non-cycling) trees doesn't seem valuable.
Test Plan: Created various objects with self-references, saw everything degrade properly (after one level of embedding) when embedded in itself and in other contexts. See attached screenshot.
Maniphest Tasks: T13678
Differential Revision: https://secure.phabricator.com/D21810
Summary: Ref T13658.
Test Plan:
This test plan is non-exhaustive.
- Ran `bin/storage databases`.
- Viewed Badges UI exmaples page.
- Used eval rule for `strings.platform.server.name`, got "Phabricator".
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21773
Summary:
Ref T13667. The Herald "content added" rule (and other similar rules) do not correctly extract content from empty commits.
When we load an empty raw diff, return an empty changed content map.
Ref T13588. Also fix some PHP8.1 null/string stuff
Test Plan:
- Ran "bin/repository reparse --publish <commit>", with an empty commit hash and a nonempty commit hash.
- Reviewed Herald transcripts for general sanity.
Maniphest Tasks: T13667, T13588
Differential Revision: https://secure.phabricator.com/D21761
Summary:
Ref T13658. This adds a simple expression evaluator to Remarkup and supports platform name expressions. The syntax is:
```
${{{strings.platform.server.name}}}
```
Note that this won't work inside code blocks (or literal blocks, or other block-level literal elements) right now, although it could be made to selectively (the ".path" expressions might be useful in documentation codeblocks).
Test Plan: {F9391006}
Reviewers: cspeckmim
Reviewed By: cspeckmim
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21713
Summary:
Ref T13608. When searching for bare URIs in remarkup text, don't look for URIs with a protocol string longer than 32 characters.
This avoids a case where the regexp engine may be tricked into executing at `O(N^2)` or some similar complexity.
Test Plan:
- Applied remarkup to "AAAA..." (512KB).
- Before: 64 seconds to process.
- After: <10ms to process.
- Ran unit tests.
Maniphest Tasks: T13608
Differential Revision: https://secure.phabricator.com/D21562
Summary: These characters are missing support in `{key ...}` but are reasonable to include.
Test Plan: {F8302969}
Differential Revision: https://secure.phabricator.com/D21508
Summary: Fixes T13530. The block parser could match too many lines in an unterminated "%%%" literal block. Adjust the logic to stop doing this (and hopefully be a little easier to read).
Test Plan: Added a failing test, made it pass.
Maniphest Tasks: T13530
Differential Revision: https://secure.phabricator.com/D21208
Summary: Ref T13395. Move cache classes, syntax highlighters, other markup classes, and sprite sheets to Phabricator.
Test Plan: Attempted to find any callers for any of this stuff in libphutil or Arcanist and couldn't.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D20977
Summary:
Fixes T5427. See PHI1630. See also T13160 and D20568.
In the full HTML table syntax with "<table>", respect linebreaks as literals inside "<td>" cells.
Test Plan: Previewed some full-HTML tables with and without linebreaks, saw what seemed like sensible rendering behavior.
Maniphest Tasks: T5427
Differential Revision: https://secure.phabricator.com/D20971
Summary: Fixes T13487. In PHI1628, an install has a 4MB remarkup corpus which takes a long time to render. This is broadly expected, but a few reasonable improvements fell out of running it through the profiler.
Test Plan:
- Saw local cold-cache end-to-end rendering time drop from 12s to 4s for the highly secret input corpus.
- Verified output has the same hashes before/after.
- Ran all remarkup unit tests.
Maniphest Tasks: T13487
Differential Revision: https://secure.phabricator.com/D20968
Summary: Fixes T13335. When processing quoted blocks, we remove leading empty lines. This logic incorrectly continued after encountering a nonempty line.
Test Plan: Added a test, made it pass. Previewed blocks in web UI.
Maniphest Tasks: T13335
Differential Revision: https://secure.phabricator.com/D20965
Summary:
See <https://hackerone.com/reports/758002>. The link rules don't test that their parameters are flat text before using them in unsafe contexts.
Since almost all rules are lower-priority than these link rules, this behavior isn't obvious. However, two rules have broadly higher priority (monospaced text, and one variation of link rules has higher priority than the other), and the latter can be used to perform an XSS attack with input in the general form `()[ [[ ... | ... ]] ]` so that the inner link rule is evaluated first, then the outer link rule uses non-flat text in an unsafe way.
Test Plan:
Tested examples in HackerOne report. A simple example of broken (but not unsafe) behavior is:
```
[[ `x` | `y` ]]
```
Differential Revision: https://secure.phabricator.com/D20937
Summary: Ref T13410. Fixes T4280. Allows you to put a named anchor into a document explicitly.
Test Plan: Used `{anchor ...}` in Remarkup, used location bar to jump to anchors.
Maniphest Tasks: T13410, T4280
Differential Revision: https://secure.phabricator.com/D20825
Summary:
Depends on D20820. Ref T13410. We currently cut anchor names in the middle, don't support emoji in anchors, and generate relatively short anchors.
Generate slightly longer anchors, allow more unicode, and try not to cut things in the middle.
Test Plan: Created a document with a variety of different anchors and saw them generate more usable names.
Maniphest Tasks: T13410
Differential Revision: https://secure.phabricator.com/D20821
Summary: See PHI1319. Ref T13291. Bump the remarkup cache version, since the old JIRA / Asana rules may exist in the partial cached representation of remarkup blocks from older versions.
Test Plan: Typed some comments with various formatting, saw remarkup work fine.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13291
Differential Revision: https://secure.phabricator.com/D20619
Summary:
Ref T13250. Ref T13249. Some remarkup rules, including `{image ...}` and `{meme ...}`, may cache URIs as objects because the remarkup cache is `serialize()`-based.
URI objects with `query` cached as a key-value map are no longer valid and can raise `__toString()` fatals.
Bump the cache version to purge them out of the cache.
Test Plan: See PHI1074.
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13250, T13249
Differential Revision: https://secure.phabricator.com/D20160
Summary: See D20136. This method is sort of inherently bad because it is destructive for some inputs (`x=1&x=2`) and had "PHP-flavored" behavior for other inputs (`x[]=1&x[]=2`). Move to explicit `...AsMap` and `...AsPairList` methods.
Test Plan: Bit of an adventure, see inlines in a minute.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D20141
Summary: Ref T13116. See PHI526. Currently, the YouTube remarkup rule writes an `<iframe ...>` but does not adjust the Content-Security-Policy appropriately.
Test Plan: Pasted a YouTube link; viewed it in Safari, Chrome and Firefox.
Maniphest Tasks: T13116
Differential Revision: https://secure.phabricator.com/D19277
Summary:
Fixes T8845. Ref T13102. See PHI467. Currently, object monograms like `L1` which appear in Remarkup headers render incorrectly (with an internal placeholder "x") in the table of contents:
{F5475505}
Instead, render them down to just, e.g., `L1` in plain text.
For `{P123}` I just rendered it to `{P123}` since it's not really clear to me what users intend. This could be adjusted if there's some reasonable thing that someone is trying to do with this.
Test Plan: Wrote a Phriction document with several object references (like `L1` and `{P123}`) in headers. After patch, saw "x"-free, sensible-looking header names in the table of contents.
Maniphest Tasks: T13102, T8845
Differential Revision: https://secure.phabricator.com/D19234
Summary:
Ref T13101. This is a minimal change to make "{meme ...}" work with the new Content-Security-Policy by using an Ajax request to generate the image and then swapping the source on the client.
This could be much cleaner (see T5258, etc).
Test Plan: Used `{meme, src=cat6, above=i am, below=cat}`, chuckled completely unironically.
Maniphest Tasks: T13101
Differential Revision: https://secure.phabricator.com/D19196
Summary:
Depends on D19192. Ref T4190. Ref T13101. Instead of directly including the proxy endpoint with `<img src="..." />`, emit a placeholder and use AJAX to make the request. If the proxy fetch fails, replace the placeholder with an error message.
This isn't the most polished implementation imaginable, but it's much less mysterious about errors.
Test Plan: Used `{image ...}` for valid and invalid images, got images and useful error messages respectively.
Maniphest Tasks: T13101, T4190
Differential Revision: https://secure.phabricator.com/D19193
Summary:
Depends on D19092. Ref T13077. This modernizes markup rendering for PhrictionContent.
This is a little messy because table of contents generation isn't straightforward.
Test Plan: Viewed Phriction documents with and without 3+ headers, saw ToC vs no ToC. Edited/previewed documents. Grepped for affected symbols. Checked DarkConsole for sensible cache behavior.
Maniphest Tasks: T13077
Differential Revision: https://secure.phabricator.com/D19093
Summary:
Depends on D19031. Fixes T11389. Currently, we render `Dxxx` in a text context (plain text email) as just a URI.
Instead, render it like `Dxxx <uri>`. This is more faithful to the original intent and preserves `T123/T456` as two separate, usable links.
Test Plan: Wrote `T123/T234` in a task, pulled mail for it with `bin/mail show-outbound`, saw separate clickable links.
Maniphest Tasks: T11389
Differential Revision: https://secure.phabricator.com/D19032
Summary:
Fixes T12867. Also:
- Simplify the code a little.
- Stop mutating this on text/mobile -- there's no inherent value in the "youtu.be" link so I think this just changes the text the user wrote unnecessarily.
Test Plan: {F5013804}
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12867
Differential Revision: https://secure.phabricator.com/D18149
Summary: The tag/shade stuff changed, so purge older markup (like Diviner documents).
Test Plan: {F4972666}
Reviewers: chad
Reviewed By: chad
Differential Revision: https://secure.phabricator.com/D17998
Summary:
Ref T12509. Many of the calls to HMAC+SHA1 are just to compute cachekeys for remarkup objects.
Make these use HMAC+SHA256 instead. There is no downside to swapping these since they just cause a cache miss in the worst case.
I also plan to get rid of `PhabricatorMarkupInterface` eventually, but this doesn't go that far.
Test Plan: Browsed some different types of documents (tasks, legalpad documents, phame blogs / posts, pholio mocks, etc).
Reviewers: chad
Reviewed By: chad
Maniphest Tasks: T12509
Differential Revision: https://secure.phabricator.com/D17631
