Summary:
Since PHP 8.1, `base64_decode()` does not accept passing null as a parameter. Thus first check that `data !== null` before calling `decodeBase64($data)` (which then calls PHP's `base64_decode()`), and throw an exception if it is.
```
EXCEPTION: (RuntimeException) base64_decode(): Passing null to parameter #1 ($string) of type string is deprecated at [<arcanist>/src/error/PhutilErrorHandler.php:261]
arcanist(head=master, ref.master=18554ea76ceb), phorge(head=conduitDashboardPanelEdit, ref.master=0d81da590923, ref.conduitDashboardPanelEdit=ab4391b30465)
#0 <#2> PhutilErrorHandler::handleError(integer, string, string, integer) called at [<arcanist>/src/error/PhutilErrorHandler.php:261]
#1 <#2> base64_decode(NULL, boolean) called at [<phorge>/src/applications/files/conduit/FileConduitAPIMethod.php:84]
```
Closes T15426
Test Plan: Applied this change; afterwards `/api/file.upload` under "Method Result", "error_info" shows "Unable to decode base64 data!" instead of "base64_decode(): Passing null to parameter #1 ($string) of type string is deprecated" which is more descriptive.
Reviewers: O1 Blessed Committers, speck
Reviewed By: O1 Blessed Committers, speck
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15426
Differential Revision: https://we.phorge.it/D25258
Summary:
`strlen()` was used in Phabricator to check if a generic value is a non-empty string.
Receiving null is deprecated since PHP 8.1. Phorge adopts `phutil_nonempty_string()` as a replacement.
Note: this may highlight other absurd input values that might be worth correcting
instead of just ignoring. If phutil_nonempty_string() throws an exception in your
instance, report it to Phorge to evaluate and fix that specific corner case.
Closes T15389
Test Plan:
Applied these three changes; afterwards writing `{F1234}` in the Task Comment text field finally rendered the image preview in the preview area, plus I could successfully add the Comment.
Additional cute tests:
```
Test alt:
{F1234,alt=0}
{F1234,alt=null}
{F1234,alt=123}
{F1234,alt=0.1}
{F1234,alt=[]}
Test width:
{F1234,width=0}
{F1234,width=10}
{F1234,width=10.1}
{F1234,width=10%}
{F1234,width=49.99%}
{F1234,width=100%}
Test height:
{F1234,height=0}
{F1234,height=10}
{F1234,height=10.1}
{F1234,height=10%}
{F1234,height=49.99%}
{F1234,height=100%}
Test mix width/height:
{F1234,height=0,width=0}
{F1234,height=10,width=15}
{F1234,height=10.1,width=15}
{F1234,height=10%,width=15%}
{F1234,height=49.99%,width=15%}
{F1234,height=100%,width=15%}
```
If you have no nuclear implosion and if dimensions are somehow respected and the alt text works, well done! Test passed.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15389
Differential Revision: https://we.phorge.it/D25221
Summary:
Since PHP 8.1, conversion from float to integer should be explicit.
https://wiki.php.net/rfc/implicit-float-int-deprecate
According to `phlog`, the alpha channel value of the automatically created user
avatar image for a new user account sometimes is a float.
Thus always `round()` the alpha channel value to be an integer.
Closes T15375
Test Plan: Applied this change, created five user accounts via http://phorge.localhost/people/new/standard/ , and each resulting alpha value in `PhabricatorFilesComposeAvatarBuiltinFile.php` was an integer according to `phlog` (see corresponding Maniphest task). Rendering of each new user page always succeeded.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15375
Differential Revision: https://we.phorge.it/D25209
Summary:
`strlen()` was used in Phabricator to check if a generic value is a non-empty string.
This behavior is deprecated since PHP 8.1. Phorge adopts `phutil_nonempty_string()` as a replacement.
Note: this may highlight other absurd input values that might be worth correcting
instead of just ignoring. If phutil_nonempty_string() throws an exception in your
instance, report it to Phorge to evaluate and fix that specific corner case.
Closes T15342
Test Plan:
Applied these two changes on top of D25186.
Added an Alt Text to a file; page of the File now rendered correctly in web browser.
Removed Alt Text from a file; page of the File now rendered correctly in web browser.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15342
Differential Revision: https://we.phorge.it/D25187
Summary:
`strlen()` was used in Phabricator to check if a generic value is a non-empty string.
This behavior is deprecated since PHP 8.1. Phorge adopts `phutil_nonempty_string()` as a replacement.
Note: this may highlight other absurd input values that might be worth correcting
instead of just ignoring. If phutil_nonempty_string() throws an exception in your
instance, report it to Phorge to evaluate and fix that specific corner case.
Closes T15341
Test Plan: Applied these four changes and file page at `/F91` finally rendered in web browser.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15341
Differential Revision: https://we.phorge.it/D25186
Summary:
`strlen()` was used in Phabricator to check if a generic value is a non-empty string.
This behavior is deprecated since PHP 8.1. Phorge adopts `phutil_nonempty_string()` as
a general replacement.
Note: this may highlight other absurd input values that might be worth correcting
instead of just ignoring. If phutil_nonempty_string() throws an exception in your
instance, report it to Phorge to evaluate and fix that specific corner case.
Use `phutil_nonempty_scalar()` instead in `PhabricatorSearchDateField.php` as
input could only be integer instead of string.
Closes T15297
Test Plan: Applied these four changes (on top of D25144, D25145, D25146) and `/people/` finally rendered in web browser.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15297
Differential Revision: https://we.phorge.it/D25147
Summary:
`strlen()` was used in Phabricator to check if a generic value is a non-empty string.
This behavior is deprecated since PHP 8.1. Phorge adopts `phutil_nonempty_string()` as a replacement.
Note: this may highlight other absurd input values that might be worth correcting
instead of just ignoring. If phutil_nonempty_string() throws an exception in your
instance, report it to Phorge to evaluate and fix that specific corner case.
Closes T15301
Test Plan: Applied this change (on top of D25150) and icons rendered in web browser.
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15301
Differential Revision: https://we.phorge.it/D25151
Summary:
Fix numerous PHP 8.1 RuntimeExceptions caused by the deprecation of strlen(null).
The strlen() was used in Phabricator to check if a generic value was a non-empty string.
For this reason, Phorge adopts phutil_nonempty_string() that checks that.
Note: this may highlight other absurd input values that might be worth correcting
instead of just ignoring. If your phutil_nonempty_string() throws an exception, just
report it to Phorge to evaluate and fix together that specific corner case.
Closes T15264
Test Plan: Phorge homepage is displayed on PHP 8.1 after applying these changes
Reviewers: O1 Blessed Committers, valerio.bozzolan
Reviewed By: O1 Blessed Committers, valerio.bozzolan
Subscribers: speck, tobiaswiese, valerio.bozzolan, Matthew, Cigaryno
Maniphest Tasks: T15264
Differential Revision: https://we.phorge.it/D25132
Summary:
This change avoids some unnecessary uses of the strlen() function,
actually fixing some deprecation warnings in PHP 8.2.
In short, this is the suggested universal replace:
-if(strlen($v))
+if(phutil_nonempty_string($v))
And, if you know PHP, this is also another adoptable replace, but
only for cases where you are sure that the string "0" is not useful:
-if(strlen($v))
+if($v))
As usual the optimal solution depends on the contest.
Other similar patches will probably follow.
Closes T15222
Ref T15190
Test Plan:
- for the first time in my life, with this change, the unit tests are passed in PHP 8.2
- check with your big eyes that there are no obvious typos
Reviewers: O1 Blessed Committers, avivey
Reviewed By: O1 Blessed Committers, avivey
Subscribers: avivey, speck, tobiaswiese, Matthew, Cigaryno
Maniphest Tasks: T15199, T15190, T15222
Differential Revision: https://we.phorge.it/D25104
Summary: Ref T13682. Make the "Attached" list in Files a bit more detailed, and add a "Detach" button.
Test Plan: Tried to detach unrelated, referenced, and attached files. Saw attached files detach.
Maniphest Tasks: T13682
Differential Revision: https://secure.phabricator.com/D21840
Summary: Ref T13682. Many subclasses of "CursorPagedPolicyAwareQuery" have the same implementation of "loadPage()", and this is a sensible default behavior.
Test Plan: Examined changes to verify that all removed methods have the same behavior.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13682
Differential Revision: https://secure.phabricator.com/D21838
Summary: Ref T13682. Allow users to manually attach files which are referenced (but not attached) via the UI.
Test Plan: Reference files via `{F...}`, then attached them via the UI workflow.
Maniphest Tasks: T13682
Differential Revision: https://secure.phabricator.com/D21837
Summary: Ref T13682. Provide a basic UI element for showing file attached to an object.
Test Plan: Viewed objects with attached files in the UI.
Maniphest Tasks: T13682
Differential Revision: https://secure.phabricator.com/D21836
Summary:
Ref T13682. This is a marginal case, but if you quote someone else's text and it has a file reference embedded in it, that isn't good enough to generate "attachment intent".
Since you need both a reference and an explicit attachment, this should never actually affect any user-visible behavior today, but makes the ruleset more thorough.
Test Plan: Dragged and dropped a file, referencing it in either a quoted or unquoted block. Saw it attach only for a quoted block.
Maniphest Tasks: T13682
Differential Revision: https://secure.phabricator.com/D21833
Summary:
Ref T13603. This removes the file attachment edge types, which no longer have readers or writers.
It does not delete the underlying data.
This indirectly removes API access to this edge. As far as I know, this was only used by one customer (see D21480) who has migrated away from Phabricator.
Test Plan: Grepped for edge constants.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21823
Summary: Ref T13603. Removes all reads and writes to the other half of the legacy edge.
Test Plan:
- Verified I could still see file content accessible only through object attachment.
- This fixes a unit test broken by removal of only half the edge access in the previous change.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21820
Summary: Ref T13603. Migrate all code which interacts with the "ObjectHasFile" edge to use the "Attachments" table instead.
Test Plan:
- Edited a paste's view policy, confirmed associated file secret was scrambled.
- Verified I could still view paste content as a user who could not naturally view the underlying file.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21819
Summary: Ref T13603. This adds a second write to new "attachment" storage to all writers except one in Paste, which creates the file inline.
Test Plan:
- Updated a macro image, confirmed a write to "attachment" storage (transaction pathway).
- Updated a blog profile image, confirmed a write to "attachment" storage (legacy pathway).
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21816
Summary:
Ref T13603. Currently, files are sometimes detached from objects. For example, when you change the image for a Macro, the old image is detached.
This is wrong: the image should remain attached so users who can view the macro can view the complete "alice change the image from X to Y" transaction. To be able to understand the change that was applied, you need to be able to view both files.
All workflows which currently detach files aren't conistent with the modern way applications behave, except maybe one callsite in a unit test, and that one's kind of moot.
Get rid of this stuff and just use PHID extraction to perform file attachment in all cases.
Test Plan: Created and edited macros, verified files were properly attached and remained attached across edits.
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21815
Summary: Ref T13603. Prepare to move this relationship out of edge storage into dedicated storage so it is easier to formalize better in the UI.
Test Plan: Ran `bin/storage upgrade`.
Subscribers: PHID-OPKG-gm6ozazyms6q6i22gyam
Maniphest Tasks: T13603
Differential Revision: https://secure.phabricator.com/D21813
Summary: Ref T13588. This allows "arc unit --everything" to more or less run cleanly on PHP 8.1.
Test Plan: Ran "arc unit --everything".
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21821
Summary: Ref T13658
Test Plan:
This test plan is non-exhaustive.
- Looked at some of the config.
- Looked at guides.
Maniphest Tasks: T13658
Differential Revision: https://secure.phabricator.com/D21769
Summary:
Ref T13588. This configuration value may not be set.
Also fix an issue in `bin/storage` and whatever else I hit between now and this diff actually uploading.
Also fix a MySQLi report mode difference, beginning in PHP 8.1.
Also update a bunch of "static" property usage in Lisk.
Test Plan: Ran `bin/files ...` locally under PHP 8.1.
Maniphest Tasks: T13588
Differential Revision: https://secure.phabricator.com/D21744
Summary:
Ref T13635. Currently, the JSON DocumentEngine uses "phutil_json_decode()", but this can confuse "{}" and "[]".
Be more careful about how the JSON value is decoded, to preserve the distinction.
Test Plan: {F8520479}
Maniphest Tasks: T13635
Differential Revision: https://secure.phabricator.com/D21605
Summary:
Ref T13629.
- Allow files to have custom alt text.
- If a file doesn't have alt text, try to generate a plausible default alt text with the information we have.
Test Plan:
- Viewed image files in DocumentEngine diffs, files, `{Fxxx}` embeds, and lightboxes.
- Saw default alt text in all cases, or custom alt text if provided.
- Set, modified, and removed file alt text. Viewed timeline and feed.
- Pulled alt text with "conduit.search".
Maniphest Tasks: T13629
Differential Revision: https://secure.phabricator.com/D21596
Summary:
See PHI1901. An install would like improved support for identifying files related to an object (like a task or revision) for retention/archival/backup/migration/snapshotting purposes.
The "attachment" edge is not really user-level: it just means "if you can see the object, that allows you to see the file". This set includes files that users may not think of as "attached", like thumbnails and internal objects which are attached for technical reasons.
However, this is generally an appropriate relationship to expose for retention purposes.
Test Plan: Used "edge.search" to find files attached to a revision and objects attached to a file.
Differential Revision: https://secure.phabricator.com/D21480
Summary:
Ref PHI1835. Generally, Jupyter notebooks in the wild may store source and markdown content as either a single string or a list of strings.
Make the renderer read these formats more consistently. In particular, this fixes rendering of code blocks stored as a single string.
This also fixes an issue where cell labels were double-rendered in diff views.
Test Plan:
Created a notebook with a code block represented on disk as a single string, rendered a diff from it.
{F7696071}
Differential Revision: https://secure.phabricator.com/D21434
Summary:
See PHI1752.
- Early exit of document layout can cause us to fail to populate available rows.
- Some Jupyter documents have "markdown" cells with plain strings, apparently.
Test Plan: Successfully rendered example diff from PHI1752.
Differential Revision: https://secure.phabricator.com/D21285
Summary:
Ref T13513. Currently, viewing a Jupyter document, hidden context just gets a plain "* * *" facade with no way to expand it.
Support click-to-expand, like source changes.
Test Plan:
- Clicked to expand various Jupyter diffs.
- Clicked to expand normal source changes.
Maniphest Tasks: T13513
Differential Revision: https://secure.phabricator.com/D21243
Summary: Ref T13528. Now that we're hinting users into Files, put the content first and move the detail panel under it. Move the most-useful details (author, size, dimensions) into the curtain.
Test Plan: {F7409925}
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21201
Summary: Ref T13528. When a file in Paste (like a Jupyter notebook) has a good/useful document engine, provide a link to Files.
Test Plan: {F7409881}
Maniphest Tasks: T13528
Differential Revision: https://secure.phabricator.com/D21196
Summary:
Ref T13507. We currently compress normal responses, but do not compress file data responses because most files we serve are images and already compressed.
However, there are some cases where large files may be highly compressible (e.g., huge XML files stored in LFS) and we can benefit from compressing responses.
Make a reasonable guess about whether compression is beneficial and enable compression if we guess it is.
Test Plan:
- Used `curl ...` to download an image with `Accept-Encoding: gzip`. Got raw image data in the response (as expected, because we don't expect images to be worthwhile to recompress).
- Used `curl ...` to download a text file with `Accept-Encoding: gzip`. Got a compressed response. Decompressed the response into the original file.
Maniphest Tasks: T13507
Differential Revision: https://secure.phabricator.com/D21125
Summary: Ref T13395. Move cache classes, syntax highlighters, other markup classes, and sprite sheets to Phabricator.
Test Plan: Attempted to find any callers for any of this stuff in libphutil or Arcanist and couldn't.
Maniphest Tasks: T13395
Differential Revision: https://secure.phabricator.com/D20977
Summary:
Fixes T13468. See that task for discussion. The older source-rendering code mixes "line number" / "1-based" lists with "block number" / "0-based" lists and then has other bugs which cancel this out.
For block-based diffs, build an explicit block-based mask with only block numbers. This sort of sidesteps the whole issue.
Test Plan: Viewed the diff with the original reproduction case, plus various other block-based diffs, including one-block image diffs, in unified and side-by-side mode. Didn't spot any oddities.
Maniphest Tasks: T13468
Differential Revision: https://secure.phabricator.com/D20959
Summary: Ref T13425. When a file (like a Jupyter notebook) is added or removed, we can still render a useful line-by-line diff.
Test Plan:
- Viewed add/modify/remove of Jupyter, source code, and images in 2up/1up mode, everything looked okay.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20865
Summary: Ref T13425. When we render a diff between two source lines, highlight intraline changes.
Test Plan: Viewed a Jupyter notebook with a code diff in it, saw the changed subsequences in the line highlighted.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20851
Summary:
Depends on D20844. Ref T13425. When we line up two blocks and they can be interdiffed (generally: they both have the same type of content), let the Engine interdiff them.
Then, make the Jupyter engine interdiff markdown.
Test Plan: {F6898583}
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20845
Summary:
Depends on D20843. Ref T13425. Add very basic support for "Show Hidden Context", in the form of folding it behind an unclickable shield. This isn't ideal, but should be better than nothing.
Prepare for "intraline" diffs on content blocks.
Fix newline handling in Markdown sections in Jupyter notebooks.
Remove the word "visibile" from the codebase.
Test Plan: {F6898192}
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20844
Summary:
Ref T13425. Some diff checks currently sequence incorrectly:
- When we're rendering block lists, syntax highlighting isn't relevant.
- The "large change" guard can prevent rendering of otherwise-renderable changes.
- Actual errors in the document engine (like bad JSON in a ".ipynb" file) aren't surfaced properly.
Improve sequencing somewhat to resolve these issues.
Test Plan:
- Viewed a notebook, no longer saw a "highlighting disabled" warning.
- Forced a notebook to fail, got a useful inline error instead of a popup dialog error.
- Forced a notebook to have a large number of differences, got a rendering out of it.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20843
Summary:
Ref T13425. Currently, code inputs and all outputs are grouped into a single block. This is fine for display notebooks but not great for diffing notebooks.
Instead, split source code input into individual lines with one line per block, and each output into its own block.
This allows you to leave actual line-by-line inlines on source, and comment on outputs individually.
Test Plan: {F6888583}
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20840
Summary: Depends on D20838. Fixes T13414. Instead of doing coarse diffing with "PhutilEditDistanceMatrix", use hash-and-diff with "DocumentEngine".
Test Plan:
- On a large document (~3K top level blocks), saw a more sensible diff, instead of the whole thing falling back to "everything changed" mode.
- On a small document, still saw a sensible granular diff.
{F6888249}
Maniphest Tasks: T13414
Differential Revision: https://secure.phabricator.com/D20839
Summary:
Depends on D20835. Ref T13425. Ref T13414. When a document has a list of content blocks, we may not be able to diff it directly, but we can hash each block and then diff the hashes (internally "diff" also does approximately the same thing).
We could do this ourselves with slightly fewer layers of indirection, but: diff already exists; we already use it; we already have a bunch of abstractions on top of it; and it's likely much faster on large inputs than the best we can do in PHP.
Test Plan: {F6888169}
Maniphest Tasks: T13425, T13414
Differential Revision: https://secure.phabricator.com/D20836
Summary:
Depends on D20832. Ref T13425. Emit Jupyter notebooks as diffable blocks with block keys.
No diffing or proper inlines yet.
Test Plan: {F6888058}
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20833
Summary:
Depends on D20830. Ref T13425. Have the image engine elect into block rendering, then emit blocks.
This is rough (the blocks aren't actually diffed yet) but image diffs were already pretty rough so this is approximately a net improvement.
Test Plan: Viewed image diffs, saw nothing worse than before.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20831
Summary:
Ref T13425. Allow DocumentEngines to claim they can produce diffs. If both sides of a change can be diffed by the same document engine and it can produce diffs, have it diff them.
This has no impact on runtime behavior because no upstream engine elects into diff generation yet.
Test Plan: Loaded some revisions, nothing broke.
Maniphest Tasks: T13425
Differential Revision: https://secure.phabricator.com/D20830
Summary: Depends on D20668. Ref T13343. Just an easy cleanup/simplification while I'm here.
Test Plan: `grep` for `getActionConstant()`
Reviewers: amckinley
Reviewed By: amckinley
Maniphest Tasks: T13343
Differential Revision: https://secure.phabricator.com/D20669
