1
0
Fork 0
mirror of https://git.tukaani.org/xz.git synced 2024-04-04 12:36:23 +02:00
Commit graph

1325 commits

Author SHA1 Message Date
Lasse Collin
f05a69685e Build: Include the CMake files in the distribution.
This was supposed to be done in 2020 with 5.2.5 release
already but it was noticed only today. 5.2.5 and 5.2.6
even mention experiemental CMake support in the NEWS entries.

Thanks to Olivier B. for reporting the problem.
2022-08-18 17:51:07 +03:00
Lasse Collin
ad5ef6d3c3 Windows: Fix broken liblzma.dll build with Visual Studio project files.
The bug was introduced in 352ba2d69a
"Windows: Fix building of resource files when config.h isn't used."

That commit fixed liblzma.dll build with CMake while keeping it
working with Autotools on Windows but the VS project files were
forgotten.

I haven't tested these changes.

Thanks to Olivier B. for reporting the bug and for the initial patch.
2022-08-18 17:51:07 +03:00
Lasse Collin
8dfed05bda Bump version and soname for 5.2.6. 2022-08-12 14:30:13 +03:00
Lasse Collin
09b4af4e04 Add NEWS for 5.2.6. 2022-08-12 14:29:28 +03:00
Lasse Collin
692de534fa Add Jia Tan to AUTHORS. 2022-08-12 14:29:08 +03:00
Lasse Collin
275de376a6 Translations: Change the copyright comment string to use with po4a.
This affects the second line in po4a/xz-man.pot. The man pages of
xzdiff, xzgrep, and xzmore are from GNU gzip and under GNU GPLv2+
while the rest of the man pages are in the public domain.
2022-07-25 19:11:17 +03:00
Jia Tan
76a5a752b8 liblzma: Refactor lzma_mf_is_supported() to use a switch-statement. 2022-07-25 18:36:49 +03:00
Jia Tan
749b86c2c1 Build: Don't allow empty LIST in --enable-match-finders=LIST.
It's enforced only when a match finder is needed, that is,
when LZMA1 or LZMA2 encoder is enabled.
2022-07-25 18:36:49 +03:00
Lasse Collin
63e3cdef80 xz: Make --keep accept symlinks, hardlinks, and setuid/setgid/sticky.
Previously this required using --force but that has other
effects too which might be undesirable. Changing the behavior
of --keep has a small risk of breaking existing scripts but
since this is a fairly special corner case I expect the
likehood of breakage to be low enough.

I think the new behavior is more logical. The only reason for
the old behavior was to be consistent with gzip and bzip2.

Thanks to Vincent Lefevre and Sebastian Andrzej Siewior.
2022-07-24 13:29:42 +03:00
Lasse Collin
9055584be0 xzgrep man page: Document exit statuses. 2022-07-24 11:38:19 +03:00
Lasse Collin
57e1ccbb7c xzgrep: Improve error handling, especially signals.
xzgrep wouldn't exit on SIGPIPE or SIGQUIT when it clearly
should have. It's quite possible that it's not perfect still
but at least it's much better.

If multiple exit statuses compete, now it tries to pick
the largest of value.

Some comments were added.

The exit status handling of signals is still broken if the shell
uses values larger than 255 in $? to indicate that a process
died due to a signal ***and*** their "exit" command doesn't take
this into account. This seems to work well with the ksh and yash
versions I tried. However, there is a report in gzip/zgrep that
OpenSolaris 5.11 (not 5.10) has a problem with "exit" truncating
the argument to 8 bits:

    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22900#25

Such a bug would break xzgrep but I didn't add a workaround
at least for now. 5.11 is old and I don't know if the problem
exists in modern descendants, or if the problem exists in other
ksh implementations in use.
2022-07-24 11:38:19 +03:00
Lasse Collin
6351ea1afb xzgrep: Make the fix for ZDI-CAN-16587 more robust.
I don't know if this can make a difference in the real world
but it looked kind of suspicious (what happens with sed
implementations that cannot process very long lines?).
At least this commit shouldn't make it worse.
2022-07-24 11:38:19 +03:00
Lasse Collin
2c1ff2ed6b xzgrep: Use grep -H --label when available (GNU, *BSDs).
It avoids the use of sed for prefixing filenames to output lines.
Using sed for that is slower and prone to security bugs so now
the sed method is only used as a fallback.

This also fixes an actual bug: When grepping a binary file,
GNU grep nowadays prints its diagnostics to stderr instead of
stdout and thus the sed-method for prefixing the filename doesn't
work. So with this commit grepping binary files gives reasonable
output with GNU grep now.

This was inspired by zgrep but the implementation is different.
2022-07-24 11:38:19 +03:00
Lasse Collin
8b0be38a79 xzgrep: Use -e to specify the pattern to grep.
Now we don't need the separate test for adding the -q option
as it can be added directly in the two places where it's needed.
2022-07-24 11:38:19 +03:00
Lasse Collin
4a61867a87 Scripts: Use printf instead of echo in a few places.
It's a good habbit as echo has some portability corner cases
when the string contents can be anything.
2022-07-24 11:38:19 +03:00
Lasse Collin
0e222bf7d7 xzgrep: Add more LC_ALL=C to avoid bugs with multibyte characters.
Also replace one use of expr with printf.

The rationale for LC_ALL=C was already mentioned in
69d1b3fc29 that fixed a security
issue. However, unrelated uses weren't changed in that commit yet.

POSIX says that with sed and such tools one should use LC_ALL=C
to ensure predictable behavior when strings contain byte sequences
that aren't valid multibyte characters in the current locale. See
under "Application usage" in here:

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html

With GNU sed invalid multibyte strings would work without this;
it's documented in its Texinfo manual. Some other implementations
aren't so forgiving.
2022-07-24 11:38:19 +03:00
Lasse Collin
62c1d2bc2d xzgrep: Fix parsing of certain options.
Fix handling of "xzgrep -25 foo" (in GNU grep "grep -25 foo" is
an alias for "grep -C25 foo"). xzgrep would treat "foo" as filename
instead of as a pattern. This bug was fixed in zgrep in gzip in 2012.

Add -E, -F, -G, and -P to the "no argument required" list.

Add -X to "argument required" list. It is an
intentionally-undocumented GNU grep option so this isn't
an important option for xzgrep but it seems that other grep
implementations (well, those that I checked) don't support -X
so I hope this change is an improvement still.

grep -d (grep --directories=ACTION) requires an argument. In
contrast to zgrep, I kept -d in the "no argument required" list
because it's not supported in xzgrep (or zgrep). This way
"xzgrep -d" gives an error about option being unsupported instead
of telling that it requires an argument. Both zgrep and xzgrep
tell that it's unsupported if an argument is specified.

Add comments.
2022-07-24 11:38:19 +03:00
Lasse Collin
372a0d12c9 Tests: Add the .lzma files to test_files.sh. 2022-07-24 11:37:44 +03:00
Lasse Collin
b8e3d0c45b Tests: Add .lzma test files. 2022-07-24 11:37:44 +03:00
Lasse Collin
e96bdf7189 liblzma: Rename a variable and improve a comment. 2022-07-24 11:37:44 +03:00
Lasse Collin
2d54fdf58e Update THANKS. 2022-07-24 11:37:44 +03:00
Lasse Collin
ff54b557fe liblzma: Add optional autodetection of LZMA end marker.
Turns out that this is needed for .lzma files as the spec in
LZMA SDK says that end marker may be present even if the size
is stored in the header. Such files are rare but exist in the
real world. The code in liblzma is so old that the spec didn't
exist in LZMA SDK back then and I had understood that such
files weren't possible (the lzma tool in LZMA SDK didn't
create such files).

This modifies the internal API so that LZMA decoder can be told
if EOPM is allowed even when the uncompressed size is known.
It's allowed with .lzma and not with other uses.

Thanks to Karl Beldan for reporting the problem.
2022-07-24 11:36:56 +03:00
Lasse Collin
bb795fe835 Tests: Add test file good-1-empty-bcj-lzma2.xz.
This is from test_bcj_exact_size.c.
It's good to have it as a standalone file.
2022-07-24 11:32:15 +03:00
Lasse Collin
dbd8b0bf45 Update THANKS. 2022-07-12 19:47:34 +03:00
Lasse Collin
bb66a98ded xzgrep: Fix escaping of malicious filenames (ZDI-CAN-16587).
Malicious filenames can make xzgrep to write to arbitrary files
or (with a GNU sed extension) lead to arbitrary code execution.

xzgrep from XZ Utils versions up to and including 5.2.5 are
affected. 5.3.1alpha and 5.3.2alpha are affected as well.
This patch works for all of them.

This bug was inherited from gzip's zgrep. gzip 1.12 includes
a fix for zgrep.

The issue with the old sed script is that with multiple newlines,
the N-command will read the second line of input, then the
s-commands will be skipped because it's not the end of the
file yet, then a new sed cycle starts and the pattern space
is printed and emptied. So only the last line or two get escaped.

One way to fix this would be to read all lines into the pattern
space first. However, the included fix is even simpler: All lines
except the last line get a backslash appended at the end. To ensure
that shell command substitution doesn't eat a possible trailing
newline, a colon is appended to the filename before escaping.
The colon is later used to separate the filename from the grep
output so it is fine to add it here instead of a few lines later.

The old code also wasn't POSIX compliant as it used \n in the
replacement section of the s-command. Using \<newline> is the
POSIX compatible method.

LC_ALL=C was added to the two critical sed commands. POSIX sed
manual recommends it when using sed to manipulate pathnames
because in other locales invalid multibyte sequences might
cause issues with some sed implementations. In case of GNU sed,
these particular sed scripts wouldn't have such problems but some
other scripts could have, see:

    info '(sed)Locale Considerations'

This vulnerability was discovered by:
cleemy desu wayo working with Trend Micro Zero Day Initiative

Thanks to Jim Meyering and Paul Eggert discussing the different
ways to fix this and for coordinating the patch release schedule
with gzip.
2022-07-12 19:47:28 +03:00
Lasse Collin
fa3af4e4c6 Update THANKS. 2022-07-12 19:45:26 +03:00
Lasse Collin
f12ce0f23a liblzma: Fix docs: lzma_block_decoder() cannot return LZMA_UNSUPPORTED_CHECK.
If Check is unsupported, it will be silently ignored.
It's the caller's job to handle it.
2022-07-12 19:30:40 +03:00
Lasse Collin
4125667311 liblzma: Index hash: Change return value type of hash_append() to void. 2022-07-12 19:30:40 +03:00
Lasse Collin
7c3ce02df0 liblzma: Minor addition to lzma_vli_size() API doc.
Thanks to Jia Tan.
2022-07-12 19:30:40 +03:00
Lasse Collin
b8f667fe0c liblzma: Check the return value of lzma_index_append() in threaded encoder.
If lzma_index_append() failed (most likely memory allocation failure)
it could have gone unnoticed and the resulting .xz file would have
an incorrect Index. Decompressing such a file would produce the
correct uncompressed data but then an error would occur when
verifying the Index field.
2022-07-12 19:30:40 +03:00
Lasse Collin
2356d53edd Update THANKS. 2022-07-12 19:30:40 +03:00
Ed Maste
748ef08338 liblzma: Use non-executable stack on FreeBSD as on Linux 2022-07-12 19:30:40 +03:00
Lasse Collin
068a6e3286 liblzma: Make Block decoder catch certain types of errors better.
Now it limits the input and output buffer sizes that are
passed to a raw decoder. This way there's no need to check
if the sizes can grow too big or overflow when updating
Compressed Size and Uncompressed Size counts. This also means
that a corrupt file cannot cause the raw decoder to process
useless extra input or output that would exceed the size info
in Block Header (and thus cause LZMA_DATA_ERROR anyway).

More importantly, now the size information is verified more
carefully in case raw decoder returns LZMA_OK. This doesn't
really matter with the current single-threaded .xz decoder
as the errors would be detected slightly later anyway. But
this helps avoiding corner cases in the upcoming threaded
decompressor, and it might help other Block decoder uses
outside liblzma too.

The test files bad-1-lzma2-{9,10,11}.xz test these conditions.
With the single-threaded .xz decoder the only difference is
that LZMA_DATA_ERROR is detected in a difference place now.
2022-07-12 19:30:40 +03:00
Lasse Collin
766df4f62c Tests: Add bad-1-lzma2-11.xz. 2022-07-12 19:30:40 +03:00
Lasse Collin
12a6d6ce2a Translations: Fix po4a failure with the French man page translations.
Thanks to Mario Blättermann for the patch.
2022-07-12 19:30:40 +03:00
Lasse Collin
00e6aad836 Translations: Add French translation of man pages.
This matches xz-utils 5.2.5-2 in Debian.

The translation was done by "bubu", proofread by the debian-l10n-french
mailing list contributors, and submitted to me on the xz-devel mailing
list by Jean-Pierre Giraud. Thanks to everyone!
2022-07-12 19:30:40 +03:00
jiat75
e20ce2b122 liblzma: Add NULL checks to LZMA and LZMA2 properties encoders.
Previously lzma_lzma_props_encode() and lzma_lzma2_props_encode()
assumed that the options pointers must be non-NULL because the
with these filters the API says it must never be NULL. It is
good to do these checks anyway.
2022-07-12 19:03:51 +03:00
huangqinjin
feb80ace86 CMake: Keep compatible with Windows 95 for 32-bit build. 2022-07-12 19:03:51 +03:00
Lasse Collin
725f2e0522 xzgrep: Update man page timestamp. 2022-07-12 19:01:09 +03:00
Lasse Collin
7955669d42 Update THANKS. 2022-07-12 19:01:09 +03:00
Ville Skyttä
671673a7a2 xzgrep: use grep -E/-F instead of egrep and fgrep
`egrep` and `fgrep` have been deprecated in GNU grep since 2007, and in
current post 3.7 Git they have been made to emit obsolescence warnings:
https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1
2022-07-12 19:01:09 +03:00
Lasse Collin
45e538257e Update THANKS. 2022-07-12 19:01:09 +03:00
Lasse Collin
ca21733d24 xz: Change the coding style of the previous commit.
It isn't any better now but it's consistent with
the rest of the code base.
2022-07-12 19:01:09 +03:00
Alexander Bluhm
906b990b15 xz: Avoid fchown(2) failure.
OpenBSD does not allow to change the group of a file if the user
does not belong to this group.  In contrast to Linux, OpenBSD also
fails if the new group is the same as the old one.  Do not call
fchown(2) in this case, it would change nothing anyway.

This fixes an issue with Perl Alien::Build module.
https://github.com/PerlAlien/Alien-Build/issues/62
2022-07-12 19:01:09 +03:00
Lasse Collin
ca83df96c4 Update THANKS. 2022-07-12 19:01:09 +03:00
Lasse Collin
d8b294af03 liblzma: Use _MSVC_LANG to detect when "noexcept" can be used with MSVC.
By default, MSVC always sets __cplusplus to 199711L. The real
C++ standard version is available in _MSVC_LANG (or one could
use /Zc:__cplusplus to set __cplusplus correctly).

Fixes <https://sourceforge.net/p/lzmautils/discussion/708858/thread/f6bc3b108a/>.

Thanks to Dan Weiss.
2022-07-12 19:01:09 +03:00
Lasse Collin
c2fde22bef xzdiff: Update the man page about the exit status.
This was forgotten from 194029ffaf.
2022-07-12 19:01:09 +03:00
Lasse Collin
8d0fd42fbe xzless: Fix less(1) version detection when it contains a dot.
Sometimes the version number from "less -V" contains a dot,
sometimes not. xzless failed detect the version number when
it does contain a dot. This fixes it.

Thanks to nick87720z for reporting this. Apparently it had been
reported here <https://bugs.gentoo.org/489362> in 2013.
2022-07-12 19:01:09 +03:00
Lasse Collin
6e2cab8579 xz: Document the special memlimit case of 2000 MiB on MIPS32.
See commit 95806a8a52.
2022-07-12 18:57:21 +03:00
Lasse Collin
38b311462b Update THANKS. 2022-07-12 18:53:41 +03:00