1
0
Fork 0
mirror of https://git.tukaani.org/xz.git synced 2024-04-04 12:36:23 +02:00
xz-archive/src
Lasse Collin 3539705108 xz: Limit --memlimit-compress to at most 4020 MiB for 32-bit xz.
See the code comment for reasoning. It's far from perfect but
hopefully good enough for certain cases while hopefully doing
nothing bad in other situations.

At presets -5 ... -9, 4020 MiB vs. 4096 MiB makes no difference
on how xz scales down the number of threads.

The limit has to be a few MiB below 4096 MiB because otherwise
things like "xz --lzma2=dict=500MiB" won't scale down the dict
size enough and xz cannot allocate enough memory. With
"ulimit -v $((4096 * 1024))" on x86-64, the limit in xz had
to be no more than 4085 MiB. Some safety margin is good though.

This is hack but it should be useful when running 32-bit xz on
a 64-bit kernel that gives full 4 GiB address space to xz.
Hopefully this is enough to solve this:

https://bugzilla.redhat.com/show_bug.cgi?id=1196786

FreeBSD has a patch that limits the result in tuklib_physmem()
to SIZE_MAX on 32-bit systems. While I think it's not the way
to do it, the results on --memlimit-compress have been good. This
commit should achieve practically identical results for compression
while leaving decompression and tuklib_physmem() and thus
lzma_physmem() unaffected.
2020-02-01 19:56:18 +02:00
..
common Rename unaligned_read32ne to read32ne, and similarly for the others. 2019-12-31 00:47:49 +02:00
liblzma Rename unaligned_read32ne to read32ne, and similarly for the others. 2019-12-31 00:47:49 +02:00
lzmainfo Build: Prepare to support Automake's subdir-objects. 2014-10-29 21:15:35 +02:00
scripts Scripts: Put /usr/xpg4/bin to the beginning of PATH on Solaris. 2019-09-24 23:02:40 +03:00
xz xz: Limit --memlimit-compress to at most 4020 MiB for 32-bit xz. 2020-02-01 19:56:18 +02:00
xzdec xzdec: Fix warnings from -Wsign-conversion. 2019-06-23 21:40:47 +03:00
Makefile.am Build: Prepare to support Automake's subdir-objects. 2014-10-29 21:15:35 +02:00