1
0
Fork 0
mirror of https://git.tukaani.org/xz.git synced 2024-04-04 12:36:23 +02:00
Commit graph

1147 commits

Author SHA1 Message Date
Lasse Collin
bc04486e36 Fix fastpos problem in Makefile.am when built with --enable-small. 2008-04-24 17:33:01 +03:00
Lasse Collin
7ab493924e Use 64-bit integer as range encoder's cache size. This fixes a
theoretical data corruption, which should be very hard to trigger
even intentionally.
2008-04-24 17:30:51 +03:00
Lasse Collin
641998c3e1 Replaced the range decoder optimization that used arithmetic
right shift with as fast version that doesn't need
arithmetic right shift. Removed the related check from
configure.ac.
2008-03-24 16:38:40 +02:00
Lasse Collin
ad999efd27 Take advantage of arithmetic right shift in range decoder. 2008-03-22 14:39:34 +02:00
Lasse Collin
7521bbdc83 Update a comment to use the variable name rep_len_decoder.
(And BTW, the previous commit actually did change the
program logic slightly.)
2008-03-22 01:26:36 +02:00
Lasse Collin
63b74d000e Demystified the "state" variable in LZMA code. Use the
word literal instead of char for better consistency.
There are still some names with _char instead of _literal
in lzma_optimum, these may be changed later.

Renamed length coder variables.

This commit doesn't change the program logic.
2008-03-22 00:57:33 +02:00
Lasse Collin
e6eb0a2675 Fix data corruption in LZMA encoder. Note that this bug was
specific to liblzma and was *not* present in LZMA SDK.
2008-03-14 23:16:11 +02:00
Lasse Collin
7d516f5129 Fix a comment API header. 2008-03-14 21:32:37 +02:00
Lasse Collin
748d6e4274 Make lzma_stream.next_in const. Let's see if anyone complains. 2008-03-12 23:14:50 +02:00
Lasse Collin
bfde3b24a5 Apply a minor speed optimization to LZMA decoder. 2008-03-11 15:35:34 +02:00
Lasse Collin
f310c50286 Initialize the last byte of the dictionary to zero so that
lz_get_byte(lz, 0) returns zero. This was broken by
1a3b218598.
2008-03-11 15:17:16 +02:00
Lasse Collin
5ead36cf7f Really fix the price count initialization. 2008-03-10 15:57:55 +02:00
Lasse Collin
0541c5ea63 Initialize align_price_count and match_price_count in
lzma_encoder_init.c. While we don't call
fill_distances_prices() and fill_align_prices() in
lzma_lzma_encoder_init(), we still need to initialize
these two variables so that the fill functions get
called in lzma_encoder_getoptimum.c in the beginning
of a stream.
2008-03-10 13:46:48 +02:00
Lasse Collin
596fa1fac7 Always initialize lz->temp_size in lz_decoder.c. temp_size did
get initialized as a side-effect after allocating a new decoder,
but not when the decoder was reused.
2008-03-10 13:44:29 +02:00
Lasse Collin
45e43e1695 Don't fill allocated memory with 0xFD when debugging is
enabled. It hides errors from Valgrind.
2008-03-10 13:41:25 +02:00
Lasse Collin
c0e19e0662 Remove two redundant validity checks from the LZMA decoder.
These are already checked elsewhere, so omitting these
gives (very) tiny speed up.
2008-02-28 10:24:31 +02:00
Lasse Collin
1a3b218598 Don't memzero() the history buffer when initializing LZ
decoder. There's no danger of information leak here, so
it isn't required. Doing memzero() takes a lot of time
with large dictionaries, which could make it easier to
construct DoS attack to consume too much CPU time.
2008-02-02 14:51:06 +02:00
Lasse Collin
7e796e312b Do uncompressed size validation in raw encoder. This way
it gets done for not only raw encoder, but also Block
and LZMA_Alone encoders.
2008-02-01 08:39:26 +02:00
Lasse Collin
7dd48578a3 Avoid unneeded function call in raw_common.c. 2008-02-01 08:32:05 +02:00
Lasse Collin
e9f6e9c075 Added note.GNU-stack to x86 assembler files. It is needed
when using non-executable stack.
2008-01-26 21:40:23 +02:00
Lasse Collin
4c7ad179c7 Added api/lzma/easy.h. I had forgot to add this to the
git repo. Thanks to Stephan Kulow.
2008-01-26 19:12:50 +02:00
Lasse Collin
f9842f7127 Return LZMA_HEADER_ERROR if LZMA_SYNC_FLUSH is used with any
of the so called simple filters. If there is demand, limited
support for LZMA_SYNC_FLUSH may be added in future.

After this commit, using LZMA_SYNC_FLUSH shouldn't cause
undefined behavior in any situation.
2008-01-26 00:25:34 +02:00
Lasse Collin
4441e00418 Combine lzma_options_block validation needed by both Block
encoder and decoder, and put the shared things to
block_private.h. Improved the checks a little so that
they may detect too big Compressed Size at initialization
time if lzma_options_block.total_size or .total_limit is
known.

Allow encoding and decoding Blocks with combinations of
fields that are not allowed by the file format specification.
Doing this requires that the application passes such a
combination in lzma_options_lzma; liblzma doesn't do that,
but it's not impossible that someone could find them useful
in some custom file format.
2008-01-25 23:12:36 +02:00
Lasse Collin
7b8fc7e6b5 Improved the memory limitter:
- Added lzma_memlimit_max() and lzma_memlimit_reached()
    API functions.
  - Added simple estimation of malloc()'s memory usage
    overhead.
  - Fixed integer overflow detection in lzma_memlimit_alloc().
  - Made some white space cleanups and added more comments.

The description of lzma_memlimit_max() in memlimit.h is bad
and should be improved.
2008-01-25 19:20:28 +02:00
Lasse Collin
db9df0a960 Fix decoding of empty Metadata Blocks, that don't have
even the Metadata Flags field. Earlier the code allowed
such files; now they are prohibited as the file format
specification requires.
2008-01-23 23:43:00 +02:00
Lasse Collin
765f0b05f6 Fix a bug related to 99e12af4e2.
lzma_metadata.header_metadata_size was not properly set to
zero if the Metadata had only the Metadata Flags field.
2008-01-23 23:38:18 +02:00
Lasse Collin
3a7cc5c3de Fix decoding of Extra Records that have empty Data. 2008-01-23 23:35:49 +02:00
Lasse Collin
e5fdec93e2 Add the trailing '\0' to lzma_extra.data as the API header
already documents.
2008-01-23 22:02:38 +02:00
Lasse Collin
ae0cd09a66 Return LZMA_STREAM_END instead of LZMA_OK if
LZMA_SYNC_FLUSH or LZMA_FULL_FLUSH is used when
there's no unfinished Block open.
2008-01-23 21:05:33 +02:00
Lasse Collin
8c8eb14055 Fixed a typo. 2008-01-23 13:42:35 +02:00
Lasse Collin
980f65a9a1 Fix a memory leak in the Subblock encoder. 2008-01-23 13:40:45 +02:00
Lasse Collin
99e12af4e2 Fix Size of Header Metadata Block handling. Now
lzma_metadata.header_metadata_size == LZMA_VLI_VALUE_UNKNOWN
is not allowed at all. To indicate missing Header Metadata
Block, header_metadata_size must be set to zero. This is
what Metadata decoder does after this patch too.

Note that other missing fields in lzma_metadata are still
indicated with LZMA_VLI_VALUE_UNKNOWN. This isn't as
illogical as it sounds at first, because missing Size of
Header Metadata Block means that Header Metadata Block is
not present in the Stream. With other Metadata fields,
a missing field means only that the value is unknown.
2008-01-23 13:36:07 +02:00
Lasse Collin
58b78ab20c Fix a memory leak in metadata_decoder.c. 2008-01-23 13:15:55 +02:00
Lasse Collin
4d8cdbdab4 Fix the fix 863028cb7a which
just moved to problem. Now it's really fixed.
2008-01-23 13:13:58 +02:00
Lasse Collin
67321de963 Take advantage of return_if_error() macro in
lzma_info_metadata_set() in info.c.
2008-01-23 00:21:04 +02:00
Lasse Collin
863028cb7a Fixed a dangling pointer that caused invalid free(). 2008-01-23 00:18:32 +02:00
Lasse Collin
cf49f42a6b Added lzma_easy_* functions. These should make using
liblzma as easy as using zlib, because the easy API
don't require developers to know any fancy LZMA options.

Note that Multi-Block Stream encoding is currently broken.
The easy API should be OK, the bug(s) are elsewhere.
2008-01-22 22:49:24 +02:00
Lasse Collin
1747b85a43 Fix Multi-Block Stream encoder's EOPM usage. 2008-01-22 21:16:22 +02:00
Lasse Collin
0ed6f1adce Made lzma_extra pointers const in lzma_options_stream. 2008-01-22 00:15:11 +02:00
Lasse Collin
107259e306 Fix alignment handling bugs in Subblock encoder.
This leaves one known alignment bug unfixed: If repeat count
doesn't fit into 28-bit integer, the encoder has to split
this to multiple Subblocks with Subblock Type `Repeating Data'.
The extra Subblocks may have wrong alignment. Correct alignment
is restored after the split Repeating Data has been completely
written out.

Since the encoder doesn't even try to fix the alignment unless
the size of Data is at least 4 bytes, to trigger this bug you
need at least 4 GiB of repeating data with sequence length of
4 or more bytes. Since the worst thing done by this bug is
misaligned data (no data corruption), this bug simply isn't
worth fixing, because a proper fix isn't simple.
2008-01-20 20:12:58 +02:00
Lasse Collin
e141fe1895 Implemented LZMA_SYNC_FLUSH support to the Subblock encoder.
The API for handing Subfilters was changed to make it
consistent with LZMA_SYNC_FLUSH.

A few sanity checks were added for Subfilter handling. Some
small bugs were fixed. More comments were added.
2008-01-19 21:16:33 +02:00
Lasse Collin
23c227a864 Revised the Delta filter implementation. The initialization
function is still shared between encoder and decoder, but the
actual coding is in separate files for encoder and decoder.

There are now separate functions for the actual delta
calculation depending on if Delta is the last filter in the
chain or not. If it is the last, the new code copies the
data from input to output buffer and does the delta
calculation at the same time. The old code first copied the
data, then did the delta in the target buffer, which required
reading through the data twice.

Support for LZMA_SYNC_FLUSH was added to the Delta encoder.
This doesn't change anything in the file format.
2008-01-19 15:19:21 +02:00
Lasse Collin
ab5feaf1fc Fix LZMA_SYNC_FLUSH handling in LZ and LZMA encoders.
That code is now almost completely in LZ coder, where
it can be shared with other LZ77-based algorithms in
future.
2008-01-18 20:02:52 +02:00
Lasse Collin
61d1784d8f Set stdin and stdout to binary mode on Windows. This patch is
a forward port of b7b22fcb979a16d3a47c8001f058c9f7d4416068
from lzma-utils-legacy.git. I don't know if the new code base
builds on Windows, but this is a start.
2008-01-18 14:17:37 +02:00
Lasse Collin
33be3c0e24 Subblock decoder: Don't exit the main loop in decode_buffer()
too early if we hit End of Input while decoding a Subblock of
type Repeating Data. To keep the loop termination condition
elegant, the order of enumerations in coder->sequence were
changed.

To keep the case-labels in roughly the same order as the
enumerations in coder->sequence, large chunks of code was
moved around. This made the diff big and ugly compared to
the amount of the actual changes made.
2008-01-17 18:56:53 +02:00
Lasse Collin
b254bd97b1 Fix wrong too small size of argument unfiltered_max
in ia64_coder_init(). It triggered assert() in
simple_coder.c, and could have caused a buffer overflow.

This error was probably a copypaste mistake, since most
of the simple filters use unfiltered_max = 4.
2008-01-17 17:39:42 +02:00
Lasse Collin
8f5794c8f1 Added --delta to the output of "lzma --help". 2008-01-17 17:27:45 +02:00
Lasse Collin
f88590e001 Fix Subblock docoder: If Subblock filter was used with known
Uncompressed Size, and the last output byte was from RLE,
the code didn't stop decoding as it should have done.
2008-01-17 13:14:20 +02:00
Lasse Collin
7599bb7064 Plugged a memory leak in stream_decoder.c. 2008-01-16 14:48:04 +02:00
Lasse Collin
0b58153931 Added memory leak detection to lzmadec.c. 2008-01-16 14:47:27 +02:00
Lasse Collin
5b5b13c7bb Added lzma_memlimit_count(). 2008-01-16 14:46:50 +02:00
Lasse Collin
19389f2b82 Added ARRAY_SIZE(array) macro. 2008-01-16 14:31:44 +02:00
Lasse Collin
9bc33a54cb Make Uncompresed Size validation more strict
in alone_decoder.c.
2008-01-16 13:27:03 +02:00
Lasse Collin
01d71d60b7 Free the allocated memory in lzmadec if debugging is
enabled. This should make it possible to detect possible
memory leaks with Valgrind.
2008-01-15 17:46:59 +02:00
Lasse Collin
f10fc6a69d Use fastpos.h when encoding LZMA dictionary size in
Filter Flags encoder.
2008-01-15 14:23:35 +02:00
Lasse Collin
e5728142a2 Revised the fastpos code. It now uses the slightly faster
table-based version from LZMA SDK 4.57. This should be
fast on most systems.

A simpler and smaller alternative version is also provided.
On some CPUs this can be even a little faster than the
default table-based version (see comments in fastpos.h),
but on most systems the table-based code is faster.
2008-01-15 14:02:22 +02:00
Lasse Collin
10437b5b56 Added bsr.h. 2008-01-15 13:32:13 +02:00
Lasse Collin
54ec204f58 Omit invalid space from printf() format string
in price_table_gen.c.
2008-01-15 12:20:41 +02:00
Lasse Collin
01b4b19f49 Removed a few unused macros from lzma_common.h. 2008-01-15 09:54:34 +02:00
Lasse Collin
19bd7f3cf2 Fix a typo in lzma_encoder.c. 2008-01-15 08:37:42 +02:00
Lasse Collin
9f9b198301 Convert bittree_get_price() and bittree_reverse_get_price()
from macros to inline functions.
2008-01-15 08:36:25 +02:00
Lasse Collin
78e85cb1a7 Fix CRC code in case --enable-small is used. 2008-01-15 07:44:59 +02:00
Lasse Collin
d13d693155 Added precomputed range coder probability price table. 2008-01-15 07:40:21 +02:00
Lasse Collin
362dc3843b Remove RC_BUFFER_SIZE from lzma_encoder_private.h
and replace it with a sanity check.
2008-01-14 13:42:43 +02:00
Lasse Collin
e22b37968d Major changes to LZ encoder, LZMA encoder, and range encoder.
These changes implement support for LZMA_SYNC_FLUSH in LZMA
encoder, and move the temporary buffer needed by range encoder
from lzma_range_encoder structure to lzma_lz_encoder.
2008-01-14 13:39:54 +02:00
Lasse Collin
b59ef39737 Added one assert() to process.c of the command line tool. 2008-01-14 13:34:29 +02:00
Lasse Collin
9547e734a0 Don't use coder->lz.stream_end_was_reached in assertions
in match_c.h.
2008-01-14 12:09:52 +02:00
Lasse Collin
3e09e1c058 In lzma_read_match_distances(), don't use
coder->lz.stream_end_was_reached. That variable
will be removed, and the check isn't required anyway.
Rearrange the check so that it doesn't make one to
think that there could be an integer overflow.
2008-01-14 12:08:02 +02:00
Lasse Collin
a670fec802 Small LZMA_SYNC_FLUSH fixes to Block and Single-Stream encoders. 2008-01-14 11:56:41 +02:00
Lasse Collin
3599dba957 More fixes to LZMA decoder's flush marker handling. 2008-01-14 11:54:56 +02:00
Lasse Collin
f73c2ab607 Eliminate lzma_lz_encoder.must_move_pos. It's needed
only in one place which isn't performance criticial.
2008-01-10 17:13:42 +02:00
Lasse Collin
379fbbe84d Take advantage of return_if_error() in block_decoder.c. 2008-01-08 23:11:59 +02:00
Lasse Collin
7054c5f588 Fix decoding of Blocks that have only Block Header. 2008-01-08 22:58:42 +02:00
Lasse Collin
faeac7b7ac Disable CRC32 from Block Headers when --check=none
has been specified.
2008-01-08 18:50:30 +02:00
Lasse Collin
a751126dbb Fixed encoding of empty files. Arguments to is_size_valid()
were in wrong order in block_encoder.c.
2008-01-08 13:36:29 +02:00
Lasse Collin
e2417b2b91 More pre-C99 inttypes.h compatibility fixes. Now the code
should work even if the system has no inttypes.h.
2008-01-08 00:48:30 +02:00
Lasse Collin
c15a7abf66 With printf(), use PRIu64 with a cast to uint64_t instead
of %zu, because some pre-C99 libc versions don't support %zu.
2008-01-06 19:45:27 +02:00
Lasse Collin
4e7e54c4c5 Introduced compatibility with systems that have pre-C99
or no inttypes.h. This is useful when the compiler has
good enough support for C99, but libc headers don't.

Changed liblzma API so that sys/types.h and inttypes.h
have to be #included before #including lzma.h. On systems
that don't have C99 inttypes.h, it's the problem of the
applications to provide the required types and macros
before #including lzma.h.

If lzma.h defined the missing types and macros, it could
conflict with third-party applications whose configure
has detected that the types are missing and defined them
in config.h already. An alternative would have been
introducing lzma_uint32 and similar types, but that would
just be an extra pain on modern systems.
2008-01-06 16:27:41 +02:00
Lasse Collin
a71864f77d Fix typo in comment (INT64_MAX -> UINT64_MAX). 2008-01-05 19:57:00 +02:00
Lasse Collin
d160ee3259 Another bug fix for flush marker detection. 2008-01-05 01:20:24 +02:00
Lasse Collin
fc67f79f60 Fix stupid bugs in flush marker detection. 2008-01-04 21:37:01 +02:00
Lasse Collin
0029cbbabe Added support for flush marker, which will be in files
that use LZMA_SYNC_FLUSH with encoder (not implemented
yet). This is a new feature in the raw LZMA format,
which isn't supported by old decoders. This shouldn't
be a problem in practice, since lzma_alone_encoder()
will not allow LZMA_SYNC_FLUSH, and thus not allow
creating files on decodable with old decoders.

Made lzma_decoder.c to require tab width of 4 characters
if one wants to fit the code in 80 columns. This makes
the code easier to read.
2008-01-04 21:30:33 +02:00
Lasse Collin
bbfd1f6ab0 Moved range decoder initialization (reading the first
five input bytes) from LZMA decoder to range decoder
header. Did the same for decoding of direct bits.
2008-01-04 20:45:05 +02:00
Lasse Collin
44b333d461 Use the filename suffix .S instead of .s for assembler files
so that the preprocessor removes the /* */ style comments,
which are not supported by some non-GNU assemblers (Solaris)
that otherwise work with this code.
2007-12-14 10:07:10 +02:00
Lasse Collin
ec1c82b2e8 Fixed wrong symbol name in crc64_x86.s. 2007-12-14 09:59:05 +02:00
Lasse Collin
2881570df6 Use .globl instead of .global in x86 assembler code for
better portability. Still needs fixing the commenting.
2007-12-14 09:53:24 +02:00
Lasse Collin
698470b8f3 Fixed a few short options that take an argument.
short_opts[] was missing colons to indicate
required argument. Thanks to Fabio Pedretti for
the bug report.
2007-12-13 20:14:37 +02:00
Lasse Collin
918bcb0e07 Removed uncompressed size tracking from Delta encoder too. 2007-12-11 17:08:04 +02:00
Lasse Collin
3e16d51dd6 Remove uncompressed size tracking from the filter encoders.
It's not strictly needed there, and just complicates the
code. LZ encoder never even had this feature.

The primary reason to have uncompressed size tracking in
filter encoders was validating that the application
doesn't give different amount of input that it had
promised. A side effect was to validate internal workings
of liblzma.

Uncompressed size tracking is still present in the Block
encoder. Maybe it should be added to LZMA_Alone and raw
encoders too. It's simpler to have one coder just to
validate the uncompressed size instead of having it
in every filter.
2007-12-11 16:49:19 +02:00
Lasse Collin
5286723e0d Get rid of no-NLS gnulib. I don't know how to get it
working with Automake. People who want smaller lzmadec
should use --disable-nls on non-GNU systems.
2007-12-11 14:10:53 +02:00
Lasse Collin
7c1ad41eb6 Fixed wrong type of flags_size in Subblock encoder. 2007-12-11 11:18:58 +02:00
Lasse Collin
ce64df7162 Bumped version number to 4.42.3alpha. 2007-12-10 20:44:16 +02:00
Lasse Collin
2ab8adb516 Added LZMA_SYNC_FLUSH support to the Copy filter. 2007-12-09 21:43:15 +02:00
Lasse Collin
329c272d50 Added missing LZMA_API to the C versions of the CRC functions.
The x86 assembler versions were already OK.
2007-12-09 17:14:07 +02:00
Lasse Collin
07ac881779 Take advantage of return_if_error() macro in more places.
Cleaned Subblock filter's initialization code too.
2007-12-09 17:06:45 +02:00
Lasse Collin
ff946ceb79 Re-enabled the security checks in Subblock decoder
that were disabled for debugging reasons.
2007-12-09 11:24:48 +02:00
Lasse Collin
5d018dc035 Imported to git. 2007-12-09 00:42:33 +02:00