commit 238b4e5458b4bd2cadefb768b8ea7c6b70a191ac Author: Jia Tan Date: 2023-05-04 20:38:52 +0800 Translations: Update the Croatian translation. po/hr.po | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit 0cee63c3c6c4f9084a5e7fa7c15a1863ac8e5ffc Author: Jia Tan Date: 2023-05-04 19:50:42 +0800 Bump version and soname for 5.4.3. src/liblzma/Makefile.am | 2 +- src/liblzma/api/lzma/version.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit 01f937ea9a49005c5091c0165fa0c1e572dd1749 Author: Jia Tan Date: 2023-05-02 20:39:56 +0800 Add NEWS for 5.4.3. NEWS | 10 ++++++++++ 1 file changed, 10 insertions(+) commit e9b9ea953179502d5245b4e6ca4f5ffe9d5797b1 Author: Lasse Collin Date: 2023-05-03 22:46:42 +0300 tuklib_integer.h: Fix a recent copypaste error in Clang detection. Wrong line was changed in 7062348bf35c1e4cbfee00ad9fffb4a21aa6eff7. Also, this has >= instead of == since ints larger than 32 bits would work too even if not relevant in practice. src/common/tuklib_integer.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit be6e39a8622f0d905072874ec25b1ffd155b12e1 Author: Jia Tan Date: 2023-04-20 20:15:00 +0800 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 9e343a46cf87a345799222c0b0b3a6f3358dde0c Author: Jia Tan Date: 2023-04-19 22:22:16 +0800 Windows: Include when needed. Legacy Windows did not need to #include to use the MSVC intrinsics. Newer versions likely just issue a warning, but the MSVC documentation says to include the header file for the intrinsics we use. GCC and Clang can "pretend" to be MSVC on Windows, so extra checks are needed in tuklib_integer.h to only include when it will is actually needed. src/common/tuklib_integer.h | 6 ++++++ src/liblzma/common/memcmplen.h | 10 ++++++++++ 2 files changed, 16 insertions(+) commit 12321a9390acc076b414035a46df9d7545ac379f Author: Jia Tan Date: 2023-04-19 21:59:03 +0800 tuklib_integer: Use __builtin_clz() with Clang. Clang has support for __builtin_clz(), but previously Clang would fallback to either the MSVC intrinsic or the regular C code. This was discovered due to a bug where a new version of Clang required the header file in order to use the MSVC intrinsics. Thanks to Anton Kochkov for notifying us about the bug. src/common/tuklib_integer.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit d1f0e01c395960efd6c29ff144eda4f4c8c1c6d3 Author: Lasse Collin Date: 2023-04-14 18:42:33 +0300 liblzma: Update project maintainers in lzma.h. AUTHORS was updated earlier, lzma.h was simply forgotten. src/liblzma/api/lzma.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 8204c5d1304e1e3b7487801b4acaf8e3179a6c52 Author: Jia Tan Date: 2023-04-13 20:45:19 +0800 liblzma: Cleans up old commented out code. src/liblzma/common/alone_encoder.c | 11 ----------- 1 file changed, 11 deletions(-) commit 32980d1562bc031013306be1c787761fa63da87d Author: Jia Tan Date: 2023-03-28 22:32:40 +0800 CMake: Update liblzma-config.cmake generation. Now that the threading is configurable, the liblzma CMake package only needs the threading library when using POSIX threads. CMakeLists.txt | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-) commit 023907faa965476fb921a6344253c50e0ba795f9 Author: Jia Tan Date: 2023-03-28 22:25:33 +0800 CMake: Allows setting thread method. The thread method is now configurable for the CMake build. It matches the Autotools build by allowing ON (pick the best threading method), OFF (no threading), posix, win95, and vista. If both Windows and posix threading are both available, then ON will choose Windows threading. Windows threading will also not use: target_link_libraries(liblzma Threads::Threads) since on systems like MinGW-w64 it would link the posix threads without purpose. CMakeLists.txt | 144 +++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 104 insertions(+), 40 deletions(-) commit ba176d77cbb164b13006516660fc1ff6c59dedb5 Author: Jia Tan Date: 2023-03-24 20:05:59 +0800 CMake: Only build xzdec if decoders are enabled. CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit c99d697df800e9bc47e09facf88bcfdb9e0264a5 Author: Jia Tan Date: 2023-03-22 15:42:04 +0800 Build: Removes redundant check for LZMA1 filter support. src/liblzma/lzma/Makefile.inc | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) commit 54a2fd8c850e626343b79c4cc8fcac2f638ec890 Author: Lasse Collin Date: 2023-03-23 15:14:29 +0200 CMake: Bump maximum policy version to 3.26. It adds only one new policy related to FOLDERS which we don't use. This makes it clear that the code is compatible with the policies up to 3.26. CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 622d0fdc02d5f565c2872d79bc70c669c95c8284 Author: Jia Tan Date: 2023-03-21 23:36:00 +0800 CMake: Conditionally build xz list.* files if decoders are enabled. CMakeLists.txt | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) commit 3e2dc523c596cd770a6a7c58cacc0d0d8761e853 Author: Jia Tan Date: 2023-02-25 11:46:50 +0800 CMake: Allow configuring features as cache variables. This allows users to change the features they build either in CMakeCache.txt or by using a CMake GUI. The sources built for liblzma are affected by this too, so only the necessary files will be compiled. CMakeLists.txt | 528 ++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 391 insertions(+), 137 deletions(-) commit 0c053f97337fa4ead77acefb577e8e86f8ef388d Author: Lasse Collin Date: 2023-03-21 14:07:51 +0200 Build: Add a comment that AC_PROG_CC_C99 is needed for Autoconf 2.69. It's obsolete in Autoconf >= 2.70 and just an alias for AC_PROG_CC but Autoconf 2.69 requires AC_PROG_CC_C99 to get a C99 compiler. configure.ac | 3 +++ 1 file changed, 3 insertions(+) commit 915d4f6058d52f84d2b58b0a5f3c8272eaa6c1bd Author: Lasse Collin Date: 2023-03-21 14:04:37 +0200 Build: configure.ac: Use AS_IF and AS_CASE where required. This makes no functional difference in the generated configure (at least with the Autotools versions I have installed) but this change might prevent future bugs like the one that was just fixed in the commit 5a5bd7f871818029d5ccbe189f087f591258c294. configure.ac | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) commit b848c039586ed2cddb6fb033680dac107cc5143f Author: Lasse Collin Date: 2023-03-21 13:12:03 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit c775ba1602a74f29dbc2088bbe4d02c23fc32ba1 Author: Lasse Collin Date: 2023-03-21 13:11:49 +0200 Build: Fix --disable-threads breaking the building of shared libs. This is broken in the releases 5.2.6 to 5.4.2. A workaround for these releases is to pass EGREP='grep -E' as an argument to configure in addition to --disable-threads. The problem appeared when m4/ax_pthread.m4 was updated in the commit 6629ed929cc7d45a11e385f357ab58ec15e7e4ad which introduced the use of AC_EGREP_CPP. AC_EGREP_CPP calls AC_REQUIRE([AC_PROG_EGREP]) to set the shell variable EGREP but this was only executed if POSIX threads were enabled. Libtool code also has AC_REQUIRE([AC_PROG_EGREP]) but Autoconf omits it as AC_PROG_EGREP has already been required earlier. Thus, if not using POSIX threads, the shell variable EGREP would be undefined in the Libtool code in configure. ax_pthread.m4 is fine. The bug was in configure.ac which called AX_PTHREAD conditionally in an incorrect way. Using AS_CASE ensures that all AC_REQUIREs get always run. Thanks to Frank Busse for reporting the bug. Fixes: https://github.com/tukaani-project/xz/issues/45 configure.ac | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) commit 0673c9ec98b6bae12b33dc295564514aaa26e2fc Author: Lasse Collin Date: 2023-03-19 22:45:59 +0200 liblzma: Silence -Wsign-conversion in SSE2 code in memcmplen.h. Thanks to Christian Hesse for reporting the issue. Fixes: https://github.com/tukaani-project/xz/issues/44 src/liblzma/common/memcmplen.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 6ca8046ecbc7a1c81ee08f544bfd1414819fb2e8 Author: Jia Tan Date: 2023-03-18 23:22:06 +0800 Bump version and soname for 5.4.2. src/liblzma/Makefile.am | 2 +- src/liblzma/api/lzma/version.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit ce4f246600db10e77fc93d492ea045b4c9830bc6 Author: Jia Tan Date: 2023-03-18 22:10:57 +0800 Add NEWS for 5.4.2. NEWS | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) commit 3634fe330788fc0df217acdb6581031a851600a3 Author: Lasse Collin Date: 2023-03-18 16:00:54 +0200 Update the copy of GNU GPLv3 from gnu.org to COPYING.GPLv3. COPYING.GPLv3 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) commit 97679d25ce7cb014328f1455bb338903c003d54f Author: Lasse Collin Date: 2023-03-18 15:51:57 +0200 Change a few HTTP URLs to HTTPS. The xz man page timestamp was intentionally left unchanged. INSTALL | 2 +- README | 8 ++++---- configure.ac | 2 +- dos/INSTALL.txt | 4 ++-- src/liblzma/api/lzma.h | 8 ++++---- src/liblzma/check/sha256.c | 2 +- src/xz/xz.1 | 2 +- windows/INSTALL-MinGW.txt | 10 +++++----- 8 files changed, 19 insertions(+), 19 deletions(-) commit 01a67e89a56c3ebf5f6681bd06f0edc22a1ae594 Author: Jia Tan Date: 2023-03-18 00:40:28 +0800 CMake: Fix typo in a comment. CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 5dca3f5cbab31105196c89cd6df0a9bbc3307c05 Author: Lasse Collin Date: 2023-03-17 18:36:22 +0200 Windows: build.bash: Copy liblzma API docs to the output package. windows/build.bash | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit ae252862b30b509ab88b2bbcaa08e8d51b57e928 Author: Lasse Collin Date: 2023-03-17 08:53:38 +0200 Windows: Add microlzma_*.c to the VS project files. These should have been included in 5.3.2alpha already. windows/vs2013/liblzma.vcxproj | 2 ++ windows/vs2013/liblzma_dll.vcxproj | 2 ++ windows/vs2017/liblzma.vcxproj | 2 ++ windows/vs2017/liblzma_dll.vcxproj | 2 ++ windows/vs2019/liblzma.vcxproj | 2 ++ windows/vs2019/liblzma_dll.vcxproj | 2 ++ 6 files changed, 12 insertions(+) commit 147d282cc3733e2723df82622b6540ddfb52635e Author: Lasse Collin Date: 2023-03-17 08:43:51 +0200 CMake: Add microlzma_*.c to the build. These should have been included in 5.3.2alpha already. CMakeLists.txt | 2 ++ 1 file changed, 2 insertions(+) commit 4523a5ee29f45c0256af67a15771bc8bbd25ba53 Author: Lasse Collin Date: 2023-03-17 08:41:36 +0200 Build: Update comments about unaligned access to mention 64-bit. cmake/tuklib_integer.cmake | 7 +++---- m4/tuklib_integer.m4 | 4 ++-- 2 files changed, 5 insertions(+), 6 deletions(-) commit 82aacb40367dc580b09c5a7b270f6c98b63e49b9 Author: Lasse Collin Date: 2023-03-17 00:02:30 +0200 Tests: Update .gitignore. .gitignore | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 5d022c24921eec938604a8cb10d70aa08dbd8575 Author: Lasse Collin Date: 2023-03-14 20:04:03 +0200 po4a/update-po: Display the script name consistently in error messages. po4a/update-po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 896295117324f323c0b8db6a31ad6ebfaa88793d Author: Jia Tan Date: 2023-03-17 01:30:36 +0800 Doc: Rename Doxygen HTML doc directory name liblzma => api. When the docs are installed, calling the directory "liblzma" is confusing since multiple other files in the doc directory are for liblzma. This should also make it more natural for distros when they package the documentation. .gitignore | 2 +- Makefile.am | 18 +++++++++--------- PACKAGERS | 4 ++-- doxygen/Doxyfile | 2 +- doxygen/update-doxygen | 18 +++++++++--------- 5 files changed, 22 insertions(+), 22 deletions(-) commit 94097157fae308b2c1a5edb4e8597c68b45eeaea Author: Jia Tan Date: 2023-03-16 22:07:15 +0800 liblzma: Remove note from lzma_options_bcj about the ARM64 exception. This was left in by mistake since an early version of the ARM64 filter used a different struct for its options. src/liblzma/api/lzma/bcj.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit da16d0b73b79d7785ece6f78a577fadd1fb59d0e Author: Lasse Collin Date: 2023-03-15 19:19:13 +0200 COPYING: Add a note about the included Doxygen-generated HTML. COPYING | 11 +++++++++++ 1 file changed, 11 insertions(+) commit e57c74f9ef19201f72c106f2c347596f40229936 Author: Jia Tan Date: 2023-03-16 21:41:09 +0800 Doc: Update PACKAGERS with details about liblzma API docs install. PACKAGERS | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) commit 7e2fa48bb73edb25457232e2e62a4f81c6b95281 Author: Jia Tan Date: 2023-03-16 21:38:32 +0800 liblzma: Add set lzma.h as the main page for Doxygen documentation. The \mainpage command is used in the first block of comments in lzma.h. This changes the previously nearly empty index.html to use the first comment block in lzma.h for its contents. lzma.h is no longer documented separately, but this is for the better since lzma.h only defined a few macros that users do not need to use. The individual API header files all have a disclaimer that they should not be #included directly, so there should be no confusion on the fact that lzma.h should be the only header used by applications. Additionally, the note "See ../lzma.h for information about liblzma as a whole." was removed since lzma.h is now the main page of the generated HTML and does not have its own page anymore. So it would be confusing in the HTML version and was only a "nice to have" when browsing the source files. src/liblzma/api/lzma.h | 1 + src/liblzma/api/lzma/base.h | 2 -- src/liblzma/api/lzma/bcj.h | 2 -- src/liblzma/api/lzma/block.h | 2 -- src/liblzma/api/lzma/check.h | 2 -- src/liblzma/api/lzma/container.h | 2 -- src/liblzma/api/lzma/delta.h | 2 -- src/liblzma/api/lzma/filter.h | 2 -- src/liblzma/api/lzma/hardware.h | 2 -- src/liblzma/api/lzma/index.h | 2 -- src/liblzma/api/lzma/index_hash.h | 4 +--- src/liblzma/api/lzma/lzma12.h | 2 -- src/liblzma/api/lzma/stream_flags.h | 2 -- src/liblzma/api/lzma/version.h | 2 -- src/liblzma/api/lzma/vli.h | 2 -- 15 files changed, 2 insertions(+), 29 deletions(-) commit d42977c7acfdf7fd9acc8803cf411eca7dc10478 Author: Jia Tan Date: 2023-03-16 21:37:32 +0800 Build: Generate doxygen documentation in autogen.sh. Another command line option (--no-doxygen) was added to disable creating the doxygen documenation in cases where it not wanted or if the doxygen tool is not installed. autogen.sh | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-) commit 8fc712fcf41d821069c670f22b8bf628e7a4a877 Author: Jia Tan Date: 2023-03-16 21:35:55 +0800 Build: Create doxygen/update-doxygen script. This is a helper script to generate the Doxygen documentation. It can be run in 'liblzma' or 'internal' mode by setting the first argument. It will default to 'liblzma' mode and only generate documentation for the liblzma API header files. The helper script will be run during the custom mydist hook when we create releases. This hook already alters the source directory, so its fine to do it here too. This way, we can include the Doxygen generated files in the distrubtion and when installing. In 'liblzma' mode, the JavaScript is stripped from the .html files and the .js files are removed. This avoids license hassle from jQuery and other libraries that Doxygen 1.9.6 puts into jquery.js in minified form. Makefile.am | 1 + doxygen/update-doxygen | 111 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) commit 77d2c025d19fbac7277c63239cfb1338f02c66c0 Author: Jia Tan Date: 2023-03-16 21:34:36 +0800 Build: Install Doxygen docs and include in distribution if generated. Added a install-data-local target to install the Doxygen documentation only when it has been generated. In order to correctly remove the docs, a corresponding uninstall-local target was added. If the doxygen docs exist in the source tree, they will also be included in the distribution now too. Makefile.am | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) commit 9af8f6f947a43974a4ff025f4c896f2fb4c91b4f Author: Jia Tan Date: 2023-01-03 20:37:30 +0800 Doxygen: Refactor Doxyfile.in to doxygen/Doxyfile. Instead of having Doxyfile.in configured by Autoconf, the Doxyfile can have the tags that need to be configured piped into the doxygen command through stdin with the overrides after Doxyfile's contents. Going forward, the documentation should be generated in two different modes: liblzma or internal. liblzma is useful for most users. It is the documentation for just the liblzma API header files. This is the default. internal is for people who want to understand how xz and liblzma work. It might be useful for people who want to contribute to the project. .gitignore | 3 +- Doxyfile.in | 1234 ------------------------- Makefile.am | 1 - configure.ac | 1 - doxygen/Doxyfile | 2684 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 2686 insertions(+), 1237 deletions(-) commit 1c558a26928b753fcf1c0d4bce9c0643285edd86 Author: Jia Tan Date: 2023-02-28 23:22:36 +0800 Tests: Remove unused macros and functions. tests/tests.h | 75 ----------------------------------------------------------- 1 file changed, 75 deletions(-) commit 7479a69a45a1b8fdb9a209e11f247dce11ac1ba0 Author: Jia Tan Date: 2023-01-12 22:29:07 +0800 Tests: Refactors existing lzma_index tests. Converts the existing lzma_index tests into tuktests and covers every API function from index.h except for lzma_file_info_decoder, which can be tested in the future. tests/test_index.c | 2036 ++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 1492 insertions(+), 544 deletions(-) commit fd56d5353360279c10e8e5e05f5bc6ea03f64584 Author: Lasse Collin Date: 2023-03-07 19:59:23 +0200 xz: Make Capsicum sandbox more strict with stdin and stdout. src/xz/file_io.c | 8 ++++++++ 1 file changed, 8 insertions(+) commit d1bdaaebc68cae7f0ba457fa990b520df2186fd1 Author: Lasse Collin Date: 2023-03-11 19:31:40 +0200 xz: Don't fail if Capsicum is enabled but kernel doesn't support it. (This commit combines related commits from the master branch.) If Capsicum support is missing from the kernel or xz is being run in an emulator that lacks Capsicum suport, the syscalls will fail and set errno to ENOSYS. Previously xz would display and error and exit, making xz unusable. Now it will check for ENOSYS and run without sandbox support. Other tools like ssh behave similarly. Displaying a warning for missing Capsicum support was considered but such extra output would quickly become annoying. It would also break test_scripts.sh in "make check". Also move cap_enter() to be the first step instead of the last one. This matches the example in the cap_rights_limit(2) man page. With the current code it shouldn't make any practical difference though. Thanks to Xin Li for the bug report, suggesting a fix, and testing: https://github.com/tukaani-project/xz/pull/43 Thanks to Jia Tan for most of the original commits. src/xz/file_io.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) commit 5d351c69c19d212ddd2cf1f3bdb24900820c6776 Author: Jia Tan Date: 2023-02-04 21:06:35 +0800 Build: Adjust CMake version search regex. Now, the LZMA_VERSION_MAJOR, LZMA_VERSION_MINOR, and LZMA_VERSION_PATCH macros do not need to be on consecutive lines in version.h. They can be separated by more whitespace, comments, or even other content, as long as they appear in the proper order (major, minor, patch). CMakeLists.txt | 2 ++ 1 file changed, 2 insertions(+) commit b82d4831e3f2339c4cdbc47776c99462088c11b7 Author: Jia Tan Date: 2023-01-26 09:50:21 +0800 liblzma: Improve documentation for version.h. Specified parameter and return values for API functions and documented a few more of the macros. src/liblzma/api/lzma/version.h | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) commit 2caba3efe3e0c2d7c6dca00e22c56812326a34e6 Author: Jia Tan Date: 2023-02-24 23:46:23 +0800 liblzma: Clarify lzma_lzma_preset() documentation in lzma12.h. lzma_lzma_preset() does not guarentee that the lzma_options_lzma are usable in an encoder even if it returns false (success). If liblzma is built with default configurations, then the options will always be usable. However if the match finders hc3, hc4, or bt4 are disabled, then the options may not be usable depending on the preset level requested. The documentation was updated to reflect this complexity, since this behavior was unclear before. src/liblzma/api/lzma/lzma12.h | 5 +++++ 1 file changed, 5 insertions(+) commit 594f904673ba55a833adfa60bbab6b60d4902d08 Author: Lasse Collin Date: 2023-02-27 18:38:35 +0200 CMake: Require that the C compiler supports C99 or a newer standard. Thanks to autoantwort for reporting the issue and suggesting a different patch: https://github.com/tukaani-project/xz/pull/42 CMakeLists.txt | 8 ++++++++ 1 file changed, 8 insertions(+) commit 88101143cb774bca6b7006c8335b09dc3f334140 Author: Jia Tan Date: 2023-02-24 18:10:37 +0800 Tests: Small tweak to test-vli.c. The static global variables can be disabled if encoders and decoders are not built. If they are not disabled and -Werror is used, it will cause an usused warning as an error. tests/test_vli.c | 2 ++ 1 file changed, 2 insertions(+) commit 4042dbf03a06e019ccdd1e9d1b94cf613d62d5a9 Author: Jia Tan Date: 2023-02-06 21:46:43 +0800 liblzma: Replace '\n' -> newline in filter.h documentation. The '\n' renders as a newline when the comments are converted to html by Doxygen. src/liblzma/api/lzma/filter.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 3971f5c5024750ce8286891c6f79ae1661047483 Author: Jia Tan Date: 2023-02-06 21:45:37 +0800 liblzma: Shorten return description for two functions in filter.h. Shorten the description for lzma_raw_encoder_memusage() and lzma_raw_decoder_memusage(). src/liblzma/api/lzma/filter.h | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) commit 5e61b39432752b6cd1a7b518f0f9e4f0c3f0f242 Author: Jia Tan Date: 2023-02-06 21:44:45 +0800 liblzma: Reword a few lines in filter.h src/liblzma/api/lzma/filter.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) commit 8a53533869defa1191b41f176a0312cb53a139e2 Author: Jia Tan Date: 2023-02-06 21:35:06 +0800 liblzma: Improve documentation in filter.h. All functions now explicitly specify parameter and return values. The notes and code annotations were moved before the parameter and return value descriptions for consistency. Also, the description above lzma_filter_encoder_is_supported() about not being able to list available filters was removed since lzma_str_list_filters() will do this. src/liblzma/api/lzma/filter.h | 226 ++++++++++++++++++++++++++---------------- 1 file changed, 143 insertions(+), 83 deletions(-) commit 6d05b08b17ac8cb93165ee8f310fdd925b0b258f Author: Lasse Collin Date: 2023-02-23 20:46:16 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit dfc9a54082e0fc7b3b796ea15336b5075acc79d5 Author: Lasse Collin Date: 2023-02-21 22:57:10 +0200 liblzma: Avoid null pointer + 0 (undefined behavior in C). In the C99 and C17 standards, section 6.5.6 paragraph 8 means that adding 0 to a null pointer is undefined behavior. As of writing, "clang -fsanitize=undefined" (Clang 15) diagnoses this. However, I'm not aware of any compiler that would take advantage of this when optimizing (Clang 15 included). It's good to avoid this anyway since compilers might some day infer that pointer arithmetic implies that the pointer is not NULL. That is, the following foo() would then unconditionally return 0, even for foo(NULL, 0): void bar(char *a, char *b); int foo(char *a, size_t n) { bar(a, a + n); return a == NULL; } In contrast to C, C++ explicitly allows null pointer + 0. So if the above is compiled as C++ then there is no undefined behavior in the foo(NULL, 0) call. To me it seems that changing the C standard would be the sane thing to do (just add one sentence) as it would ensure that a huge amount of old code won't break in the future. Based on web searches it seems that a large number of codebases (where null pointer + 0 occurs) are being fixed instead to be future-proof in case compilers will some day optimize based on it (like making the above foo(NULL, 0) return 0) which in the worst case will cause security bugs. Some projects don't plan to change it. For example, gnulib and thus many GNU tools currently require that null pointer + 0 is defined: https://lists.gnu.org/archive/html/bug-gnulib/2021-11/msg00000.html https://www.gnu.org/software/gnulib/manual/html_node/Other-portability-assumptions.html In XZ Utils null pointer + 0 issue should be fixed after this commit. This adds a few if-statements and thus branches to avoid null pointer + 0. These check for size > 0 instead of ptr != NULL because this way bugs where size > 0 && ptr == NULL will likely get caught quickly. None of them are in hot spots so it shouldn't matter for performance. A little less readable version would be replacing ptr + offset with offset != 0 ? ptr + offset : ptr or creating a macro for it: #define my_ptr_add(ptr, offset) \ ((offset) != 0 ? ((ptr) + (offset)) : (ptr)) Checking for offset != 0 instead of ptr != NULL allows GCC >= 8.1, Clang >= 7, and Clang-based ICX to optimize it to the very same code as ptr + offset. That is, it won't create a branch. So for hot code this could be a good solution to avoid null pointer + 0. Unfortunately other compilers like ICC 2021 or MSVC 19.33 (VS2022) will create a branch from my_ptr_add(). Thanks to Marcin Kowalczyk for reporting the problem: https://github.com/tukaani-project/xz/issues/36 src/liblzma/common/block_decoder.c | 5 ++++- src/liblzma/common/block_encoder.c | 7 +++++-- src/liblzma/common/common.c | 20 ++++++++++++++------ src/liblzma/common/index_decoder.c | 13 ++++++++++--- src/liblzma/common/index_encoder.c | 11 +++++++++-- src/liblzma/common/index_hash.c | 13 ++++++++++--- src/liblzma/common/lzip_decoder.c | 6 +++++- src/liblzma/delta/delta_decoder.c | 7 ++++++- src/liblzma/delta/delta_encoder.c | 12 ++++++++++-- src/liblzma/simple/simple_coder.c | 6 ++++-- 10 files changed, 77 insertions(+), 23 deletions(-) commit f6dce49cb656b358e2fb2a3032e35e20af34dc84 Author: Jia Tan Date: 2023-02-07 00:00:44 +0800 liblzma: Adjust container.h for consistency with filter.h. src/liblzma/api/lzma/container.h | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) commit 173d240bb4763aedc8c01df4d9c83e311d954c33 Author: Jia Tan Date: 2023-02-07 00:00:09 +0800 liblzma: Fix small typos and reword a few things in filter.h. src/liblzma/api/lzma/container.h | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) commit 17797bacde3f7264048ef0976c137a600148d6cf Author: Jia Tan Date: 2023-02-06 23:42:08 +0800 liblzma: Convert list of flags in lzma_mt to bulleted list. src/liblzma/api/lzma/container.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) commit 37da0e727161b401b8bfd8dccf163a4b2fc0460b Author: Jia Tan Date: 2023-01-26 23:17:41 +0800 liblzma: Fix typo in documentation in container.h lzma_microlzma_decoder -> lzma_microlzma_encoder src/liblzma/api/lzma/container.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit b8331077c626e6bf14f4671d09b561146eaf816a Author: Jia Tan Date: 2023-01-26 23:16:34 +0800 liblzma: Improve documentation for container.h Standardizing each function to always specify parameters and return values. Also moved the parameters and return values to the end of each function description. src/liblzma/api/lzma/container.h | 146 +++++++++++++++++++++++++-------------- 1 file changed, 93 insertions(+), 53 deletions(-) commit b9a3511bb61d3b6ce49abd33dce6155a573f5272 Author: Jia Tan Date: 2023-02-22 20:59:41 +0800 CMake: Add LZIP decoder test to list of tests. CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) commit cd82ef2fb48b174c57cd03b84a9a0f978772cb89 Author: Lasse Collin Date: 2023-02-17 20:56:49 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 076e911ba25cadf2cbdfbd0f65991a002e210c0b Author: Lasse Collin Date: 2023-02-17 20:48:28 +0200 Build: Use only the generic symbol versioning on MicroBlaze. On MicroBlaze, GCC 12 is broken in sense that __has_attribute(__symver__) returns true but it still doesn't support the __symver__ attribute even though the platform is ELF and symbol versioning is supported if using the traditional __asm__(".symver ...") method. Avoiding the traditional method is good because it breaks LTO (-flto) builds with GCC. See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101766 For now the only extra symbols in liblzma_linux.map are the compatibility symbols with the patch that spread from RHEL/CentOS 7. These require the use of __symver__ attribute or __asm__(".symver ...") in the C code. Compatibility with the patch from CentOS 7 doesn't seem valuable on MicroBlaze so use liblzma_generic.map on MicroBlaze instead. It doesn't require anything special in the C code and thus no LTO issues either. An alternative would be to detect support for __symver__ attribute in configure.ac and CMakeLists.txt and fall back to __asm__(".symver ...") but then LTO would be silently broken on MicroBlaze. It sounds likely that MicroBlaze is a special case so let's treat it as a such because that is simpler. If a similar issue exists on some other platform too then hopefully someone will report it and this can be reconsidered. (This doesn't do the same fix in CMakeLists.txt. Perhaps it should but perhaps CMake build of liblzma doesn't matter much on MicroBlaze. The problem breaks the build so it's easy to notice and can be fixed later.) Thanks to Vincent Fazio for reporting the problem and proposing a patch (in the end that solution wasn't used): https://github.com/tukaani-project/xz/pull/32 configure.ac | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) commit bc34e5ac9996667d2c1ec9a7895ec5931ac4caea Author: Lasse Collin Date: 2023-02-16 21:09:00 +0200 liblzma: Very minor API doc tweaks. Use "member" to refer to struct members as that's the term used by the C standard. Use lzma_options_delta.dist and such in docs so that in Doxygen's HTML output they will link to the doc of the struct member. Clean up a few trailing white spaces too. src/liblzma/api/lzma/block.h | 6 +++--- src/liblzma/api/lzma/delta.h | 6 +++--- src/liblzma/api/lzma/index.h | 10 +++++----- src/liblzma/api/lzma/stream_flags.h | 6 +++--- 4 files changed, 14 insertions(+), 14 deletions(-) commit d31fbd28be5b79eb682db50004b0fb7aad5299ec Author: Jia Tan Date: 2023-02-17 00:54:33 +0800 liblzma: Adjust spacing in doc headers in bcj.h. src/liblzma/api/lzma/bcj.h | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) commit 701e9be6bede19771895f756082db0c017f86f07 Author: Jia Tan Date: 2023-02-17 00:44:44 +0800 liblzma: Adjust documentation in bcj.h for consistent style. src/liblzma/api/lzma/bcj.h | 43 ++++++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 21 deletions(-) commit 762c4d0b62d2694cf3a01e030fdfe68e09e7b503 Author: Jia Tan Date: 2023-02-17 00:36:05 +0800 liblzma: Rename field => member in documentation. Also adjusted preset value => preset level. src/liblzma/api/lzma/base.h | 18 +++++++-------- src/liblzma/api/lzma/block.h | 44 ++++++++++++++++++------------------- src/liblzma/api/lzma/container.h | 26 +++++++++++----------- src/liblzma/api/lzma/delta.h | 12 +++++----- src/liblzma/api/lzma/index.h | 30 ++++++++++++------------- src/liblzma/api/lzma/lzma12.h | 28 +++++++++++------------ src/liblzma/api/lzma/stream_flags.h | 32 +++++++++++++-------------- 7 files changed, 95 insertions(+), 95 deletions(-) commit 0ce1db0223854d94b4a0d17737ac0486a75d9e6a Author: Lasse Collin Date: 2023-02-16 17:59:50 +0200 liblzma: Silence a warning from MSVC. It gives C4146 here since unary minus with unsigned integer is still unsigned (which is the intention here). Doing it with substraction makes it clearer and avoids the warning. Thanks to Nathan Moinvaziri for reporting this. src/liblzma/check/crc64_fast.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit d83da006b3e8dbad9112e6cde6b27049d193c9ec Author: Jia Tan Date: 2023-02-16 21:04:54 +0800 liblzma: Improve documentation for stream_flags.h Standardizing each function to always specify parameters and return values. Also moved the parameters and return values to the end of each function description. A few small things were reworded and long sentences broken up. src/liblzma/api/lzma/stream_flags.h | 76 ++++++++++++++++++++++--------------- 1 file changed, 46 insertions(+), 30 deletions(-) commit 2796bb4736f645d34939b5d62d5958f534e41e69 Author: Jia Tan Date: 2023-02-14 21:50:16 +0800 liblzma: Improve documentation in lzma12.h. All functions now explicitly specify parameter and return values. src/liblzma/api/lzma/lzma12.h | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-) commit ebebaa8d9381afea440eb1b01917117551adf68f Author: Jia Tan Date: 2023-01-27 22:44:06 +0800 liblzma: Improve documentation in check.h. All functions now explicitly specify parameter and return values. Also moved the note about SHA-256 functions not being exported to the top of the file. src/liblzma/api/lzma/check.h | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) commit 765fa2865aab97ea713c1715922e322b3cf599a7 Author: Jia Tan Date: 2023-02-08 21:33:52 +0800 liblzma: Improve documentation in index.h All functions now explicitly specify parameter and return values. src/liblzma/api/lzma/index.h | 177 ++++++++++++++++++++++++++++++------------- 1 file changed, 126 insertions(+), 51 deletions(-) commit 918e208af5960728b6def01e692b395f7d8e3701 Author: Jia Tan Date: 2023-02-08 20:35:32 +0800 liblzma: Reword a comment in index.h. src/liblzma/api/lzma/index.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 1f157d214bc09338876d2467b549935679abc0bf Author: Jia Tan Date: 2023-02-08 20:30:23 +0800 liblzma: Omit lzma_index_iter's internal field from Doxygen docs. Add \private above this field and its sub-fields since it is not meant to be modified by users. src/liblzma/api/lzma/index.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) commit 28757fa46d8e0f0a9c17628b2b6af9bcb3cd96fc Author: Jia Tan Date: 2023-01-21 21:32:03 +0800 liblzma: Fix documentation for LZMA_MEMLIMIT_ERROR. LZMA_MEMLIMIT_ERROR was missing the "<" character needed to put documentation after a member. src/liblzma/api/lzma/base.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 135d5a1a65a9e199b9a5550c1f788cf43cd81018 Author: Jia Tan Date: 2023-01-21 00:29:38 +0800 liblzma: Improve documentation for base.h. Standardizing each function to always specify params and return values. Also fixed a small grammar mistake. src/liblzma/api/lzma/base.h | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) commit 2287d5668384e289d51e72724997dc920483768e Author: Jia Tan Date: 2023-02-14 00:08:33 +0800 liblzma: Minor improvements to vli.h. Added [out] annotations to parameters that are pointers and can have their value changed. Also added a clarification to lzma_vli_is_valid. src/liblzma/api/lzma/vli.h | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) commit 7124b8a16ae60cb2e93218ff531868eebd673bde Author: Jia Tan Date: 2023-02-10 21:38:02 +0800 liblzma: Add comments for macros in delta.h. Document LZMA_DELTA_DIST_MIN and LZMA_DELTA_DIST_MAX for completeness and to avoid Doxygen warnings. src/liblzma/api/lzma/delta.h | 8 ++++++++ 1 file changed, 8 insertions(+) commit 59c7bb8931159fdb1a31bbbeaed0e6984e2d3c81 Author: Jia Tan Date: 2023-02-10 21:35:23 +0800 liblzma: Improve documentation in index_hash.h. All functions now explicitly specify parameter and return values. Also reworded the description of lzma_index_hash_init() for readability. src/liblzma/api/lzma/index_hash.h | 36 +++++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-) commit e970c28ac3cb2e8051925f81db2fe953664c2645 Author: Jia Tan Date: 2023-02-03 00:33:32 +0800 liblzma: Fix bug in lzma_str_from_filters() not checking filters[] length. The bug is only a problem in applications that do not properly terminate the filters[] array with LZMA_VLI_UNKNOWN or have more than LZMA_FILTERS_MAX filters. This bug does not affect xz. src/liblzma/common/string_conversion.c | 7 +++++++ 1 file changed, 7 insertions(+) commit 85e01266a96689448abb249da6c6abc3afcd4a4e Author: Jia Tan Date: 2023-02-03 00:32:47 +0800 Tests: Create test_filter_str.c. Tests lzma_str_to_filters(), lzma_str_from_filters(), and lzma_str_list_filters() API functions. CMakeLists.txt | 1 + tests/Makefile.am | 2 + tests/test_filter_str.c | 593 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 596 insertions(+) commit 3fa0f3ba12dd3383f62dbfa18a864f9b4217fa0a Author: Jia Tan Date: 2023-01-22 08:49:00 +0800 liblzma: Fix typos in comments in string_conversion.c. src/liblzma/common/string_conversion.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 32dbe045d74e94f75c53236fa2a6c0454d7b6d9e Author: Jia Tan Date: 2023-02-03 00:20:20 +0800 liblzma: Clarify block encoder and decoder documentation. Added a few sentences to the description for lzma_block_encoder() and lzma_block_decoder() to highlight that the Block Header must be coded before calling these functions. src/liblzma/api/lzma/block.h | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) commit ccf12acbfa7331b1bbb99ec55879186eb35f879f Author: Jia Tan Date: 2023-02-03 00:12:24 +0800 Update lzma_block documentation for lzma_block_uncomp_encode(). src/liblzma/api/lzma/block.h | 3 +++ 1 file changed, 3 insertions(+) commit 6a0b168dd9dc1ef437255c4bfdc6eff8b96fdb24 Author: Jia Tan Date: 2023-02-03 00:11:37 +0800 liblzma: Minor edits to lzma_block header_size documentation. src/liblzma/api/lzma/block.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 84ce36f90e68471fec1f0e61cd93ac7ed9ab4883 Author: Jia Tan Date: 2023-02-03 00:11:07 +0800 liblzma: Enumerate functions that read version in lzma_block. src/liblzma/api/lzma/block.h | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) commit d6620774682830d606f57109861b6763805b3492 Author: Jia Tan Date: 2023-02-03 00:10:34 +0800 liblzma: Clarify comment in block.h. src/liblzma/api/lzma/block.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 880adb5aa25f66a53d81c2f3acc716f7a4d373d1 Author: Jia Tan Date: 2023-02-03 00:07:23 +0800 liblzma: Improve documentation for block.h. Standardizing each function to always specify params and return values. Output pointer parameters are also marked with doxygen style [out] to make it clear. Any note sections were also moved above the parameter and return sections for consistency. src/liblzma/api/lzma/block.h | 96 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 75 insertions(+), 21 deletions(-) commit b5b1b1f061c342271e4977ce5cba604a19c0ca52 Author: Jia Tan Date: 2023-02-01 23:38:30 +0800 liblzma: Clarify a comment about LZMA_STR_NO_VALIDATION. The flag description for LZMA_STR_NO_VALIDATION was previously confusing about the treatment for filters than cannot be used with .xz format (lzma1) without using LZMA_STR_ALL_FILTERS. Now, it is clear that LZMA_STR_NO_VALIDATION is not a super set of LZMA_STR_ALL_FILTERS. src/liblzma/api/lzma/filter.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) commit e904e778b82b14f2779aab80d6c8f3c01a3fc54b Author: Jia Tan Date: 2023-01-27 20:14:51 +0800 Translations: Add Brazilian Portuguese translation of man pages. Thanks to Rafael Fontenelle. po4a/po4a.conf | 2 +- po4a/pt_BR.po | 3677 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 3678 insertions(+), 1 deletion(-) commit e9c47e79c9584ba1126f5fa0dbe6c96e67411aa6 Author: Jia Tan Date: 2023-01-24 20:48:50 +0800 liblzma: Fix documentation in filter.h for lzma_str_to_filters() The previous documentation for lzma_str_to_filters() was technically correct, but misleading. lzma_str_to_filters() returns NULL on success, which is in practice always defined to 0. This is the same value as LZMA_OK, but lzma_str_to_filters() does not return lzma_ret so we should be more clear. src/liblzma/api/lzma/filter.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 99575947a58a60416c570eb78038d18a1ea4cef4 Author: Jia Tan Date: 2023-01-07 21:55:06 +0800 xz: Refactor duplicated check for custom suffix when using --format=raw src/xz/args.c | 8 ++++++++ src/xz/suffix.c | 26 ++++++++------------------ src/xz/suffix.h | 8 ++++++++ 3 files changed, 24 insertions(+), 18 deletions(-) commit 76dec92fcca4a9ccd2063ed6d5d9e3474665baad Author: Jia Tan Date: 2023-01-20 21:53:14 +0800 liblzma: Set documentation on all reserved fields to private. This prevents the reserved fields from being part of the generated Doxygen documentation. src/liblzma/api/lzma/base.h | 17 +++++++++++++++ src/liblzma/api/lzma/block.h | 43 +++++++++++++++++++++++++++++++++++++ src/liblzma/api/lzma/container.h | 24 +++++++++++++++++++++ src/liblzma/api/lzma/delta.h | 12 +++++++++++ src/liblzma/api/lzma/index.h | 27 +++++++++++++++++++++++ src/liblzma/api/lzma/lzma12.h | 22 +++++++++++++++++++ src/liblzma/api/lzma/stream_flags.h | 28 ++++++++++++++++++++++++ 7 files changed, 173 insertions(+) commit bd213d06ebf92cf42eeb97e7c578bddc857f8ea8 Author: Jia Tan Date: 2022-12-21 23:59:43 +0800 liblzma: Highlight liblzma API headers should not be included directly. This improves the generated Doxygen HTML files to better highlight how to properly use the liblzma API header files. src/liblzma/api/lzma/base.h | 5 +++-- src/liblzma/api/lzma/bcj.h | 5 +++-- src/liblzma/api/lzma/block.h | 5 +++-- src/liblzma/api/lzma/check.h | 5 +++-- src/liblzma/api/lzma/container.h | 5 +++-- src/liblzma/api/lzma/delta.h | 5 +++-- src/liblzma/api/lzma/filter.h | 5 +++-- src/liblzma/api/lzma/hardware.h | 5 +++-- src/liblzma/api/lzma/index.h | 5 +++-- src/liblzma/api/lzma/index_hash.h | 5 +++-- src/liblzma/api/lzma/lzma12.h | 5 +++-- src/liblzma/api/lzma/stream_flags.h | 5 +++-- src/liblzma/api/lzma/version.h | 5 +++-- src/liblzma/api/lzma/vli.h | 5 +++-- 14 files changed, 42 insertions(+), 28 deletions(-) commit 257dbff0ba1a7bc45a74d203ece015c9f70c88cd Author: Jia Tan Date: 2023-01-19 20:35:09 +0800 tuklib_physmem: Silence warning from -Wcast-function-type on MinGW-w64. tuklib_physmem depends on GetProcAddress() for both MSVC and MinGW-w64 to retrieve a function address. The proper way to do this is to cast the return value to the type of function pointer retrieved. Unfortunately, this causes a cast-function-type warning, so the best solution is to simply ignore the warning. src/common/tuklib_physmem.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) commit 720ad4a44282a7ee59aa9920eaf495d85d245d97 Author: Jia Tan Date: 2023-01-16 21:35:45 +0800 xz: Add missing comment for coder_set_compression_settings() src/xz/coder.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 88dc19163421282369c0989e997c05f9a447edc6 Author: Jia Tan Date: 2023-01-16 20:55:10 +0800 xz: Do not set compression settings with raw format in list mode. Calling coder_set_compression_settings() in list mode with verbose mode on caused the filter chain and memory requirements to print. This was unnecessary since the command results in an error and not consistent with other formats like lzma and alone. src/xz/args.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 039e0ab13efb144642f9d99eeeb9c668e76bb430 Author: Jia Tan Date: 2023-01-13 20:37:06 +0800 Translations: Update the Brazilian Portuguese translation. po/pt_BR.po | 603 ++++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 344 insertions(+), 259 deletions(-) commit 718f7a60e740b26b8353965edaaddc7d4deb4d85 Author: Lasse Collin Date: 2023-01-12 13:04:05 +0200 Build: Omit -Wmissing-noreturn from the default warnings. It's not that important. It can be annoying in builds that disable many features since in those cases the tests programs will correctly trigger this warning with Clang. configure.ac | 1 - 1 file changed, 1 deletion(-) commit 3ccedb09724c998c39d708c945f6da5852c39e13 Author: Lasse Collin Date: 2023-01-12 06:05:58 +0200 xz: Use ssize_t for the to-be-ignored return value from write(fd, ptr, 1). It makes no difference here as the return value fits into an int too and it then gets ignored but this looks better. src/xz/file_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 09fbd2f0527def89e839c8907de4fe7ef6bc5019 Author: Lasse Collin Date: 2023-01-12 06:01:12 +0200 xz: Silence warnings from -Wsign-conversion in a 32-bit build. src/common/tuklib_mbstr_fw.c | 2 +- src/xz/list.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) commit 683d3f178ef1487b5418be49f331b0131a101b40 Author: Lasse Collin Date: 2023-01-12 05:38:48 +0200 liblzma: Silence another warning from -Wsign-conversion in a 32-bit build. It doesn't warn on a 64-bit system because truncating a ptrdiff_t (signed long) to uint32_t is diagnosed under -Wconversion by GCC and -Wshorten-64-to-32 by Clang. src/liblzma/lz/lz_encoder_mf.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) commit 2b8062ef94a38d0f9ad0d1b309e8748db52f5c15 Author: Lasse Collin Date: 2023-01-12 04:46:45 +0200 liblzma: Silence a warning from -Wsign-conversion in a 32-bit build. src/common/mythread.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit b16b9c0d22227012518595c2222facc73bd833a8 Author: Lasse Collin Date: 2023-01-12 04:17:24 +0200 Build: Make configure add more warning flags for GCC and Clang. -Wstrict-aliasing was removed from the list since it is enabled by -Wall already. A normal build is clean with these on GNU/Linux x86-64 with GCC 12.2.0 and Clang 14.0.6. configure.ac | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) commit c47ecd6d3909d0d3ff48dfd6e2ee41e7c7130b94 Author: Lasse Collin Date: 2023-01-12 04:14:18 +0200 Tests: Fix warnings from clang --Wassign-enum. Explicitly casting the integer to lzma_check silences the warning. Since such an invalid value is needed in multiple tests, a constant INVALID_LZMA_CHECK_ID was added to tests.h. The use of 0x1000 for lzma_block.check wasn't optimal as if the underlying type is a char then 0x1000 will be truncated to 0. However, in these test cases the value is ignored, thus even with such truncation the test would have passed. tests/test_block_header.c | 6 +++--- tests/test_check.c | 2 +- tests/test_stream_flags.c | 8 ++++---- tests/tests.h | 9 +++++++++ 4 files changed, 17 insertions(+), 8 deletions(-) commit 34e13ce015232968731de2a9ec3440a08b0084b1 Author: Lasse Collin Date: 2023-01-12 03:51:07 +0200 Tests: Silence warnings from -Wsign-conversion. Note that assigning an unsigned int to lzma_check doesn't warn on GNU/Linux x86-64 since the enum type is unsigned on that platform. The enum can be signed on some other platform though so it's best to use enumeration type lzma_check in these situations. tests/test_check.c | 6 +++--- tests/test_stream_flags.c | 10 +++++----- 2 files changed, 8 insertions(+), 8 deletions(-) commit 6671d0fe46b77f0fafce860836b7a12dc3cda14a Author: Lasse Collin Date: 2023-01-12 03:19:59 +0200 liblzma: Silence warnings from clang -Wconditional-uninitialized. This is similar to 2ce4f36f179a81d0c6e182a409f363df759d1ad0. The actual initialization of the variables is done inside mythread_sync() macro. Clang doesn't seem to see that the initialization code inside the macro is always executed. src/liblzma/common/stream_decoder_mt.c | 8 +++++--- src/liblzma/common/stream_encoder_mt.c | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-) commit d3e833ca1d2abda54648494c33aca73a40a47efe Author: Lasse Collin Date: 2023-01-12 03:11:40 +0200 Fix warnings from clang -Wdocumentation. src/liblzma/check/check.h | 4 ---- src/liblzma/lz/lz_encoder_mf.c | 4 ++-- src/xz/options.c | 4 ++-- 3 files changed, 4 insertions(+), 8 deletions(-) commit 977dd2e26bc50efec8d30fb90380394042d24558 Author: Lasse Collin Date: 2023-01-12 03:04:28 +0200 Tests: test_lzip_decoder: Remove trailing white-space. tests/test_lzip_decoder.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit c55157ed7437ab14f2afb7fecf331e321f2edf9a Author: Lasse Collin Date: 2023-01-12 03:03:55 +0200 Tests: test_lzip_decoder: Silence warnings from -Wsign-conversion. tests/test_lzip_decoder.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) commit 18b845e69752c975dfeda418ec00eda22605c2ee Author: Lasse Collin Date: 2023-01-11 18:52:54 +0200 Bump version and soname for 5.4.1. src/liblzma/Makefile.am | 2 +- src/liblzma/api/lzma/version.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit 4080bbb844fb36701ffb978f0c41ea2c2c9f8960 Author: Jia Tan Date: 2023-01-11 23:58:16 +0800 Add NEWS for 5.4.1. NEWS | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) commit 674c89fdb8c457ebc3a0652e235d8b6cd7b7dee2 Author: Lasse Collin Date: 2023-01-10 11:56:11 +0200 sysdefs.h: Don't include strings.h anymore. On some platforms src/xz/suffix.c may need for strcasecmp() but suffix.c includes the header when it needs it. Unless there is an old system that otherwise supports enough C99 to build XZ Utils but doesn't have C89/C90-compatible , there should be no need to include in sysdefs.h. src/common/sysdefs.h | 6 ------ 1 file changed, 6 deletions(-) commit 2a6b938084fac9ddb39cd69c9beeed15c3b6f6f6 Author: Lasse Collin Date: 2023-01-10 11:23:41 +0200 xz: Include in suffix.c if needed for strcasecmp(). SUSv2 and POSIX.1‐2017 declare only a few functions in . Of these, strcasecmp() is used on some platforms in suffix.c. Nothing else in the project needs (at least if building on a modern system). sysdefs.h currently includes if HAVE_STRINGS_H is defined and suffix.c relied on this. Note that dos/config.h doesn't #define HAVE_STRINGS_H even though DJGPP does have strings.h. It isn't needed with DJGPP as strcasecmp() is also in in DJGPP. src/xz/suffix.c | 3 +++ 1 file changed, 3 insertions(+) commit aea639e81beb548e3114c74b6d9a894d6e036189 Author: Jia Tan Date: 2023-01-11 22:46:48 +0800 xz: Fix warning -Wformat-nonliteral on clang in message.c. clang and gcc differ in how they handle -Wformat-nonliteral. gcc will allow a non-literal format string as long as the function takes its format arguments as a va_list. src/xz/message.c | 9 +++++++++ 1 file changed, 9 insertions(+) commit e3b42bfcb0f67988beee7c7022fed0361282be45 Author: Jia Tan Date: 2023-01-11 20:58:31 +0800 Tests: Fix test_filter_flags copy/paste error. tests/test_filter_flags.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 21625b7e11d004788e40eb5eb88d9d89f65fe347 Author: Jia Tan Date: 2023-01-11 20:42:29 +0800 Tests: Fix type-limits warning in test_filter_flags. This only occurs in test_filter_flags when the BCJ filters are not configured and built. In this case, ARRAY_SIZE() returns 0 and causes a type-limits warning with the loop variable since an unsigned number will always be >= 0. tests/test_filter_flags.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) commit c337983e928682d56ce3470b286a8d5b8646e0ad Author: Lasse Collin Date: 2023-01-10 22:14:03 +0200 liblzma: CLMUL CRC64: Work around a bug in MSVC, second attempt. This affects only 32-bit x86 builds. x86-64 is OK as is. I still cannot easily test this myself. The reporter has tested this and it passes the tests included in the CMake build and performance is good: raw CRC64 is 2-3 times faster than the C version of the slice-by-four method. (Note that liblzma doesn't include a MSVC-compatible version of the 32-bit x86 assembly code for the slice-by-four method.) Thanks to Iouri Kharon for figuring out a fix, testing, and benchmarking. src/liblzma/check/crc64_fast.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) commit b7fb438ea0e3ee02e3a164f3b72fae456cbe34d7 Author: Jia Tan Date: 2023-01-11 01:18:50 +0800 Tests: Fix unused function warning in test_block_header. One of the global arrays of filters was only used in a test that required both encoders and decoders to be configured in the build. tests/test_block_header.c | 4 ++++ 1 file changed, 4 insertions(+) commit 68e9ef036d18d7c3952bff0b391d5989b86934da Author: Jia Tan Date: 2023-01-11 01:08:03 +0800 Tests: Fix unused function warning in test_index_hash. test_index_hash does not use fill_index_hash() unless both encoders and decoders are configured in the build. tests/test_index_hash.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) commit a387707cd8cdefbffb5b7429bda4b7fcc192954a Author: Lasse Collin Date: 2023-01-10 00:33:14 +0200 Windows: Update INSTALL-MSVC.txt to recommend CMake over project files. windows/INSTALL-MSVC.txt | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) commit 52902ad69518255a14b0144f0a2379e06fde5b6e Author: Lasse Collin Date: 2023-01-10 12:47:16 +0200 Revert "liblzma: CLMUL CRC64: Workaround a bug in MSVC (VS2015-2022)." This reverts commit 36edc65ab4cf10a131f239acbd423b4510ba52d5. It was reported that it wasn't a good enough fix and MSVC still produced (different kind of) bad code when building for 32-bit x86 if optimizations are enabled. Thanks to Iouri Kharon. src/liblzma/check/crc64_fast.c | 6 ------ 1 file changed, 6 deletions(-) commit e81b9fc48ca70f9228308d3f1871cd81f9a5a496 Author: Lasse Collin Date: 2023-01-10 10:05:13 +0200 sysdefs.h: Fix a comment. src/common/sysdefs.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 6e89ab58b031aa046308a0b3504ff0a5be042571 Author: Lasse Collin Date: 2023-01-10 10:04:06 +0200 sysdefs.h: Don't include memory.h anymore even if it were available. It quite probably was never needed, that is, any system where memory.h was required likely couldn't compile XZ Utils for other reasons anyway. XZ Utils 5.2.6 and later source packages were generated using Autoconf 2.71 which no longer defines HAVE_MEMORY_H. So the code being removed is no longer used anyway. src/common/sysdefs.h | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) commit 65c59ad429aa59f9df0326d9fc82931ba4a9d123 Author: Lasse Collin Date: 2023-01-10 08:50:26 +0200 CMake/Windows: Add a workaround for windres from GNU binutils. This is combined from the following commits in the master branch: 443dfebced041adc88f10d824188eeef5b5821a9 6b117d3b1fe91eb26d533ab16a2e552f84148d47 5e34774c31d1b7509b5cb77a3be9973adec59ea0 Thanks to Iouri Kharon for the bug report, the original patch, and testing. CMakeLists.txt | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) commit 43521e77acc907863fa4f94aae276366172cb9ee Author: Lasse Collin Date: 2023-01-06 22:53:38 +0200 Tests: test_filter_flags: Clean up minor issues. Here are the list of the most significant issues addressed: - Avoid using internal common.h header. It's not good to copy the constants like this but common.h cannot be included for use outside of liblzma. This is the quickest thing to do that could be fixed later. - Omit the INIT_FILTER macro. Initialization should be done with just regular designated initializers. - Use start_offset = 257 for BCJ tests. It demonstrates that Filter Flags encoder and decoder don't validate the options thoroughly. 257 is valid only for the x86 filter. This is a bit silly but not a significant problem in practice because the encoder and decoder initialization functions will catch bad alignment still. Perhaps this should be fixed but it's not urgent and doesn't need to be in 5.4.x. - Various tweaks to comments such as filter id -> Filter ID tests/test_filter_flags.c | 153 +++++++++++++++++++++++----------------------- 1 file changed, 78 insertions(+), 75 deletions(-) commit 6b44cead95d767414272dc3a67898a36bfdf95b3 Author: Jia Tan Date: 2022-12-29 23:33:33 +0800 Tests: Refactors existing filter flags tests. Converts the existing filter flags tests into tuktests. tests/test_filter_flags.c | 655 ++++++++++++++++++++++++++++++++-------------- 1 file changed, 457 insertions(+), 198 deletions(-) commit 1bbefa9659b202ba31bd244a9d0e4f0d37ff3ed7 Author: Lasse Collin Date: 2023-01-08 00:32:29 +0200 Tests: tuktest.h: Support tuktest_malloc(0). It's not needed in XZ Utils at least for now. It's good to support it still because if such use is needed later, it wouldn't be caught on GNU/Linux since malloc(0) from glibc returns non-NULL. tests/tuktest.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit ce3a3fbc7c2c399aeed644d54f3bd56ac914dfee Author: Lasse Collin Date: 2023-01-07 21:57:11 +0200 CMake: Update cmake_minimum_required from 3.13...3.16 to 3.13...3.25. The changes listed on cmake-policies(7) for versions 3.17 to 3.25 shouldn't affect this project. CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 99fcd57f2ea35eaa94e09f674d5364329c880fa2 Author: Lasse Collin Date: 2023-01-08 00:24:23 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit c0c13d9d82eb8a4302c8bbb8b4c5178d285fe9ab Author: Lasse Collin Date: 2023-01-07 19:50:35 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 3d45987451b1c3bb42697b29341824c0e5484cba Author: Lasse Collin Date: 2023-01-09 11:27:24 +0200 CMake: Fix a copypaste error in xzdec Windows resource file handling. It was my mistake. Thanks to Iouri Kharon for the bug report. CMakeLists.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 706bce5018d7cf83094e13454a0731169ec119b5 Author: Lasse Collin Date: 2023-01-07 19:50:03 +0200 CMake/Windows: Add resource files to xz.exe and xzdec.exe. The command line tools cannot be built with MSVC for now but they can be built with MinGW-w64. Thanks to Iouri Kharon for the bug report and the original patch. CMakeLists.txt | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) commit e96dee55df04113c33b387ccdb6cb70935422d91 Author: Lasse Collin Date: 2023-01-09 12:22:05 +0200 liblzma: CLMUL CRC64: Workaround a bug in MSVC (VS2015-2022). I haven't tested with MSVC myself and there doesn't seem to be information about the problem online, so I'm relying on the bug report. Thanks to Iouri Kharon for the bug report and the patch. src/liblzma/check/crc64_fast.c | 6 ++++++ 1 file changed, 6 insertions(+) commit 52bc1ee34dda9bb6fb40175e5952863066681b77 Author: Lasse Collin Date: 2023-01-07 19:31:15 +0200 Build: Require that _mm_set_epi64x() is usable to enable CLMUL support. VS2013 doesn't have _mm_set_epi64x() so this way CLMUL gets disabled with VS2013. Thanks to Iouri Kharon for the bug report. CMakeLists.txt | 3 ++- configure.ac | 8 ++++++-- 2 files changed, 8 insertions(+), 3 deletions(-) commit bad44cfe19e8be8ae76755369be2a34abcd2b4fa Author: Jia Tan Date: 2022-12-29 00:25:18 +0800 Tests: Creates test_index_hash.c Tests all API functions exported from index_hash.h. Does not have a dedicated test for lzma_index_hash_end. [Minor edits were made by Lasse Collin.] .gitignore | 1 + CMakeLists.txt | 2 + tests/Makefile.am | 3 + tests/test_index_hash.c | 388 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 394 insertions(+) commit 692ccdf5516dfe55fb6e9c5cdfb31f4c02c1ecd1 Author: Jia Tan Date: 2023-01-05 20:57:25 +0800 liblzma: Remove common.h include from common/index.h. common/index.h is needed by liblzma internally and tests. common.h will include and define many things that are not needed by the tests. Also, this prevents include order problems because both common.h and lzma.h define LZMA_API. On most platforms it results only in a warning but on Windows it would break the build as the definition in common.h must be used only for building liblzma itself. src/liblzma/common/index.c | 1 + src/liblzma/common/index.h | 9 +++++++-- src/liblzma/common/index_decoder.h | 1 + src/liblzma/common/stream_buffer_encoder.c | 1 + 4 files changed, 10 insertions(+), 2 deletions(-) commit 2ac7bafc8f07c1edefe96a4a7a040ddfff0eb5bb Author: Jia Tan Date: 2022-08-17 20:20:16 +0800 liblzma: Add NULL check to lzma_index_hash_append. This is for consistency with lzma_index_append. src/liblzma/common/index_hash.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit db714d30e0c74d1dd4af1a23ed62b44e0e8e4efc Author: Jia Tan Date: 2022-08-17 17:59:51 +0800 liblzma: Replaced hardcoded 0x0 index indicator byte with macro src/liblzma/common/index.h | 3 +++ src/liblzma/common/index_decoder.c | 2 +- src/liblzma/common/index_encoder.c | 2 +- src/liblzma/common/index_hash.c | 2 +- src/liblzma/common/stream_decoder.c | 3 ++- src/liblzma/common/stream_decoder_mt.c | 2 +- 6 files changed, 9 insertions(+), 5 deletions(-) commit 39d2585dcd3e827cfc3c46025ab6708c4aeb36c6 Author: Jia Tan Date: 2023-01-06 20:43:31 +0800 Style: Change #if !defined() to #ifndef in mythread.h. src/common/mythread.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 3f0130aa288e4ed57ace609517db9700a41223af Author: Jia Tan Date: 2023-01-06 20:35:55 +0800 Build: Add missing stream_decoder_mt.c to .vcxproj files. The line in the .vcxproj files for building with was missing in 5.4.0. Thank to Hajin Jang for reporting the issue. windows/vs2013/liblzma.vcxproj | 1 + windows/vs2013/liblzma_dll.vcxproj | 1 + windows/vs2017/liblzma.vcxproj | 1 + windows/vs2017/liblzma_dll.vcxproj | 1 + windows/vs2019/liblzma.vcxproj | 1 + windows/vs2019/liblzma_dll.vcxproj | 1 + 6 files changed, 6 insertions(+) commit f204d1050a515d17851eed9049862ce5a9c729c1 Author: Lasse Collin Date: 2023-01-04 22:40:54 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 34a9c2d650d6c30bd88e1b21910dd863209aa884 Author: Lasse Collin Date: 2023-01-04 18:40:28 +0200 Tests: Adjust style in test_compress.sh. tests/test_compress.sh | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) commit 761c208d58e0c3daa0f46e68b406adfc318d2a46 Author: Jia Tan Date: 2023-01-04 23:58:58 +0800 Tests: Replace non portable shell parameter expansion The shell parameter expansion using # and ## is not supported in Solaris 10 Bourne shell (/bin/sh). Even though this is POSIX, it is not fully portable, so we should avoid it. tests/create_compress_files.c | 2 +- tests/test_compress.sh | 20 +++++++++++++------- tests/test_compress_prepared_bcj_sparc | 2 +- tests/test_compress_prepared_bcj_x86 | 2 +- 4 files changed, 16 insertions(+), 10 deletions(-) commit 8a7cbc074547e55e57f4f3696f69bedeb05e14c4 Author: Jia Tan Date: 2023-01-03 21:02:38 +0800 Translations: Add Korean translation of man pages. Thanks to Seong-ho Cho po4a/ko.po | 5552 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ po4a/po4a.conf | 2 +- 2 files changed, 5553 insertions(+), 1 deletion(-) commit ca2af49bb8be5995eb0e6a3abf457622626d49a7 Author: Jia Tan Date: 2023-01-03 20:47:27 +0800 Translations: Update the Esperanto translation. po/eo.po | 620 ++++++++++++++++++++++++++++++++++----------------------------- 1 file changed, 332 insertions(+), 288 deletions(-) commit bfba3394aed03311fe9a746d3141b2e16d8b9325 Author: Lasse Collin Date: 2023-01-02 17:05:07 +0200 Build: Fix config.h comments. configure.ac | 2 +- m4/tuklib_progname.m4 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit 507648ad114c2ae0cd6d181063e1ac07e8106718 Author: Jia Tan Date: 2023-01-02 22:33:48 +0800 Build: Only define HAVE_PROGRAM_INVOCATION_NAME if it is set to 1. HAVE_DECL_PROGRAM_INVOCATION_NAME is renamed to HAVE_PROGRAM_INVOCATION_NAME. Previously, HAVE_DECL_PROGRAM_INVOCATION_NAME was always set when building with autotools. CMake would only set this when it was 1, and the dos/config.h did not define it. The new macro definition is consistent across build systems. cmake/tuklib_progname.cmake | 5 ++--- m4/tuklib_progname.m4 | 5 ++++- src/common/tuklib_progname.c | 2 +- src/common/tuklib_progname.h | 2 +- 4 files changed, 8 insertions(+), 6 deletions(-) commit ab5229d32adfec1f3fbc95228d9dd6f560732ab5 Author: Lasse Collin Date: 2022-12-30 20:10:08 +0200 Tests: test_check: Test corner cases of CLMUL CRC64. tests/test_check.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) commit 8791826f31733fda0a13b411c2ed930faaeb25aa Author: Lasse Collin Date: 2022-12-30 19:36:49 +0200 Tests: Clarify a comment in test_lzip_decoder.c. tests/test_lzip_decoder.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) commit c410d812ea12bfc74f6b727c1a799478c79f19ca Author: Jia Tan Date: 2022-12-29 01:55:19 +0800 xz: Includes and conditionally in mytime.c. Previously, mytime.c depended on mythread.h for to be included. src/xz/mytime.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) commit 501c6013d4a59fae5d4368e9657c4885493db809 Author: Jia Tan Date: 2022-12-29 01:15:27 +0800 liblzma: Includes sys/time.h conditionally in mythread Previously, was always included, even if mythread only used clock_gettime. is still needed even if clock_gettime is not used though because struct timespec is needed for mythread_condtime. src/common/mythread.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) commit 9e3cb514b5b95bd235dcdff3db4436f57444ee4f Author: Jia Tan Date: 2022-12-29 01:10:53 +0800 Build: No longer require HAVE_DECL_CLOCK_MONOTONIC to always be set. Previously, if threading was enabled HAVE_DECL_CLOCK_MONOTONIC would always be set to 0 or 1. However, this macro was needed in xz so if xz was not built with threading and HAVE_DECL_CLOCK_MONOTONIC was not defined but HAVE_CLOCK_GETTIME was, it caused a warning during build. Now, HAVE_DECL_CLOCK_MONOTONIC has been renamed to HAVE_CLOCK_MONOTONIC and will only be set if it is 1. CMakeLists.txt | 8 +++----- configure.ac | 5 ++++- src/common/mythread.h | 4 ++-- src/xz/mytime.c | 5 ++--- 4 files changed, 11 insertions(+), 11 deletions(-) commit 6fc3e5467911572fa9af4021ea46396261aae796 Author: Jia Tan Date: 2022-12-28 01:14:07 +0800 Translations: Add Ukrainian translations of man pages. Thanks to Yuri Chornoivan po4a/po4a.conf | 2 +- po4a/uk.po | 3676 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 3677 insertions(+), 1 deletion(-) commit e84f2ab7f8bc38cd8f8befa0bb398656c3c11f8e Author: Jia Tan Date: 2022-12-22 23:14:53 +0800 liblzma: Update documentation for lzma_filter_encoder. src/liblzma/common/filter_encoder.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) commit b14b8dbba9a3b232787ae218b46430b9246383dd Author: Jia Tan Date: 2022-12-21 21:12:03 +0800 Tests: Adds lzip decoder tests .gitignore | 1 + tests/Makefile.am | 2 + tests/test_lzip_decoder.c | 471 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 474 insertions(+) commit 09a114805e1d4f9a02a06cee7dbf2f5014d1f710 Author: Jia Cheong Tan Date: 2022-12-20 22:05:21 +0800 Doxygen: Update .gitignore for generating docs for in source build. In source builds are not recommended, but we should still ignore the generated artifacts. .gitignore | 2 ++ 1 file changed, 2 insertions(+) commit d3e6fe44196bf9478ad193522e2b48febf2eca6b Author: Jia Tan Date: 2022-12-20 20:46:44 +0800 liblzma: Fix lzma_microlzma_encoder() return value. Using return_if_error on lzma_lzma_lclppb_encode was improper because return_if_error is expecting an lzma_ret value, but lzma_lzma_lclppb_encode returns a boolean. This could result in lzma_microlzma_encoder, which would be misleading for applications. src/liblzma/common/microlzma_encoder.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit b55a27b46f52524a4a8d9cdef03e6689cefc1375 Author: Lasse Collin Date: 2022-12-16 18:30:02 +0200 liblzma: Update authors list in arm64.c. src/liblzma/simple/arm64.c | 1 + 1 file changed, 1 insertion(+) commit 2fd28d2b7cec3468324a6f15eff7e73c285b1d7d Author: Jia Tan Date: 2022-12-16 20:58:55 +0800 CMake: Update .gitignore for CMake artifacts from in source build. In source builds are not recommended, but we can make it easier by ignoring the generated artifacts from CMake. .gitignore | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) commit b69da6d4bb6bb11fc0cf066920791990d2b22a06 Author: Lasse Collin Date: 2022-12-13 20:37:17 +0200 Bump version to 5.4.0 and soname to 5.4.0. src/liblzma/Makefile.am | 2 +- src/liblzma/api/lzma/version.h | 6 +++--- src/liblzma/liblzma_generic.map | 2 +- src/liblzma/liblzma_linux.map | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) commit 20869eb3fb280ff4f271ef527b12b6bf68b05e19 Author: Lasse Collin Date: 2022-12-13 20:29:39 +0200 Update INSTALL: CMake on Windows isn't experimental anymore. Using CMake to build liblzma should work on a few other OSes but building the command line tools is still subtly broken. It is known that shared library versioning may differ between CMake and Libtool builds on some OSes, most notably Darwin. INSTALL | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) commit cbbd84451944e3e8c63acfaa3c923f6d8aff7852 Author: Lasse Collin Date: 2022-12-13 19:47:53 +0200 Add NEWS for 5.4.0. NEWS | 202 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 202 insertions(+) commit c3e94d37e8d10a3e96019864b6f5d7b578db2c14 Author: Lasse Collin Date: 2022-12-13 17:41:20 +0200 Fix a typo in NEWS. NEWS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 0d2a2e0a545c3da2b3e9500f1e531eb903087245 Author: Lasse Collin Date: 2022-12-13 17:41:03 +0200 Add NEWS for 5.2.10. NEWS | 12 ++++++++++++ 1 file changed, 12 insertions(+) commit 177ece1c8eb007188fb1b04eff09ca2193fbdea6 Author: Lasse Collin Date: 2022-12-13 12:30:45 +0200 Tests: Fix a typo in tests/files/README. tests/files/README | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 60f45bdbaa6b07558b3f4baac285739b0c6342f5 Author: Lasse Collin Date: 2022-12-13 12:30:09 +0200 Tests: Add two ARM64 test files. tests/files/README | 7 +++++++ tests/files/good-1-arm64-lzma2-1.xz | Bin 0 -> 512 bytes tests/files/good-1-arm64-lzma2-2.xz | Bin 0 -> 488 bytes tests/test_files.sh | 5 +++++ 4 files changed, 12 insertions(+) commit f5e419550619c548c7c35d7e367cf00580a56521 Author: Lasse Collin Date: 2022-12-12 22:44:21 +0200 Translations: Update the Catalan translation. po/ca.po | 657 +++++++++++++++++++++++++++++---------------------------------- 1 file changed, 306 insertions(+), 351 deletions(-) commit 0fb9d355da3789b1757040af475b4e6bbc8b8af8 Author: Lasse Collin Date: 2022-12-12 19:18:12 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit e5b6c161c61a37c54dcb76a99bbb83ac4abe02dc Author: Lasse Collin Date: 2022-12-12 19:07:58 +0200 Update AUTHORS. AUTHORS | 12 ++++++++++++ 1 file changed, 12 insertions(+) commit f2d98e691099d82054d5f3071ef6f5e809932e44 Author: Lasse Collin Date: 2022-12-12 15:31:14 +0200 Docs: Omit multi-threaded decompress from TODO. The TODO file outdated still. TODO | 2 -- 1 file changed, 2 deletions(-) commit b42908c42a4cc091db45a7e5ba0e0ecceaa3f6da Author: Lasse Collin Date: 2022-12-11 21:16:09 +0200 Docs: Update xz-file-format.txt to 1.1.0 for ARM64 filter. doc/xz-file-format.txt | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) commit 854f2f5946b353cb0963fd6dfd54d363adc89b9f Author: Lasse Collin Date: 2022-12-11 21:13:57 +0200 xz: Rename --experimental-arm64 to --arm64. src/xz/args.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 31dbd1e5fb65831915a7bbb531c3f19aea8d57a5 Author: Lasse Collin Date: 2022-12-11 21:13:06 +0200 liblzma: Change LZMA_FILTER_ARM64 to the official Filter ID 0x0A. src/liblzma/api/lzma/bcj.h | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) commit 01b3549e523edac899ec4925b282ceddd20da116 Author: Lasse Collin Date: 2022-12-08 19:24:22 +0200 xz: Make args_info.files_name a const pointer. src/xz/args.c | 2 +- src/xz/args.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit bc665b84ea6bf7946394a08122177efe41b26a5f Author: Lasse Collin Date: 2022-12-08 19:18:16 +0200 xz: Don't modify argv[]. The code that parses --memlimit options and --block-list modified the argv[] when parsing the option string from optarg. This was visible in "ps auxf" and such and could be confusing. I didn't understand it back in the day when I wrote that code. Now a copy is allocated when modifiable strings are needed. src/xz/args.c | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) commit a13064e1c290de7933db72b6dffbd65cfce59c9f Author: Lasse Collin Date: 2022-12-08 18:18:57 +0200 Translations: Update the German man page translations. po4a/de.po | 4570 ++++++++++++++++++------------------------------------------ 1 file changed, 1374 insertions(+), 3196 deletions(-) commit 8bdbe42a8d0d75dff70206b923fc4bce5c69a40a Author: Jia Tan Date: 2022-12-06 23:05:56 +0800 Translations: Update the German translation. po/de.po | 586 ++++++++++++++++++++++++++++++++++----------------------------- 1 file changed, 315 insertions(+), 271 deletions(-) commit 5c304b57c24ef40ff57f864301065f0244c05bde Author: Jia Tan Date: 2022-12-06 23:04:25 +0800 Translations: Update the Turkish translation. po/tr.po | 221 +++++++++++++++++++++++++++++++-------------------------------- 1 file changed, 108 insertions(+), 113 deletions(-) commit 6d86781fdb937970486500447ebb49b98244235b Author: Jia Tan Date: 2022-12-06 23:02:11 +0800 Translations: Update the Croatian translation. po/hr.po | 228 +++++++++++++++++++++++++++++++-------------------------------- 1 file changed, 113 insertions(+), 115 deletions(-) commit 7a5b4b8075eb36026b1796f04ffed5830c42396a Author: Jia Tan Date: 2022-12-06 22:56:30 +0800 Translations: Add Romanian translation of man pages. Thanks to Remus-Gabriel Chelu. po4a/po4a.conf | 2 +- po4a/ro.po | 3692 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 3693 insertions(+), 1 deletion(-) commit c6977e7400088177556e8771bcb839eb7d90caa3 Author: Jia Tan Date: 2022-12-06 22:52:13 +0800 Translations: Update the Romanian translation. po/ro.po | 294 +++++++++++++++++++++++++++++++-------------------------------- 1 file changed, 147 insertions(+), 147 deletions(-) commit ac2a747e939c2cbccff7a49c399769af5e02d2ab Author: Lasse Collin Date: 2022-12-08 17:30:09 +0200 liblzma: Check for unexpected NULL pointers in block_header_decode(). The API docs gave an impression that such checks are done but they actually weren't done. In practice it made little difference since the calling code has a bug if these are NULL. Thanks to Jia Tan for the original patch that checked for block->filters == NULL. src/liblzma/common/block_header_decoder.c | 4 ++++ 1 file changed, 4 insertions(+) commit 24790f49ae66938c1c7574315e1c0aba1ed5ed25 Author: Lasse Collin Date: 2022-12-01 20:59:32 +0200 Bump version number for 5.3.5beta. This also sorts the symbol names alphabetically in liblzma_*.map. src/liblzma/api/lzma/version.h | 4 ++-- src/liblzma/liblzma_generic.map | 10 +++++----- src/liblzma/liblzma_linux.map | 10 +++++----- 3 files changed, 12 insertions(+), 12 deletions(-) commit 7e53c5bcb3c2c17f47c096c06ff6b1481e6ecafa Author: Lasse Collin Date: 2022-12-01 20:57:26 +0200 Add NEWS for 5.3.5beta. NEWS | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) commit 5865f2aaac326fcbd9f8a7d62defa230e4cb644e Author: Lasse Collin Date: 2022-12-01 20:57:09 +0200 Update THANKS. THANKS | 3 +++ 1 file changed, 3 insertions(+) commit 62b270988ec67314d69976df484d2974c6eacfda Author: Lasse Collin Date: 2022-12-01 20:04:17 +0200 liblzma: Use __has_attribute(__symver__) to fix Clang detection. If someone sets up Clang to define __GNUC__ to 10 or greater then symvers broke. __has_attribute is supported by such GCC and Clang versions that don't support __symver__ so this should be much better and simpler way to detect if __symver__ is actually supported. Thanks to Tomasz Gajc for the bug report. src/liblzma/common/common.h | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) commit f9ca7d45162664ddd9fb70e19335c2426e5d75bb Author: Lasse Collin Date: 2022-12-01 18:51:52 +0200 liblzma: Omit zero-skipping from ARM64 filter. It has some complicated downsides and its usefulness is more limited than I originally thought. So this change is bad for certain very specific situations but a generic solution that works for other filters (and is otherwise better too) is planned anyway. And this way 7-Zip can use the same compatible filter for the .7z format. This is still marked as experimental with a new temporary Filter ID. src/liblzma/api/lzma/bcj.h | 2 +- src/liblzma/simple/arm64.c | 81 +++++++++++++--------------------------------- 2 files changed, 24 insertions(+), 59 deletions(-) commit 5baec3f0a9c85e6abf45c0f652f699b074129a8b Author: Lasse Collin Date: 2022-12-01 18:13:27 +0200 xz: Omit the special notes about ARM64 filter on the man page. src/xz/xz.1 | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) commit 0c3627b51862eb0dcdd4fc283d046250571991c6 Author: Lasse Collin Date: 2022-12-01 18:12:03 +0200 liblzma: Don't be over-specific in lzma_str_to_filters API doc. src/liblzma/api/lzma/filter.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) commit 94adf057f27b1970f493dc99cd166407d7255639 Author: Lasse Collin Date: 2022-12-01 17:54:23 +0200 liblzma: Silence unused variable warning when BCJ filters are disabled. Thanks to Jia Tan for the original patch. src/liblzma/common/string_conversion.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) commit c68af4441744e5ffc41a472e1be9c9d53a1d9780 Author: Lasse Collin Date: 2022-12-01 17:38:03 +0200 Translations: Update the Chinese (simplified) translation. po/zh_CN.po | 608 ++++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 348 insertions(+), 260 deletions(-) commit 3be6942e5c27d29995d41da52fbe274e4ce4a537 Author: Lasse Collin Date: 2022-11-30 18:55:03 +0200 Add NEWS for 5.2.9. NEWS | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) commit 7c16e312cb2f40b81154c0e5be13a3c6b8da485d Author: Jia Tan Date: 2022-11-30 23:33:08 +0800 xz: Remove message_filters_to_str function prototype from message.h. This was forgotten from 7484744af6cbabe81e92af7d9e061dfd597fff7b. src/xz/message.h | 16 ---------------- 1 file changed, 16 deletions(-) commit 764955e2d4f2a5e8d6d6fec63af694f799e050e7 Author: Lasse Collin Date: 2022-11-30 18:08:34 +0200 Change the bug report address. It forwards to me and Jia Tan. Also update the IRC reference in README as #tukaani was moved to Libera Chat long ago. CMakeLists.txt | 2 +- README | 11 +++++------ configure.ac | 2 +- dos/config.h | 2 +- windows/README-Windows.txt | 2 +- 5 files changed, 9 insertions(+), 10 deletions(-) commit c21983c76031e01da01ad3c6cc716fe4b8a75070 Author: Lasse Collin Date: 2022-11-30 17:50:17 +0200 Build: Add string_conversion.c to CMake, DOS, and VS files. CMakeLists.txt | 1 + dos/Makefile | 1 + windows/vs2013/liblzma.vcxproj | 1 + windows/vs2013/liblzma_dll.vcxproj | 1 + windows/vs2017/liblzma.vcxproj | 1 + windows/vs2017/liblzma_dll.vcxproj | 1 + windows/vs2019/liblzma.vcxproj | 1 + windows/vs2019/liblzma_dll.vcxproj | 1 + 8 files changed, 8 insertions(+) commit 30be0c35d24eb5175459d69dbf7d92e2b087ef82 Author: Lasse Collin Date: 2022-11-30 17:38:32 +0200 Update to HTTPS URLs in AUTHORS. AUTHORS | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit 0a72b9ca2fe20082da9b7128fe0d908af947a851 Author: Jia Tan Date: 2022-11-30 00:52:06 +0800 liblzma: Improve documentation for string to filter functions. src/liblzma/api/lzma/filter.h | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) commit a6e21fcede3b196160a52dd294d965c508a4bb33 Author: Lasse Collin Date: 2022-11-29 22:27:42 +0200 liblzma: Two fixes to lzma_str_list_filters() API docs. Thanks to Jia Tan. src/liblzma/api/lzma/filter.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit 7484744af6cbabe81e92af7d9e061dfd597fff7b Author: Lasse Collin Date: 2022-11-28 21:57:47 +0200 xz: Use lzma_str_from_filters(). Two uses: Displaying encoder filter chain when compressing with -vv, and displaying the decoder filter chain in --list -vv. src/xz/list.c | 28 ++++++--- src/xz/message.c | 175 +++---------------------------------------------------- 2 files changed, 28 insertions(+), 175 deletions(-) commit cedeeca2ea6ada5b0411b2ae10d7a859e837f203 Author: Lasse Collin Date: 2022-11-28 21:37:48 +0200 liblzma: Add lzma_str_to_filters, _from_filters, and _list_filters. lzma_str_to_filters() uses static error messages which makes them not very precise. It tells the position in the string where an error occurred though which helps quite a bit if applications take advantage of it. Dynamic error messages can be added later with a new flag if it seems important enough. src/liblzma/api/lzma/filter.h | 258 +++++++ src/liblzma/common/Makefile.inc | 1 + src/liblzma/common/string_conversion.c | 1302 ++++++++++++++++++++++++++++++++ src/liblzma/liblzma_generic.map | 3 + src/liblzma/liblzma_linux.map | 3 + 5 files changed, 1567 insertions(+) commit 072ebf7b1335421193ffa9d4a70d5533786b8995 Author: Lasse Collin Date: 2022-11-28 21:02:19 +0200 liblzma: Make lzma_validate_chain() available outside filter_common.c. src/liblzma/common/filter_common.c | 8 ++++---- src/liblzma/common/filter_common.h | 3 +++ 2 files changed, 7 insertions(+), 4 deletions(-) commit 5f22bd2d37e3bd01a5d701b51750eb51f09c11bf Author: Lasse Collin Date: 2022-11-28 10:51:03 +0200 liblzma: Remove lzma_lz_decoder_uncompressed() as it's now unused. src/liblzma/lz/lz_decoder.c | 14 -------------- src/liblzma/lz/lz_decoder.h | 3 --- 2 files changed, 17 deletions(-) commit cee83206465b95729ab649aa2f57fdbde8dcaf89 Author: Lasse Collin Date: 2022-11-28 10:48:53 +0200 liblzma: Use LZMA1EXT feature in lzma_microlzma_decoder(). Here too this avoids the slightly ugly method to set the uncompressed size. Also moved the setting of dict_size to the struct initializer. src/liblzma/common/microlzma_decoder.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) commit e310e8b6a490dfb468f4ed68feff246d776b323c Author: Lasse Collin Date: 2022-11-28 10:28:20 +0200 liblzma: Use LZMA1EXT feature in lzma_alone_decoder(). This avoids the need to use the slightly ugly method to set the uncompressed size. src/liblzma/common/alone_decoder.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) commit 33b8a24b6646a9dbfd8358405aec466b13078559 Author: Lasse Collin Date: 2022-11-27 23:16:21 +0200 liblzma: Add LZMA_FILTER_LZMA1EXT to support LZMA1 without end marker. Some file formats need support for LZMA1 streams that don't use the end of payload marker (EOPM) alias end of stream (EOS) marker. So far liblzma API has supported decompressing such streams via lzma_alone_decoder() when .lzma header specifies a known uncompressed size. Encoding support hasn't been available in the API. Instead of adding a new LZMA1-only API for this purpose, this commit adds a new filter ID for use with raw encoder and decoder. The main benefit of this approach is that then also filter chains are possible, for example, if someone wants to implement support for .7z files that use the x86 BCJ filter with LZMA1 (not BCJ2 as that isn't supported in liblzma). src/liblzma/api/lzma/lzma12.h | 123 ++++++++++++++++++++++++++++++-- src/liblzma/common/filter_common.c | 7 ++ src/liblzma/common/filter_decoder.c | 6 ++ src/liblzma/common/filter_encoder.c | 9 +++ src/liblzma/lzma/lzma2_encoder.c | 2 +- src/liblzma/lzma/lzma_decoder.c | 26 ++++++- src/liblzma/lzma/lzma_encoder.c | 40 +++++++++-- src/liblzma/lzma/lzma_encoder.h | 3 +- src/liblzma/lzma/lzma_encoder_private.h | 3 + 9 files changed, 204 insertions(+), 15 deletions(-) commit 9a304bf1e45b3ddf61aaeaa7c764915b34618ede Author: Lasse Collin Date: 2022-11-27 18:43:07 +0200 liblzma: Avoid unneeded use of void pointer in LZMA decoder. src/liblzma/lzma/lzma_decoder.c | 3 +-- src/liblzma/lzma/lzma_decoder.h | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) commit 218394958c7683f892275bb40eae880620feebcc Author: Lasse Collin Date: 2022-11-27 18:20:33 +0200 liblzma: Pass the Filter ID to LZ encoder and decoder. This allows using two Filter IDs with the same initialization function and data structures. src/liblzma/common/alone_decoder.c | 1 + src/liblzma/common/alone_encoder.c | 1 + src/liblzma/common/common.h | 7 +++++-- src/liblzma/common/lzip_decoder.c | 1 + src/liblzma/common/microlzma_decoder.c | 1 + src/liblzma/common/microlzma_encoder.c | 1 + src/liblzma/lz/lz_decoder.c | 5 +++-- src/liblzma/lz/lz_decoder.h | 3 ++- src/liblzma/lz/lz_encoder.c | 5 +++-- src/liblzma/lz/lz_encoder.h | 3 ++- src/liblzma/lzma/lzma2_decoder.c | 3 ++- src/liblzma/lzma/lzma2_encoder.c | 3 ++- src/liblzma/lzma/lzma_decoder.c | 2 +- src/liblzma/lzma/lzma_encoder.c | 2 +- 14 files changed, 26 insertions(+), 12 deletions(-) commit 1663c7676b76f4c514031797f3db1896e8100f7f Author: Lasse Collin Date: 2022-11-27 01:03:16 +0200 liblzma: Remove two FIXME comments. src/liblzma/common/filter_encoder.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 11fe708db783ac36ebeeb85da164e29e8c300910 Author: Lasse Collin Date: 2022-11-26 22:25:30 +0200 xz: Use lzma_filters_free(). src/xz/list.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) commit e782af9110d8499c7ac2929bc871540eefea5ea1 Author: Lasse Collin Date: 2022-11-26 22:21:13 +0200 liblzma: Use lzma_filters_free() in more places. src/liblzma/common/block_header_decoder.c | 20 ++------------------ src/liblzma/common/stream_decoder.c | 4 +--- src/liblzma/common/stream_decoder_mt.c | 22 +++++----------------- 3 files changed, 8 insertions(+), 38 deletions(-) commit 90caaded2dc6db1d6a55b01160d7e87f4a423628 Author: Lasse Collin Date: 2022-11-25 18:04:37 +0200 liblzma: Omit simple coder init functions if they are disabled. src/liblzma/simple/arm.c | 4 ++++ src/liblzma/simple/armthumb.c | 4 ++++ src/liblzma/simple/ia64.c | 4 ++++ src/liblzma/simple/powerpc.c | 4 ++++ src/liblzma/simple/sparc.c | 4 ++++ src/liblzma/simple/x86.c | 4 ++++ 6 files changed, 24 insertions(+) commit 5cd9f0df78cc4f8a7807bf6104adea13034fbb45 Author: Lasse Collin Date: 2022-11-24 23:24:59 +0200 xz: Allow nice_len 2 and 3 even if match finder requires 3 or 4. Now that liblzma accepts these, we avoid the extra check and there's one message less for translators too. src/xz/options.c | 5 ----- 1 file changed, 5 deletions(-) commit 3be88ae071371caa279b44e13f4836fb178fe4ae Author: Lasse Collin Date: 2022-11-24 23:23:55 +0200 liblzma: Allow nice_len 2 and 3 even if match finder requires 3 or 4. That is, if the specified nice_len is smaller than the minimum of the match finder, silently use the match finder's minimum value instead of reporting an error. The old behavior is annoying to users and it complicates xz options handling too. src/liblzma/lz/lz_encoder.c | 14 +++++++++----- src/liblzma/lz/lz_encoder.h | 9 +++++++++ src/liblzma/lzma/lzma_encoder.c | 11 ++++++++--- 3 files changed, 26 insertions(+), 8 deletions(-) commit 93439cfafe1768b3b18d67d2356ef7e7559bba59 Author: Lasse Collin Date: 2022-11-24 16:25:10 +0200 liblzma: Add lzma_filters_update() support to the multi-threaded encoder. A tiny downside of this is that now a 1-4 tiny allocations are made for every Block because each worker thread needs its own copy of the filter chain. src/liblzma/api/lzma/filter.h | 36 +++++++------ src/liblzma/common/stream_encoder_mt.c | 96 +++++++++++++++++++++++++++++++--- 2 files changed, 109 insertions(+), 23 deletions(-) commit 17ac51e689794eb41cab3e80946fec689caea2d2 Author: Lasse Collin Date: 2022-11-24 14:53:22 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 06824396b2b6c84f3a235cb7c19c2a9701167797 Author: Lasse Collin Date: 2022-11-24 14:52:44 +0200 Build: Don't put GNU/Linux-specific symbol versions into static liblzma. It not only makes no sense to put symbol versions into a static library but it can also cause breakage. By default Libtool #defines PIC if building a shared library and doesn't define it for static libraries. This is documented in the Libtool manual. It can be overriden using --with-pic or --without-pic. configure.ac detects if --with-pic or --without-pic is used and then gives an error if neither --disable-shared nor --disable-static was used at the same time. Thus, in normal situations it works to build both shared and static library at the same time on GNU/Linux, only --with-pic or --without-pic requires that only one type of library is built. Thanks to John Paul Adrian Glaubitz from Debian for reporting the problem that occurred on ia64: https://www.mail-archive.com/xz-devel@tukaani.org/msg00610.html CMakeLists.txt | 5 +- configure.ac | 143 +++++++++++++++++++++++++++++--------------- src/liblzma/common/common.h | 12 ++++ 3 files changed, 111 insertions(+), 49 deletions(-) commit e1acf7107291f8b3d6d609a7133331ff36d35d14 Author: Lasse Collin Date: 2022-11-24 01:32:16 +0200 liblzma: Refactor to use lzma_filters_free(). lzma_filters_free() sets the options to NULL and ids to LZMA_VLI_UNKNOWN so there is no need to do it by caller; the filter arrays will always be left in a safe state. Also use memcpy() instead of a loop to copy a filter chain when it is known to be safe to copy LZMA_FILTERS_MAX + 1 (even if the elements past the terminator might be uninitialized). src/liblzma/common/stream_encoder.c | 16 ++++------------ src/liblzma/common/stream_encoder_mt.c | 11 ++--------- 2 files changed, 6 insertions(+), 21 deletions(-) commit cb05dbcf8b868441ec805016222f3fd77f1c5caa Author: Lasse Collin Date: 2022-11-24 01:26:37 +0200 liblzma: Fix another invalid free() after memory allocation failure. This time it can happen when lzma_stream_encoder_mt() is used to reinitialize an existing multi-threaded Stream encoder and one of 1-4 tiny allocations in lzma_filters_copy() fail. It's very similar to the previous bug 10430fbf3820dafd4eafd38ec8be161a6978ed2b, happening with an array of lzma_filter structures whose old options are freed but the replacement never arrives due to a memory allocation failure in lzma_filters_copy(). src/liblzma/common/stream_encoder_mt.c | 4 ++++ 1 file changed, 4 insertions(+) commit 75f1a6c26df4ce329da0882786403e3ccf5cd898 Author: Jia Tan Date: 2022-05-05 20:53:42 +0800 liblzma: Add support for LZMA_SYNC_FLUSH in the Block encoder. The documentation mentions that lzma_block_encoder() supports LZMA_SYNC_FLUSH but it was never added to supported_actions[] in the internal structure. Because of this, LZMA_SYNC_FLUSH could not be used with the Block encoder unless it was the next coder after something like stream_encoder() or stream_encoder_mt(). src/liblzma/common/block_encoder.c | 1 + 1 file changed, 1 insertion(+) commit d0901645170b638c517f5c50866b6ef48f491c65 Author: Lasse Collin Date: 2022-11-24 01:02:50 +0200 liblzma: Add new API function lzma_filters_free(). This is small but convenient and should have been added a long time ago. src/liblzma/api/lzma/filter.h | 21 +++++++++++++++++++++ src/liblzma/common/filter_common.c | 26 ++++++++++++++++++++++++++ src/liblzma/liblzma_generic.map | 1 + src/liblzma/liblzma_linux.map | 1 + 4 files changed, 49 insertions(+) commit ae1f8a723dcde2f2c5cf444bcbb5fc5026b3c3c5 Author: Lasse Collin Date: 2022-11-24 00:02:31 +0200 CMake: Don't use symbol versioning with static library. CMakeLists.txt | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) commit 48c1b99dc537a27e1ca929d8837e778e5ba32191 Author: Lasse Collin Date: 2022-11-23 21:55:22 +0200 liblzma: Add lzma_attr_warn_unused_result to lzma_filters_copy(). src/liblzma/api/lzma/filter.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 10430fbf3820dafd4eafd38ec8be161a6978ed2b Author: Lasse Collin Date: 2022-11-23 21:26:21 +0200 liblzma: Fix invalid free() after memory allocation failure. The bug was in the single-threaded .xz Stream encoder in the code that is used for both re-initialization and for lzma_filters_update(). To trigger it, an application had to either re-initialize an existing encoder instance with lzma_stream_encoder() or use lzma_filters_update(), and then one of the 1-4 tiny allocations in lzma_filters_copy() (called from stream_encoder_update()) must fail. An error was correctly reported but the encoder state was corrupted. This is related to the recent fix in f8ee61e74eb40600445fdb601c374d582e1e9c8a which is good but it wasn't enough to fix the main problem in stream_encoder.c. src/liblzma/common/stream_encoder.c | 39 +++++++++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 8 deletions(-) commit cafd6dc397ca8b5b5f7775e8d6876b8fe70f8e70 Author: Lasse Collin Date: 2022-11-22 16:37:15 +0200 liblzma: Fix language in a comment. src/liblzma/common/stream_encoder.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit c392bf8ccba857baaf50399c4b460119befacd54 Author: Lasse Collin Date: 2022-11-22 11:20:17 +0200 liblzma: Fix infinite loop in LZMA encoder init with dict_size >= 2 GiB. The encoder doesn't support dictionary sizes larger than 1536 MiB. This is validated, for example, when calculating the memory usage via lzma_raw_encoder_memusage(). It is also enforced by the LZ part of the encoder initialization. However, LZMA encoder with LZMA_MODE_NORMAL did an unsafe calculation with dict_size before such validation and that results in an infinite loop if dict_size was 2 << 30 or greater. src/liblzma/lzma/lzma_encoder.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) commit f50534c973a591ccf65485adfc827a8a7126ca6c Author: Lasse Collin Date: 2022-11-21 13:02:33 +0200 liblzma: Fix two Doxygen commands in the API headers. These were caught by clang -Wdocumentation. src/liblzma/api/lzma/hardware.h | 2 +- src/liblzma/api/lzma/index_hash.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit 649d4872ed2f55196114a061d45b416fc4353569 Author: Lasse Collin Date: 2022-11-19 19:09:55 +0200 xz: Refactor duplicate code from hardware_memlimit_mtenc_get(). src/xz/hardware.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit d327743bb547a53364e5951a16e5f1663fe4b9ff Author: Lasse Collin Date: 2022-11-19 19:06:13 +0200 xz: Add support --threads=+N so that -T+1 gives threaded mode. src/xz/args.c | 18 +++++++++++++++--- src/xz/hardware.c | 17 +++++++++++++++-- src/xz/hardware.h | 1 + src/xz/xz.1 | 21 ++++++++++++++++++++- 4 files changed, 51 insertions(+), 6 deletions(-) commit a11a2b8b5e830ba682c1d81aaa7078842b296995 Author: Jia Tan Date: 2022-11-19 23:18:04 +0800 CMake: Adds test_memlimit to CMake tests CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) commit 2af8d9e9b3f44f62d19e7c39297ec63af2e8c64f Author: Lasse Collin Date: 2022-11-15 19:10:21 +0200 Translations: Update the Korean translation. po/ko.po | 652 ++++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 371 insertions(+), 281 deletions(-) commit 16ac05677292f7e21a4feaddcfb2ab062ea5f385 Author: Lasse Collin Date: 2022-11-15 19:09:28 +0200 Translations: Update the Turkish translation. po/tr.po | 568 ++++++++++++++++++++++++++++++++++----------------------------- 1 file changed, 310 insertions(+), 258 deletions(-) commit b9a67d9a5fa207062d4aa8a01639234609315d31 Author: Lasse Collin Date: 2022-11-15 10:58:39 +0200 Bump version number for 5.3.4alpha. src/liblzma/api/lzma/version.h | 2 +- src/liblzma/liblzma_generic.map | 2 +- src/liblzma/liblzma_linux.map | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) commit 5b999ba289b3280457b7386b9ac65dbbdf1575a5 Author: Lasse Collin Date: 2022-11-15 10:54:40 +0200 Add NEWS for 5.3.4alpha. NEWS | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) commit ce8db9e37da4f6c87691c5066f51f91f2411c44a Author: Lasse Collin Date: 2022-11-15 10:54:08 +0200 Add NEWS for 5.2.8. NEWS | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) commit b56bc8251d2736224af6bdaaae734ceb8926a879 Author: Lasse Collin Date: 2022-11-14 23:19:57 +0200 Revert "liblzma: Simple/BCJ filters: Allow disabling generic BCJ options." This reverts commit 177bdc922cb17bd0fd831ab8139dfae912a5c2b8 and also does equivalent change to arm64.c. Now that ARM64 filter will use lzma_options_bcj, this change is not needed anymore. src/liblzma/simple/arm.c | 2 +- src/liblzma/simple/arm64.c | 2 +- src/liblzma/simple/armthumb.c | 2 +- src/liblzma/simple/ia64.c | 2 +- src/liblzma/simple/powerpc.c | 2 +- src/liblzma/simple/simple_coder.c | 4 ++-- src/liblzma/simple/simple_private.h | 2 +- src/liblzma/simple/sparc.c | 2 +- src/liblzma/simple/x86.c | 3 +-- 9 files changed, 10 insertions(+), 11 deletions(-) commit 8370ec8edf9ddf8d1d9fef03d8d1027503ec4c35 Author: Lasse Collin Date: 2022-11-14 23:14:41 +0200 Replace the experimental ARM64 filter with a new experimental version. This is incompatible with the previous version. This has space/tab fixes in filter_*.c and bcj.h too. src/liblzma/api/lzma/bcj.h | 41 +----- src/liblzma/common/filter_common.c | 14 +- src/liblzma/common/filter_decoder.c | 12 +- src/liblzma/common/filter_encoder.c | 17 +-- src/liblzma/simple/arm64.c | 283 ++++++++++++++---------------------- src/liblzma/simple/simple_decoder.h | 4 - src/liblzma/simple/simple_encoder.h | 2 - src/xz/args.c | 2 +- src/xz/message.c | 13 +- src/xz/options.c | 39 ----- src/xz/options.h | 7 - 11 files changed, 147 insertions(+), 287 deletions(-) commit f644473a211394447824ea00518d0a214ff3f7f2 Author: Lasse Collin Date: 2022-11-14 21:34:57 +0200 liblzma: Add fast CRC64 for 32/64-bit x86 using SSSE3 + SSE4.1 + CLMUL. It also works on E2K as it supports these intrinsics. On x86-64 runtime detection is used so the code keeps working on older processors too. A CLMUL-only build can be done by using -msse4.1 -mpclmul in CFLAGS and this will reduce the library size since the generic implementation and its 8 KiB lookup table will be omitted. On 32-bit x86 this isn't used by default for now because by default on 32-bit x86 the separate assembly file crc64_x86.S is used. If --disable-assembler is used then this new CLMUL code is used the same way as on 64-bit x86. However, a CLMUL-only build (-msse4.1 -mpclmul) won't omit the 8 KiB lookup table on 32-bit x86 due to a currently-missing check for disabled assembler usage. The configure.ac check should be such that the code won't be built if something in the toolchain doesn't support it but --disable-clmul-crc option can be used to unconditionally disable this feature. CLMUL speeds up decompression of files that have compressed very well (assuming CRC64 is used as a check type). It is know that the CLMUL code is significantly slower than the generic code for tiny inputs (especially 1-8 bytes but up to 16 bytes). If that is a real-world problem then there is already a commented-out variant that uses the generic version for small inputs. Thanks to Ilya Kurdyukov for the original patch which was derived from a white paper from Intel [1] (published in 2009) and public domain code from [2] (released in 2016). [1] https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-crc-computation-generic-polynomials-pclmulqdq-paper.pdf [2] https://github.com/rawrunprotected/crc CMakeLists.txt | 26 ++- INSTALL | 12 ++ configure.ac | 59 +++++- src/liblzma/check/crc64_fast.c | 449 +++++++++++++++++++++++++++++++++++++++- src/liblzma/check/crc64_table.c | 21 +- 5 files changed, 554 insertions(+), 13 deletions(-) commit 3b466bc79672bb2b06d1245a500588e6026e0ba0 Author: Lasse Collin Date: 2022-11-14 20:14:34 +0200 Translations: Update the Swedish translation one more time. po/sv.po | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit e963379a8622ebdff6ce78e76b803bcd1e1d16d6 Author: Lasse Collin Date: 2022-11-14 19:34:15 +0200 Translations: Update the Swedish translation again. po/sv.po | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) commit a4bc689a823a2254f29ac9d233170add5121b307 Author: Lasse Collin Date: 2022-11-14 19:07:45 +0200 Translations: Update the Swedish translation. po/sv.po | 671 ++++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 382 insertions(+), 289 deletions(-) commit bbf2073d824ab4ba33bed4b77f467435abd333a5 Author: Lasse Collin Date: 2022-11-14 18:58:09 +0200 Translations: Update the Ukrainian translation. po/uk.po | 618 ++++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 354 insertions(+), 264 deletions(-) commit ac10b1b3622e70881595586edfb8a3ebdcd76bb6 Author: Lasse Collin Date: 2022-11-14 17:58:07 +0200 Build: Omit x86_64 from --enable-assembler. It didn't do anything. There are only 32-bit x86 assembly files and it feels likely that new files won't be added as intrinsics in C are more portable across toolchains and OSes. configure.ac | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) commit eb0f1450ad9f23dac03050d9c8375980240aee21 Author: Lasse Collin Date: 2022-11-14 16:00:52 +0200 liblzma: Use __attribute__((__constructor__)) if available. This uses it for CRC table initializations when using --disable-small. It avoids mythread_once() overhead. It also means that then --disable-small --disable-threads is thread-safe if this attribute is supported. CMakeLists.txt | 15 +++++++++++++++ INSTALL | 4 +++- configure.ac | 31 ++++++++++++++++++++++++++++--- src/liblzma/check/crc32_small.c | 7 +++++++ src/liblzma/check/crc64_small.c | 5 +++++ src/liblzma/lz/lz_encoder.c | 2 +- 6 files changed, 59 insertions(+), 5 deletions(-) commit 6553f49b11dafad35c73b05f12e14865ea1fd8a1 Author: Lasse Collin Date: 2022-11-12 21:19:52 +0200 Translations: Update the Romanian translation. po/ro.po | 651 +++++++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 380 insertions(+), 271 deletions(-) commit db97e69e12393becc29f8febd53133d0d36989bd Author: Lasse Collin Date: 2022-11-12 21:17:45 +0200 Translations: Update the Hungarian translation. po/hu.po | 625 ++++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 357 insertions(+), 268 deletions(-) commit 2bbb9c0f3829a8b121b36998d273a6c6f92000f4 Author: Lasse Collin Date: 2022-11-11 17:58:57 +0200 Translations: Update the Finnish translation. po/fi.po | 610 ++++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 348 insertions(+), 262 deletions(-) commit 3c8cbb8137b6f8ed9416c1209d73cdbcb015251f Author: Lasse Collin Date: 2022-11-11 17:58:18 +0200 Translations: Update the Croatian translation. po/hr.po | 680 +++++++++++++++++++++++++++++++++++---------------------------- 1 file changed, 381 insertions(+), 299 deletions(-) commit 26c3359eac0988d6f3986735cd1363bec1678e8e Author: Lasse Collin Date: 2022-11-11 17:57:18 +0200 Translations: Update the Polish translation. po/pl.po | 569 ++++++++++++++++++++++++++++++++++----------------------------- 1 file changed, 309 insertions(+), 260 deletions(-) commit 577e467b137c735afb8de6ae71ac7a73c2960cc4 Author: Lasse Collin Date: 2022-11-11 17:56:44 +0200 Translations: Update the Spanish translation. po/es.po | 598 ++++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 344 insertions(+), 254 deletions(-) commit f9b4ff6e9a0f1678650775582d3e4fe782abce97 Author: Lasse Collin Date: 2022-11-11 17:16:03 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit a39961ef211e1bf030b17edeea3cff29fe263b67 Author: Lasse Collin Date: 2022-11-11 17:15:25 +0200 liblzma: Fix building with Intel ICC (the classic compiler). It claims __GNUC__ >= 10 but doesn't support __symver__ attribute. Thanks to Stephen Sachs. src/liblzma/common/common.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit c715f683dcb1a817d565da292cddfbceda643e12 Author: Lasse Collin Date: 2022-11-11 14:35:58 +0200 liblzma: Fix incorrect #ifdef for x86 SSE2 support. __SSE2__ is the correct macro for SSE2 support with GCC, Clang, and ICC. __SSE2_MATH__ means doing floating point math with SSE2 instead of 387. Often the latter macro is defined if the first one is but it was still a bug. src/liblzma/common/memcmplen.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) commit bd334ae56afe7f642ad4d0f1ac19e74e82daa1ce Author: Lasse Collin Date: 2022-11-11 13:27:06 +0200 Add NEWS for 5.2.7 (forgotten cherry-pick from v5.2). NEWS | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) commit 3c7860cf49de6f81046b3a4034a89f3a4803a576 Author: Lasse Collin Date: 2022-11-11 13:16:21 +0200 xzdiff: Add support for .lz files. The other scripts don't need changes for .lz support because in those scripts it is enough that xz supports .lz. src/scripts/xzdiff.in | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) commit d76c752a6d77052e5ad57ade555082585f7ac5d8 Author: Lasse Collin Date: 2022-11-11 12:23:58 +0200 Scripts: Ignore warnings from xz. In practice this means making the scripts work when the input files have an unsupported check type which isn't a problem in practice unless support for some check types has been disabled at build time. src/scripts/xzdiff.in | 5 +++-- src/scripts/xzgrep.in | 2 +- src/scripts/xzless.in | 4 ++-- src/scripts/xzmore.in | 4 ++-- 4 files changed, 8 insertions(+), 7 deletions(-) commit 6552535afd1fe29d726ab6e68cf14ce3624fd48c Author: Lasse Collin Date: 2022-11-10 12:34:43 +0200 Translations: Rename poa4/fr_FR.po to po4a/fr.po. That's how it is preferred at the Translation Project. On my system /usr/share/man/fr_FR doesn't contain any other man pages than XZ Utils while /usr/share/man/fr has quite a few, so this will fix that too. Thanks to Benno Schulenberg from the Translation Project. po4a/{fr_FR.po => fr.po} | 0 po4a/po4a.conf | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) commit 0918159ce4c75bfb60aff0193b559f8a9f41d25a Author: Lasse Collin Date: 2022-11-09 18:48:50 +0200 xz: Update the man page about BCJ filters, including upcoming --arm64. The --arm64 isn't actually implemented yet in the form described in this commit. Thanks to Jia Tan. src/xz/xz.1 | 66 +++++++++++++++++++++++++++---------------------------------- 1 file changed, 29 insertions(+), 37 deletions(-) commit ba2ae3596f6be1587495f33b367488f6e00e56f1 Author: Lasse Collin Date: 2022-11-09 18:14:14 +0200 xz: Add --arm64 to --long-help and omit endianness from ARM(-Thumb). Modern 32-bit ARM in big endian mode use little endian for instruction encoding still, so the filters work on such executables too. It's likely less confusing for users this way. The --arm64 option hasn't been implemented yet (there is --experimental-arm64 but it's different). The --arm64 option is added now anyway because this is the likely result and the strings need to be ready for translators. Thanks to Jia Tan. src/xz/message.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) commit 802d57d9215d9c81dbee86edb43c9e93a7f7ec55 Author: Lasse Collin Date: 2022-11-09 15:12:13 +0200 Windows: Update the VS project files for ARM64 and .lz support. windows/vs2013/config.h | 9 +++++++++ windows/vs2013/liblzma.vcxproj | 5 ++++- windows/vs2013/liblzma_dll.vcxproj | 5 ++++- windows/vs2017/config.h | 9 +++++++++ windows/vs2017/liblzma.vcxproj | 3 +++ windows/vs2017/liblzma_dll.vcxproj | 3 +++ windows/vs2019/config.h | 9 +++++++++ windows/vs2019/liblzma.vcxproj | 5 ++++- windows/vs2019/liblzma_dll.vcxproj | 5 ++++- 9 files changed, 49 insertions(+), 4 deletions(-) commit 5846aeda05972bc803c6094821ae836229ebe691 Author: Lasse Collin Date: 2022-11-09 14:57:48 +0200 DOS: Update Makefile and config.h to include ARM64 and .lz support. dos/Makefile | 2 ++ dos/config.h | 9 +++++++++ 2 files changed, 11 insertions(+) commit 781da8d6c44de6aa278c916375250668a0b107f2 Author: Lasse Collin Date: 2022-11-09 14:45:05 +0200 CMake: Add lzip decoder files and #define to the build. CMakeLists.txt | 3 +++ 1 file changed, 3 insertions(+) commit df8ad4af65a9c4846b108550d0083770a69dee64 Author: Lasse Collin Date: 2022-11-09 14:41:56 +0200 Docs: Update INSTALL and also add new prohibited options to PACKAGERS. INSTALL | 49 +++++++++++++++++++++++++++++++++++++++++-------- PACKAGERS | 2 ++ 2 files changed, 43 insertions(+), 8 deletions(-) commit c8ef089c149afaab413c3a51be827dd1d11afe0e Author: Lasse Collin Date: 2022-10-20 17:39:06 +0300 Tests: Test the .lz files in test_files.sh. tests/test_files.sh | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) commit c8f70ebb4628ceb6cb29cc9195d9deadf69d2bd7 Author: Lasse Collin Date: 2022-10-20 15:35:59 +0300 Tests: Add .lz (lzip) test files. tests/files/README | 109 +++++++++++++++++++++++++++++---- tests/files/bad-1-v0-uncomp-size.lz | Bin 0 -> 42 bytes tests/files/bad-1-v1-crc32.lz | Bin 0 -> 50 bytes tests/files/bad-1-v1-dict-1.lz | Bin 0 -> 50 bytes tests/files/bad-1-v1-dict-2.lz | Bin 0 -> 50 bytes tests/files/bad-1-v1-magic-1.lz | Bin 0 -> 50 bytes tests/files/bad-1-v1-magic-2.lz | Bin 0 -> 50 bytes tests/files/bad-1-v1-member-size.lz | Bin 0 -> 50 bytes tests/files/bad-1-v1-trailing-magic.lz | Bin 0 -> 54 bytes tests/files/bad-1-v1-uncomp-size.lz | Bin 0 -> 50 bytes tests/files/good-1-v0-trailing-1.lz | Bin 0 -> 59 bytes tests/files/good-1-v0.lz | Bin 0 -> 42 bytes tests/files/good-1-v1-trailing-1.lz | Bin 0 -> 67 bytes tests/files/good-1-v1-trailing-2.lz | Bin 0 -> 70 bytes tests/files/good-1-v1.lz | Bin 0 -> 50 bytes tests/files/good-2-v0-v1.lz | Bin 0 -> 78 bytes tests/files/good-2-v1-v0.lz | Bin 0 -> 78 bytes tests/files/good-2-v1-v1.lz | Bin 0 -> 86 bytes tests/files/unsupported-1-v234.lz | Bin 0 -> 50 bytes 19 files changed, 98 insertions(+), 11 deletions(-) commit 731db13e6fa3ad3e3fc786c0ccf6eac4cce6865f Author: Lasse Collin Date: 2022-10-19 22:32:51 +0300 xz: Remove the commented-out FORMAT_GZIP, gzip, .gz, and .tgz. src/xz/args.c | 2 -- src/xz/coder.h | 1 - src/xz/suffix.c | 9 --------- 3 files changed, 12 deletions(-) commit 3176f992c55b8d788c4633809aaf9447376a5a12 Author: Lasse Collin Date: 2022-10-08 21:28:15 +0300 xz: Add .lz (lzip) decompression support. If configured with --disable-lzip-decoder then --long-help will still list `lzip' in --format but I left it like that since due to translations it would be messy to have two help strings. Features are disabled only in special situations so wrong help in such a situation shouldn't matter much. Thanks to Michał Górny for the original patch. src/xz/args.c | 9 ++++++++ src/xz/coder.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++--- src/xz/coder.h | 3 +++ src/xz/message.c | 2 +- src/xz/suffix.c | 26 ++++++++++++++++++---- src/xz/xz.1 | 46 +++++++++++++++++++++++++++++++++----- 6 files changed, 141 insertions(+), 13 deletions(-) commit 034086e1ae1459210837a24e04878435c86dc41b Author: Lasse Collin Date: 2022-10-08 00:29:20 +0300 liblzma: Add .lz support to lzma_auto_decoder(). Thanks to Michał Górny for the original patch. src/liblzma/api/lzma/container.h | 10 ++++++---- src/liblzma/common/Makefile.inc | 3 ++- src/liblzma/common/auto_decoder.c | 23 +++++++++++++++++------ src/liblzma/common/lzip_decoder.h | 22 ++++++++++++++++++++++ 4 files changed, 47 insertions(+), 11 deletions(-) commit 0538db038f3cdc352007dacb42454aa1806b8e40 Author: Lasse Collin Date: 2022-10-06 15:50:20 +0300 liblzma: Add .lz (lzip) decompression support (format versions 0 and 1). Support for format version 0 was removed from lzip 1.18 for some reason. .lz format version 0 files are rare (and old) but some source packages were released in this format, and some people might have personal files in this format too. It's very little extra code to support it along side format version 1 so this commits adds support for both. The Sync Flush marker extentension to the original .lz format version 1 isn't supported. It would require changes to the LZMA decoder itself. Such files are very rare anyway. See the API doc for lzma_lzip_decoder() for more details about the .lz format support. Thanks to Michał Górny for the original patch. configure.ac | 21 ++ src/liblzma/api/lzma/container.h | 62 +++++- src/liblzma/common/Makefile.inc | 5 + src/liblzma/common/lzip_decoder.c | 413 ++++++++++++++++++++++++++++++++++++++ src/liblzma/liblzma_generic.map | 1 + src/liblzma/liblzma_linux.map | 1 + 6 files changed, 501 insertions(+), 2 deletions(-) commit 633d48a075b9ce4b9c08a7a56a7eb4cabc18100c Author: Lasse Collin Date: 2022-11-09 14:17:23 +0200 liblzma: Add the missing Makefile.inc change for --disable-microlzma. This was forgotten from commit 59c4d6e1390f6f4176f43ac1dad1f7ac03c449b8. src/liblzma/common/Makefile.inc | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) commit 724285dadbdc88765c8fb83eab9816575a260966 Author: Lasse Collin Date: 2022-11-09 14:10:52 +0200 xz: Add comments about stdin and src_st.st_size. "xz -v < regular_file > out.xz" doesn't display the percentage and estimated remaining time because it doesn't even try to check the input file size when input is read from stdin. This could be improved but for now there's just a comment to remind about it. src/xz/coder.c | 9 +++++++++ src/xz/file_io.c | 4 ++++ 2 files changed, 13 insertions(+) commit f723eec68b0e44234910f669a29119de33018967 Author: Lasse Collin Date: 2022-11-09 12:48:22 +0200 xz: Fix displaying of file sizes in progress indicator in passthru mode. It worked for one input file since the counters are zero when xz starts but they weren't reset when starting a new file in passthru mode. For example, if files A, B, and C are one byte each, then "xz -dcvf A B C" would show file sizes as 1, 2, and 3 bytes instead of 1, 1, and 1 byte. src/xz/coder.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) commit 69265d0f223ddf1d66f799b8b047df22923e376f Author: Lasse Collin Date: 2022-11-09 11:27:20 +0200 xz: Add a comment why --to-stdout is not in --help. It is on the man page still. src/xz/message.c | 3 +++ 1 file changed, 3 insertions(+) commit fe6b8852a3c6a0eb5a3c33512e0a69af257d3bc7 Author: Lasse Collin Date: 2022-11-08 23:05:37 +0200 xz: Make xz -lvv show that the upcoming --arm64 needs 5.4.0 to decompress. src/xz/list.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) commit fb3f05ac9f2b4b0e3643401960fbeab31997ac7a Author: Lasse Collin Date: 2022-11-08 22:26:54 +0200 Docs: Update faq.txt a little. doc/faq.txt | 66 ++++++++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 43 insertions(+), 23 deletions(-) commit 05331f091ec3b68eccbfb2a9a7a576072768fb4b Author: Lasse Collin Date: 2022-11-08 16:57:17 +0200 Translations: Update Turkish translation. po/tr.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit ed3a4822963b4940d84e6f44d47277c394fc046d Author: Lasse Collin Date: 2022-11-08 14:55:32 +0200 Translations: Update Croatian translation. po/hr.po | 190 ++++++++++++++++++++++++++++++++------------------------------- 1 file changed, 96 insertions(+), 94 deletions(-) commit 4746f5ec721316bc4c6fec9905b2902e0360e0af Author: Lasse Collin Date: 2022-11-08 14:13:03 +0200 liblzma: Update API docs about decoder flags. src/liblzma/api/lzma/container.h | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) commit 8779a9db5d0cec00c9dc9e9965dd2dda04f9d80d Author: Lasse Collin Date: 2022-11-08 14:01:50 +0200 liblzma: Use the return_if_error() macro in alone_decoder.c. src/liblzma/common/alone_decoder.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) commit 3f4990b6822961e75cd9b4e2e82b1df63f6f8fcc Author: Lasse Collin Date: 2022-11-08 14:00:58 +0200 liblzma: Fix a comment in auto_decoder.c. src/liblzma/common/auto_decoder.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 026a5897c72a2041ae08ceec54ce8b1cdeb51334 Author: Lasse Collin Date: 2022-11-08 13:43:19 +0200 xz: Initialize the pledge(2) sandbox at the very beginning of main(). It feels better that the initializations are sandboxed too. They don't do anything that the pledge() call wouldn't allow. src/xz/main.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) commit 49a59f6ca001c3ce9affa2c162b437aad021b4d5 Author: Lasse Collin Date: 2022-11-07 22:51:16 +0200 xz: Extend --robot --info-memory output. Now it includes everything that the human-readable --info-memory shows. src/xz/hardware.c | 24 +++++++++++++++--------- src/xz/xz.1 | 47 +++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 56 insertions(+), 15 deletions(-) commit 5e2450c75cbac966c62cf2231c824f2cc91ddba8 Author: Lasse Collin Date: 2022-11-07 17:22:04 +0200 liblzma: Include cached memory in reported memusage in threaded decoder. This affects lzma_memusage() and lzma_memlimit_set() when used with the threaded decompressor. Now all allocations are reported by lzma_memusage() (so it's not misleading) and lzma_memlimit_set() cannot lower the limit below that value. The alternative would have been to allow lowering the limit if doing so is possible by freeing the cached memory but since the primary use case of lzma_memlimit_set() is to increase memlimit after LZMA_MEMLIMIT_ERROR this simple approach was selected. The cached memory was always included when enforcing the memory usage limit while decoding. Thanks to Jia Tan. src/liblzma/common/stream_decoder_mt.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) commit 1fc6e7dd1fabdb60124d449b99273330ccab3ff1 Author: Jia Tan Date: 2022-11-07 16:24:14 +0200 xz: Avoid a compiler warning in progress_speed() in message.c. This should be smaller too since it avoids the string constants. src/xz/message.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) commit cf118c014683069b5dbe91898acdc40f2f0a1f5d Author: Lasse Collin Date: 2022-10-31 16:26:05 +0200 Build: Clarify comment in configure.ac about SSE2. configure.ac | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) commit aad3c609ffb72f581a7a2b67be3ad70b2b327840 Author: Lasse Collin Date: 2022-10-31 16:16:37 +0200 Build: Remove obsolete commented-out lines from configure.ac. configure.ac | 4 ---- 1 file changed, 4 deletions(-) commit e53e0e2186c6b8ce866bd19aec52f1c318ed31ba Author: Lasse Collin Date: 2022-10-31 13:31:58 +0200 Windows: Fix mythread_once() macro with Vista threads. Don't call InitOnceComplete() if initialization was already done. So far mythread_once() has been needed only when building with --enable-small. windows/build.bash does this together with --disable-threads so the Vista-specific mythread_once() is never needed by those builds. VS project files or CMake-builds don't support HAVE_SMALL builds at all. src/common/mythread.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) commit 48dde3bab9dc04081acb5aa7cf7c5044b8a49f58 Author: Lasse Collin Date: 2022-10-31 11:54:44 +0200 liblzma: Silence -Wconversion warning from crc64_fast.c. src/liblzma/check/crc64_fast.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) commit a243c617ff249d915ac123de4f536b80322c1fdb Author: Lasse Collin Date: 2022-10-31 11:49:47 +0200 CMake: Sync tuklib_cpucores.cmake with tuklib_cpucores.m4. This was forgotten from commit 2611c4d90535652d3eb7ef4a026a6691276fab43. cmake/tuklib_cpucores.cmake | 5 +++++ 1 file changed, 5 insertions(+) commit 05c72de06fcaaedc78f8abba7d5ec568ddcf1e75 Author: Lasse Collin Date: 2022-10-27 15:49:18 +0300 Tests: test_files.sh: Make it not fail if features were disabled at build. It now tries to test as many files as easily possible. The exit status indicates skipping if any of the files were skipped. This way it is easy to notice if something is being skipped when it isn't expected. tests/test_files.sh | 50 ++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 4 deletions(-) commit b3459327a51f4b8239d19e6c34b4e0c6bc2d81de Author: Lasse Collin Date: 2022-10-27 15:30:13 +0300 Tests: test_files.sh: Suppress an expected warning from the log. xz (but not xzdec) will normally warn about unsupported check but since we are testing specifically such a file, it's better to silence that warning so that it doesn't look suspicious in test_files.sh.log. The use of -q and -Q in xzdec is just for consistency and doesn't affect the result at least for now. tests/test_files.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 798c86e4231e0835ab76ccd0810c8ea30833b2ce Author: Lasse Collin Date: 2022-10-27 15:27:50 +0300 Tests: test_files.sh: Print the reason for skipping if xz & xzdec missing. tests/test_files.sh | 1 + 1 file changed, 1 insertion(+) commit c1dd8524e1af07f16b790463899de06a6a5fcc08 Author: Lasse Collin Date: 2022-10-27 01:12:40 +0300 Tests: Keep test_compress_* working when some filters are unavailable. tests/test_compress.sh | 34 ++++++++++++++++++++-------------- 1 file changed, 20 insertions(+), 14 deletions(-) commit ce30ada91951d0746879ae438da11f1ee8a90aa0 Author: Jia Tan Date: 2022-10-23 21:01:08 +0800 Tests: test_bcj_exact_size skips properly now if PowerPC filter disabled. tests/test_bcj_exact_size.c | 3 +++ 1 file changed, 3 insertions(+) commit 89c5cfcacaca7130509fac836e2f30c46b824502 Author: Lasse Collin Date: 2022-10-26 00:05:57 +0300 Tests: Test also unsupported-*.xz. tests/test_files.sh | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) commit a4b214b93ac741edef9c41e55865b0b867ca2587 Author: Lasse Collin Date: 2022-10-25 23:45:03 +0300 Build: Use AC_CONFIG_HEADERS instead of the ancient AC_CONFIG_HEADER. We require Autoconf >= 2.69 and that has AC_CONFIG_HEADERS. There is a warning about AC_PROG_CC_C99 being obsolete but it cannot be removed because it is needed with Autoconf 2.69. configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 04f299b64e73f50afc188c2590ebebc6b73ed744 Author: Lasse Collin Date: 2022-10-25 23:31:44 +0300 Build: Update m4/ax_pthread.m4 from Autoconf Archive. m4/ax_pthread.m4 | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) commit 59c4d6e1390f6f4176f43ac1dad1f7ac03c449b8 Author: Lasse Collin Date: 2022-10-25 23:28:34 +0300 Build: Add configure option --disable-microlzma. MicroLZMA was made for EROFS and used by erofs-utils. It might be used by something else in the future but those wanting a smaller build for specific situations can now disable this rarely-needed feature. configure.ac | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) commit 054ccd6d14b2cc6eddc56897af280d3221414150 Author: Lasse Collin Date: 2022-10-25 23:09:11 +0300 xz: Fix --single-stream with an empty .xz Stream. Example: $ xz -dc --single-stream good-0-empty.xz xz: good-0-empty.xz: Internal error (bug) The code, that is tries to catch some input file issues early, didn't anticipate LZMA_STREAM_END which is possible in that code only when --single-stream is used. src/xz/coder.c | 9 +++++++++ 1 file changed, 9 insertions(+) commit 563288ea705e83ff5cb292adf794650c263bca1d Author: Lasse Collin Date: 2022-10-25 21:11:58 +0300 xz: Add support for OpenBSD's pledge() sandbox. configure.ac | 12 +++++++++--- src/xz/file_io.c | 11 +++++++++++ src/xz/main.c | 13 +++++++++++++ src/xz/private.h | 2 +- 4 files changed, 34 insertions(+), 4 deletions(-) commit f9913e8ee2ba0b1e4ff4d0aa4c001aae305ed944 Author: Lasse Collin Date: 2022-10-25 19:07:17 +0300 xz: Fix decompressor behavior if input uses an unsupported check type. Now files with unsupported check will make xz display a warning, set the exit status to 2 (unless --no-warn is used), and then decompress the file normally. This is how it was supposed to work since the beginning but this was broken by the commit 231c3c7098f1099a56abb8afece76fc9b8699f05, that is, a little before 5.0.0 was released. The buggy behavior displayed a message, set exit status 1 (error), and xz didn't attempt to to decompress the file. This doesn't matter today except for special builds that disable CRC64 or SHA-256 at build time (but such builds should be used in special situations only). The bug matters if new check type is added in the future and an old xz version is used to decompress such a file; however, it's likely that such files would use a new filter too and an old xz wouldn't be able to decompress the file anyway. The first hunk in the commit is the actual fix. The second hunk is a cleanup since LZMA_TELL_ANY_CHECK isn't used in xz. There is a test file for unsupported check type but it wasn't used by test_files.sh, perhaps due to different behavior between xz and the simpler xzdec. src/xz/coder.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) commit aa4fe145b9486adc454f44fd3e09be9add808a0f Author: Lasse Collin Date: 2022-10-25 18:36:19 +0300 xz: Clarify the man page: input file isn't removed if an error occurs. src/xz/xz.1 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) commit 8b46ae8cdeddfd7dc01fec92971b8696e9a96c5d Author: Lasse Collin Date: 2022-10-25 18:30:55 +0300 xz: Refactor to remove is_empty_filename(). Long ago it was used in list.c too but nowadays it's needed only in io_open_src() so it's nicer to avoid a separate function. src/xz/file_io.c | 4 +++- src/xz/util.c | 12 ------------ src/xz/util.h | 4 ---- 3 files changed, 3 insertions(+), 17 deletions(-) commit 85624015978b0de294cff3df79006df987c552b1 Author: Lasse Collin Date: 2022-10-25 18:23:54 +0300 xz: If input file cannot be removed, treat it as a warning, not error. Treating it as a warning (message + exit status 2) matches gzip and it seems more logical as at that point the output file has already been successfully closed. When it's a warning it is possible to suppress it with --no-warn. src/xz/file_io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit fda9f85f52c546f7ca0313cf89481da4707fecb3 Author: Lasse Collin Date: 2022-10-24 16:25:09 +0300 liblzma: Threaded decoder: Stop the worker threads on errors. It's waste of CPU time and electricity to leave the unfinished worker threads running when it is known that their output will get ignored. src/liblzma/common/stream_decoder_mt.c | 33 ++++++++++++++++++++++++++------- 1 file changed, 26 insertions(+), 7 deletions(-) commit 2611c4d90535652d3eb7ef4a026a6691276fab43 Author: Lasse Collin Date: 2022-10-20 20:22:50 +0300 tuklib_cpucores: Use HW_NCPUONLINE on OpenBSD. On OpenBSD the number of cores online is often less than what HW_NCPU would return because OpenBSD disables simultaneous multi-threading (SMT) by default. Thanks to Christian Weisgerber. m4/tuklib_cpucores.m4 | 5 +++++ src/common/tuklib_cpucores.c | 9 +++++++++ 2 files changed, 14 insertions(+) commit 424ac91c7e0419393ff2bde4f62e21fa611c776d Author: Lasse Collin Date: 2022-10-19 19:39:35 +0300 Tests: Skip tests in test_*.sh if encoders and/or decoders are disabled. This isn't perfect as the scripts can still fail if only certain filters are disabled. This is still an improvement as now "make check" has better behavior when all encoders or decoders are disabled. Grepping ../config.h is simple and fairly clean but it only works if config.h was created. CMake builds don't create config.h but they don't use these test scripts either. Thanks to Sebastian Andrzej Siewior for reporting the problem. Thanks to Jia Tan for the original patch which grepped xz error messages instead of config.h. tests/test_compress.sh | 12 ++++++++++++ tests/test_files.sh | 11 +++++++++++ tests/test_scripts.sh | 11 +++++++++++ 3 files changed, 34 insertions(+) commit ca8bf9d7c5a30be8ba1eeb106fd892f19e83ed09 Author: Lasse Collin Date: 2022-10-19 18:54:34 +0300 Test: Remove the (exit 1) lines. I suspect that I used these in the original version because Autoconf's manual describes that such a trick is needed in some specific situations for portability reasons. None of those situations listed on Autoconf 2.71 manual apply to these test scripts though so this cleans them up. tests/test_compress.sh | 10 ---------- tests/test_files.sh | 9 --------- tests/test_scripts.sh | 6 ------ 3 files changed, 25 deletions(-) commit 82fcb7cfc17ce62f79ebc7ca2374e1daca5e4d5e Author: Lasse Collin Date: 2022-10-19 17:14:57 +0300 Tests: Fix a warning in test_memlimit.c when decoders are disabled. tests/test_memlimit.c | 3 +++ 1 file changed, 3 insertions(+) commit b5f8271b45b9b0e59485ffba3640ca3418835ec4 Author: Lasse Collin Date: 2022-10-19 17:11:46 +0300 Tests: Add test_memlimit to .gitignore. Thanks to Jia Tan. .gitignore | 1 + 1 file changed, 1 insertion(+) commit 6a86e81cab202d0a812a7b2e9efacaf70c58ba38 Author: Jia Tan Date: 2022-10-06 21:53:09 +0300 Tests: Refactor test_stream_flags.c. Converts test_stream_flags to tuktest. Also the test will now compile and skip properly if encoders or decoders are disabled. Thanks to Sebastian Andrzej Siewior. tests/test_stream_flags.c | 533 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 416 insertions(+), 117 deletions(-) commit 827ac5b4821491fd3afe0d0e1ddac326253aeb66 Author: Jia Tan Date: 2022-10-06 17:00:38 +0800 Tests: Refactor test_block_header.c. test_block_header now achieves higher test coverage. Also the test will now compile and skip properly if encoders or decoders are disabled. Thanks to Sebastian Andrzej Siewior. tests/test_block_header.c | 486 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 370 insertions(+), 116 deletions(-) commit 84963318952064a93bfc52edd6b0ef70593384ee Author: Jia Tan Date: 2022-10-05 23:54:12 +0800 Tests: Fix compilation issues. test_bcj_exact_size, test_check, test_hardware, and test_index will all now compile and skip properly if encoders or decoders are disabled. Also fixed a small typo (disabed -> disabled). Thanks to Sebastian Andrzej Siewior. tests/test_bcj_exact_size.c | 20 ++++++++++++++------ tests/test_check.c | 8 +++++++- tests/test_hardware.c | 2 +- tests/test_index.c | 6 ++++++ tests/test_memlimit.c | 16 +++++++++++++++- tests/test_vli.c | 13 +++++++++++++ 6 files changed, 56 insertions(+), 9 deletions(-) commit 7dcabeec63d46b436fa5f043c3d1f09d0e15be16 Author: Lasse Collin Date: 2022-10-05 16:20:47 +0300 Tests: Include mythread.h in the tests that use MYTHREAD_ENABLED. tests/test_check.c | 1 + tests/test_hardware.c | 1 + tests/test_memlimit.c | 1 + 3 files changed, 3 insertions(+) commit 14af758a770c7781af18fb66d6d21ee5b1c27f04 Author: Jia Tan Date: 2022-10-05 20:57:16 +0800 liblzma: Fix a compilation issue when encoders are disabled. When encoders were disabled and threading enabled, outqueue.c and outqueue.h were not compiled. The multi threaded decoder required these files, so compilation failed. src/liblzma/common/Makefile.inc | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) commit 6ca5c354bd4620aa7f81da68870eef1b1f26288f Author: Jia Tan Date: 2022-10-05 16:41:38 +0800 Tests: Fix compilation error when threading support has been disabled. Now tests that require threading are skipped when threading support has been disabled. Thanks to Sebastian Andrzej Siewior. tests/test_check.c | 4 ++++ tests/test_hardware.c | 4 ++++ tests/test_memlimit.c | 4 ++++ 3 files changed, 12 insertions(+) commit fae37ad2affd8fe8871f4ff93d5cab5ec14d5e58 Author: Lasse Collin Date: 2022-10-05 14:26:00 +0300 tuklib_integer: Add 64-bit endianness-converting reads and writes. Also update the comment in liblzma's memcmplen.h. Thanks to Michał Górny for the original patch for the reads. m4/tuklib_integer.m4 | 8 ++++---- src/common/tuklib_integer.h | 46 ++++++++++++++++++++++++++++++++++++++++-- src/liblzma/common/memcmplen.h | 9 +++------ 3 files changed, 51 insertions(+), 12 deletions(-) commit 508a44372c5b0dede8863fd0d358d4a9d8645c95 Author: Lasse Collin Date: 2022-09-30 12:06:13 +0300 liblzma: Add API doc note about the .xz decoder LZMA_MEMLIMIT_ERROR bug. The bug was fixed in 660739f99ab211edec4071de98889fb32ed04e98. src/liblzma/api/lzma/base.h | 11 +++++++++++ 1 file changed, 11 insertions(+) commit 8cc9874a7974cd575aee44f218836f7acdbeb0ed Author: Jia Tan Date: 2022-09-21 16:15:50 +0800 liblzma: Add dest and src NULL checks to lzma_index_cat. The documentation states LZMA_PROG_ERROR can be returned from lzma_index_cat. Previously, lzma_index_cat could not return LZMA_PROG_ERROR. Now, the validation is similar to lzma_index_append, which does a NULL check on the index parameter. src/liblzma/common/index.c | 3 +++ 1 file changed, 3 insertions(+) commit afd5a8bf5374eba82804a999e1ea7af680784086 Author: Jia Tan Date: 2022-09-21 20:29:28 +0800 Tests: Create a test for the lzma_index_cat bug. tests/test_index.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) commit 3d5a99ca373a4e86faf671226ca6487febb9eeac Author: Jia Tan Date: 2022-09-21 19:28:53 +0800 liblzma: Fix copying of check type statistics in lzma_index_cat(). The check type of the last Stream in dest was never copied to dest->checks (the code tried to copy it but it was done too late). This meant that the value returned by lzma_index_checks() would only include the check type of the last Stream when multiple lzma_indexes had been concatenated. In xz --list this meant that the summary would only list the check type of the last Stream, so in this sense this was only a visual bug. However, it's possible that some applications use this information for purposes other than merely showing it to the users in an informational message. I'm not aware of such applications though and it's quite possible that such applications don't exist. Regular streamed decompression in xz or any other application doesn't use lzma_index_cat() and so this bug cannot affect them. src/liblzma/common/index.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) commit a61d32172789735350a941e23baf6b587c49e5d9 Author: Lasse Collin Date: 2022-09-28 12:20:41 +0300 tuklib_physmem: Fix Unicode builds on Windows. Thanks to ArSaCiA Game. src/common/tuklib_physmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 5a4d3548ab214fdca364d5c734baf1d1fab47308 Author: Lasse Collin Date: 2022-09-28 11:12:07 +0300 Tests: Add test_memlimit.c to test restarting after LZMA_MEMLIMIT_ERROR. tests/Makefile.am | 2 + tests/test_memlimit.c | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 153 insertions(+) commit 660739f99ab211edec4071de98889fb32ed04e98 Author: Lasse Collin Date: 2022-09-28 11:05:15 +0300 liblzma: Stream decoder: Fix restarting after LZMA_MEMLIMIT_ERROR. If lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible to use lzma_memlimit_set() to increase the limit and continue decoding. This was supposed to work from the beginning but there was a bug. With other decoders (.lzma or threaded .xz) this already worked correctly. src/liblzma/common/stream_decoder.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) commit 7e68fda58c74ad9e5b876cc22fcbe80fc0e4747b Author: Lasse Collin Date: 2022-09-28 11:00:23 +0300 liblzma: Stream decoder: Fix comments. src/liblzma/common/stream_decoder.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) commit f664cb25841fc1c478b819034a224a558e2ac6e7 Author: Lasse Collin Date: 2022-09-20 16:58:22 +0300 liblzma: ARM64: Add comments. src/liblzma/simple/arm64.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) commit b557b4a0eea05470fae8ba5ef4ad5a6dfb36ac41 Author: Lasse Collin Date: 2022-09-20 16:27:50 +0300 liblzma: ARM64: Fix wrong comment in API doc. Thanks to Jia Tan. src/liblzma/api/lzma/bcj.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit d5b0906fa55157f48c200188a3951d80df9cb308 Author: Lasse Collin Date: 2022-09-19 20:24:26 +0300 xz: Add --experimental-arm64[=width=WIDTH]. It will be renamed to --arm64 once it is stable. Man page or --long-help weren't updated yet. src/xz/args.c | 7 +++++++ src/xz/message.c | 7 +++++++ src/xz/options.c | 39 +++++++++++++++++++++++++++++++++++++++ src/xz/options.h | 7 +++++++ 4 files changed, 60 insertions(+) commit ecb966de308c255bb4735a7307ef9901c643a9de Author: Lasse Collin Date: 2022-09-19 19:34:56 +0300 liblzma: Add experimental ARM64 BCJ filter with a temporary Filter ID. That is, the Filter ID will be changed once the design is final. The current version will be removed. So files created with the tempoary Filter ID won't be supported in the future. CMakeLists.txt | 3 + configure.ac | 4 +- src/liblzma/api/lzma/bcj.h | 35 +++++- src/liblzma/common/filter_common.c | 9 ++ src/liblzma/common/filter_decoder.c | 8 ++ src/liblzma/common/filter_encoder.c | 11 ++ src/liblzma/simple/Makefile.inc | 4 + src/liblzma/simple/arm64.c | 227 ++++++++++++++++++++++++++++++++++++ src/liblzma/simple/simple_coder.h | 9 ++ src/liblzma/simple/simple_decoder.h | 4 + src/liblzma/simple/simple_encoder.h | 2 + 11 files changed, 313 insertions(+), 3 deletions(-) commit 177bdc922cb17bd0fd831ab8139dfae912a5c2b8 Author: Lasse Collin Date: 2022-09-17 22:42:18 +0300 liblzma: Simple/BCJ filters: Allow disabling generic BCJ options. This will be needed for the ARM64 BCJ filter as it will use its own options struct. src/liblzma/simple/arm.c | 2 +- src/liblzma/simple/armthumb.c | 2 +- src/liblzma/simple/ia64.c | 2 +- src/liblzma/simple/powerpc.c | 2 +- src/liblzma/simple/simple_coder.c | 4 ++-- src/liblzma/simple/simple_private.h | 2 +- src/liblzma/simple/sparc.c | 2 +- src/liblzma/simple/x86.c | 3 ++- 8 files changed, 10 insertions(+), 9 deletions(-) commit c3592d0a55114144686ecf960cb516d6b31c98e9 Author: Lasse Collin Date: 2022-09-16 17:08:53 +0300 Tests: Add a test file for lzma_index_append() integer overflow bug. This test fails before commit 18d7facd3802b55c287581405c4d49c98708c136. test_files.sh now runs xz -l for bad-3-index-uncomp-overflow.xz because only then the previously-buggy code path gets tested. Normal decompression doesn't use lzma_index_append() at all. Instead, lzma_index_hash functions are used and those already did the overflow check. tests/files/README | 10 ++++++++++ tests/files/bad-3-index-uncomp-overflow.xz | Bin 0 -> 132 bytes tests/test_files.sh | 8 ++++++++ 3 files changed, 18 insertions(+) commit 982b29f828079a2a26253a40e975127a40a7d2bd Author: Lasse Collin Date: 2022-09-16 15:10:07 +0300 Translations: Add Turkish translation. po/LINGUAS | 1 + po/tr.po | 977 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 978 insertions(+) commit 1fc088d9f6d1697924aaeac8cd1fb9918d1532e2 Author: Lasse Collin Date: 2022-09-16 14:09:07 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 097c7b67ce86ff16a7cef7631b39e5ca4ee3d420 Author: Lasse Collin Date: 2022-09-16 14:07:03 +0300 xzgrep: Fix compatibility with old shells. Running the current xzgrep on Slackware 10.1 with GNU bash 3.00.15: xzgrep: line 231: syntax error near unexpected token `;;' On SCO OpenServer 5.0.7 with Korn Shell 93r: syntax error at line 231 : `;;' unexpected Turns out that some old shells don't like apostrophes (') inside command substitutions. For example, the following fails: x=$(echo foo # asdf'zxcv echo bar) printf '%s\n' "$x" The problem was introduced by commits 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 (2022-03-29), bd7b290f3fe4faeceb7d3497ed9bf2e6ed5e7dc5 (2022-07-18), and a648978b20495b7aa4a8b029c5a810b5ad9d08ff (2022-07-19). 5.2.6 is the only stable release that included this problem. Thanks to Kevin R. Bulgrien for reporting the problem on SCO OpenServer 5.0.7 and for providing the fix. src/scripts/xzgrep.in | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit f2d084fe3f0d6d71488bfc6987f26542f67bfd99 Author: Lasse Collin Date: 2022-09-09 14:12:30 +0300 Tests: Silence warnings about unused functions from tuktest.h. Warnings about unused tuktest_run_test conveniently tell which test programs haven't been converted to tuktest.h yet but I silenced that warning too for now anyway. It is fine to use __attribute__((__unused__)) even when the function is actually used because the attribute only means that the function might be unused. tests/tuktest.h | 10 ++++++++++ 1 file changed, 10 insertions(+) commit f8ee61e74eb40600445fdb601c374d582e1e9c8a Author: Lasse Collin Date: 2022-09-09 13:51:57 +0300 liblzma: lzma_filters_copy: Keep dest[] unmodified if an error occurs. lzma_stream_encoder() and lzma_stream_encoder_mt() always assumed this. Before this patch, failing lzma_filters_copy() could result in free(invalid_pointer) or invalid memory reads in stream_encoder.c or stream_encoder_mt.c. To trigger this, allocating memory for a filter options structure has to fail. These are tiny allocations so in practice they very rarely fail. Certain badness in the filter chain array could also make lzma_filters_copy() fail but both stream_encoder.c and stream_encoder_mt.c validate the filter chain before trying to copy it, so the crash cannot occur this way. src/liblzma/api/lzma/filter.h | 4 +++- src/liblzma/common/filter_common.c | 18 ++++++++++++------ 2 files changed, 15 insertions(+), 7 deletions(-) commit 18d7facd3802b55c287581405c4d49c98708c136 Author: Jia Tan Date: 2022-09-02 20:18:55 +0800 liblzma: lzma_index_append: Add missing integer overflow check. The documentation in src/liblzma/api/lzma/index.h suggests that both the unpadded (compressed) size and the uncompressed size are checked for overflow, but only the unpadded size was checked. The uncompressed check is done first since that is more likely to occur than the unpadded or index field size overflows. src/liblzma/common/index.c | 4 ++++ 1 file changed, 4 insertions(+) commit 9ac06cb5b85274d18f9f70d82cf2d8c9c1151bd4 Author: Lasse Collin Date: 2022-09-08 15:11:08 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit ba3e4ba2de034ae93a513f9c3a0823b80cdb66dc Author: Jia Tan Date: 2022-09-08 15:07:00 +0300 CMake: Clarify a comment about Windows symlinks without file extension. CMakeLists.txt | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) commit 17485e884ce5c74315f29a8a1507bc706cd5cd1d Author: Lasse Collin Date: 2022-09-08 15:02:41 +0300 CMake: Update for liblzma_*.map files and fix wrong common_w32res.rc dep. The previous commit split liblzma.map into liblzma_linux.map and liblzma_generic.map. This commit updates the CMake build for those. common_w32res.rc dependency was listed under Linux/FreeBSD while obviously it belongs to Windows when building a DLL. CMakeLists.txt | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) commit 913ddc5572b9455fa0cf299be2e35c708840e922 Author: Lasse Collin Date: 2022-09-04 23:23:00 +0300 liblzma: Vaccinate against an ill patch from RHEL/CentOS 7. RHEL/CentOS 7 shipped with 5.1.2alpha, including the threaded encoder that is behind #ifdef LZMA_UNSTABLE in the API headers. In 5.1.2alpha these symbols are under XZ_5.1.2alpha in liblzma.map. API/ABI compatibility tracking isn't done between development releases so newer releases didn't have XZ_5.1.2alpha anymore. Later RHEL/CentOS 7 updated xz to 5.2.2 but they wanted to keep the exported symbols compatible with 5.1.2alpha. After checking the ABI changes it turned out that >= 5.2.0 ABI is backward compatible with the threaded encoder functions from 5.1.2alpha (but not vice versa as fixes and extensions to these functions were made between 5.1.2alpha and 5.2.0). In RHEL/CentOS 7, XZ Utils 5.2.2 was patched with xz-5.2.2-compat-libs.patch to modify liblzma.map: - XZ_5.1.2alpha was added with lzma_stream_encoder_mt and lzma_stream_encoder_mt_memusage. This matched XZ Utils 5.1.2alpha. - XZ_5.2 was replaced with XZ_5.2.2. It is clear that this was an error; the intention was to keep using XZ_5.2 (XZ_5.2.2 has never been used in XZ Utils). So XZ_5.2.2 lists all symbols that were listed under XZ_5.2 before the patch. lzma_stream_encoder_mt and _mt_memusage are included too so they are listed both here and under XZ_5.1.2alpha. The patch didn't add any __asm__(".symver ...") lines to the .c files. Thus the resulting liblzma.so exports the threaded encoder functions under XZ_5.1.2alpha only. Listing the two functions also under XZ_5.2.2 in liblzma.map has no effect without matching .symver lines. The lack of XZ_5.2 in RHEL/CentOS 7 means that binaries linked against unpatched XZ Utils 5.2.x won't run on RHEL/CentOS 7. This is unfortunate but this alone isn't too bad as the problem is contained within RHEL/CentOS 7 and doesn't affect users of other distributions. It could also be fixed internally in RHEL/CentOS 7. The second problem is more serious: In XZ Utils 5.2.2 the API headers don't have #ifdef LZMA_UNSTABLE for obvious reasons. This is true in RHEL/CentOS 7 version too. Thus now programs using new APIs can be compiled without an extra #define. However, the programs end up depending on symbol version XZ_5.1.2alpha (and possibly also XZ_5.2.2) instead of XZ_5.2 as they would with an unpatched XZ Utils 5.2.2. This means that such binaries won't run on other distributions shipping XZ Utils >= 5.2.0 as they don't provide XZ_5.1.2alpha or XZ_5.2.2; they only provide XZ_5.2 (and XZ_5.0). (This includes RHEL/CentOS 8 as the patch luckily isn't included there anymore with XZ Utils 5.2.4.) Binaries built by RHEL/CentOS 7 users get distributed and then people wonder why they don't run on some other distribution. Seems that people have found out about the patch and been copying it to some build scripts, seemingly curing the symptoms but actually spreading the illness further and outside RHEL/CentOS 7. The ill patch seems to be from late 2016 (RHEL 7.3) and in 2017 it had spread at least to EasyBuild. I heard about the events only recently. :-( This commit splits liblzma.map into two versions: one for GNU/Linux and another for other OSes that can use symbol versioning (FreeBSD, Solaris, maybe others). The Linux-specific file and the matching additions to .c files add full compatibility with binaries that have been built against a RHEL/CentOS-patched liblzma. Builds for OSes other than GNU/Linux won't get the vaccine as they should be immune to the problem (I really hope that no build script uses the RHEL/CentOS 7 patch outside GNU/Linux). The RHEL/CentOS compatibility symbols XZ_5.1.2alpha and XZ_5.2.2 are intentionally put *after* XZ_5.2 in liblzma_linux.map. This way if one forgets to #define HAVE_SYMBOL_VERSIONS_LINUX when building, the resulting liblzma.so.5 will have lzma_stream_encoder_mt@@XZ_5.2 since XZ_5.2 {...} is the first one that lists that function. Without HAVE_SYMBOL_VERSIONS_LINUX @XZ_5.1.2alpha and @XZ_5.2.2 will be missing but that's still a minor problem compared to only having lzma_stream_encoder_mt@@XZ_5.1.2alpha! The "local: *;" line was moved to XZ_5.0 so that it doesn't need to be moved around. It doesn't matter where it is put. Having two similar liblzma_*.map files is a bit silly as it is, at least for now, easily possible to generate the generic one from the Linux-specific file. But that adds extra steps and increases the risk of mistakes when supporting more than one build system. So I rather maintain two files in parallel and let validate_map.sh check that they are in sync when "make mydist" is run. This adds .symver lines for lzma_stream_encoder_mt@XZ_5.2.2 and lzma_stream_encoder_mt_memusage@XZ_5.2.2 even though these weren't exported by RHEL/CentOS 7 (only @@XZ_5.1.2alpha was for these two). I added these anyway because someone might misunderstand the RHEL/CentOS 7 patch and think that @XZ_5.2.2 (@@XZ_5.2.2) versions were exported too. At glance one could suggest using __typeof__ to copy the function prototypes when making aliases. However, this doesn't work trivially because __typeof__ won't copy attributes (lzma_nothrow, lzma_pure) and it won't change symbol visibility from hidden to default (done by LZMA_API()). Attributes could be copied with __copy__ attribute but that needs GCC 9 and a fallback method would be needed anyway. This uses __symver__ attribute with GCC >= 10 and __asm__(".symver ...") with everything else. The attribute method is required for LTO (-flto) support with GCC. Using -flto with GCC older than 10 is now broken on GNU/Linux and will not be fixed (can silently result in a broken liblzma build that has dangerously incorrect symbol versions). LTO builds with Clang seem to work with the traditional __asm__(".symver ...") method. Thanks to Boud Roukema for reporting the problem and discussing the details and testing the fix. configure.ac | 23 +++- src/liblzma/Makefile.am | 10 +- src/liblzma/common/block_buffer_encoder.c | 18 ++++ src/liblzma/common/common.c | 14 +++ src/liblzma/common/common.h | 28 +++++ src/liblzma/common/hardware_cputhreads.c | 12 +++ src/liblzma/common/stream_encoder_mt.c | 42 ++++++++ src/liblzma/{liblzma.map => liblzma_generic.map} | 6 +- src/liblzma/liblzma_linux.map | 131 +++++++++++++++++++++++ src/liblzma/validate_map.sh | 113 +++++++++++++++++-- 10 files changed, 382 insertions(+), 15 deletions(-) commit 80a1a8bb838842a2be343bd88ad1462c21c5e2c9 Author: Lasse Collin Date: 2022-08-31 16:42:04 +0300 CMake: Add xz symlinks. These are a minor thing especially since the xz build has some real problems still like lack of large file support on 32-bit systems but I'll commit this since the code exists. Thanks to Jia Tan. CMakeLists.txt | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) commit a4193bb6d85d7765b1b87faeab3e50106d3ab2e9 Author: Lasse Collin Date: 2022-08-31 16:29:38 +0300 CMake: Put xz man page install under if(UNIX) like is for xzdec. Thanks to Jia Tan. CMakeLists.txt | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) commit c1555b1a221a1427d4d650647531016d754bc4da Author: Lasse Collin Date: 2022-08-22 18:16:40 +0300 Bump version number for 5.3.3alpha. src/liblzma/api/lzma/version.h | 2 +- src/liblzma/liblzma.map | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit 44fedddc11c3f6ec2f7fe35a3e38f15ca93f90eb Author: Lasse Collin Date: 2022-08-22 18:13:56 +0300 Add NEWS for 5.3.3alpha. NEWS | 119 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+) commit a93e235d7ca764cc19f8f9f9063b40ff361c3cfa Author: Lasse Collin Date: 2022-07-12 17:59:41 +0300 Translations: Add Portuguese translation. Jia Tan made white-space changes and also changed "Language: pt_BR\n" to pt. The translator wasn't reached so I'm hoping these changes are OK and will commit it without translator's approval. Thanks to Pedro Albuquerque and Jia Tan. po/LINGUAS | 1 + po/pt.po | 1001 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 1002 insertions(+) commit e7cf5a946f25e40d77f45e41f0dee7d42a04e9ae Author: Lasse Collin Date: 2022-07-10 21:16:40 +0300 Translations: Add Serbian translation. Quite a few white-space changes were made by Jia Tan to make this look good. Contacting the translator didn't succeed so I'm committing this without getting translator's approval. Thanks to Мирослав Николић (Miroslav Nikolic) and Jia Tan. po/LINGUAS | 1 + po/sr.po | 987 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 988 insertions(+) commit cc8617ab53b1f2a4da212fa76c92fe538269c5de Author: Lasse Collin Date: 2022-07-04 23:51:36 +0300 Translations: Add Swedish translation. Thanks to Sebastian Rasmussen and Jia Tan. po/LINGUAS | 1 + po/sv.po | 983 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 984 insertions(+) commit c613598c358b640682d0ca2aed38fa7df763e8c7 Author: Lasse Collin Date: 2022-07-04 23:40:27 +0300 Translations: Add Esperanto translation. Thanks to Keith Bowes and Jia Tan. po/LINGUAS | 1 + po/eo.po | 984 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 985 insertions(+) commit 659a587d678f21e98e91d2751c31d4ce050c081a Author: Lasse Collin Date: 2022-07-01 00:22:33 +0300 Translations: Add Catalan translation. Thanks to Jordi Mas and Jia Tan. po/LINGUAS | 1 + po/ca.po | 1076 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 1077 insertions(+) commit 739fd8c9bdf1d85f57d56642aad87148d4779530 Author: Lasse Collin Date: 2022-06-30 17:47:08 +0300 Translations: Add Ukrainian translation. Thanks to Yuri Chornoivan and Jia Tan. po/LINGUAS | 1 + po/uk.po | 996 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 997 insertions(+) commit 73280550b111930c62a667e56add8fd574f80bc8 Author: Lasse Collin Date: 2022-06-30 17:45:26 +0300 Translators: Add Romanian translation. Thanks to Remus-Gabriel Chelu and Jia Tan. po/LINGUAS | 1 + po/ro.po | 1016 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 1017 insertions(+) commit 2465f5b825152714b2c357d96c2422c31109d320 Author: Lasse Collin Date: 2022-06-29 18:33:32 +0300 Translations: Update Brazilian Portuguese translation. One msgstr was changed. The diff is long due to changes in the source code line numbers in the comments. Thanks to Rafael Fontenelle. po/pt_BR.po | 186 ++++++++++++++++++++++++++++++------------------------------ 1 file changed, 92 insertions(+), 94 deletions(-) commit 434e1ffd3e62719d509da10b646216f5ef53fe4d Author: Lasse Collin Date: 2022-06-29 18:04:44 +0300 Translations: Add Croatian translation. Thanks to Božidar Putanec and Jia Tan. po/LINGUAS | 1 + po/hr.po | 987 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 988 insertions(+) commit 0732d0f7065c9bd48bfe4f5983144ae970c4a499 Author: Lasse Collin Date: 2022-06-29 17:58:48 +0300 Translations: Add Spanish translation. Thanks to Cristian Othón Martínez Vera and Jia Tan. po/LINGUAS | 1 + po/es.po | 984 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 985 insertions(+) commit 9899b0f82bc130998d1f1f618a6ab805b73f2696 Author: Lasse Collin Date: 2022-06-29 17:49:43 +0300 Translations: Add Korean translation. Thanks to Seong-ho Cho and Jia Tan. po/LINGUAS | 1 + po/ko.po | 972 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 973 insertions(+) commit 65217eaf6bd195f3ef027d4ac55d57b7d133d69f Author: Lasse Collin Date: 2020-03-16 21:57:21 +0200 Translations: Rebuild cs.po to avoid incorrect fuzzy strings. "make dist" updates the .po files and the fuzzy strings would result in multiple very wrong translations. po/cs.po | 592 ++++++++++++++++++++++++++++++++++----------------------------- 1 file changed, 322 insertions(+), 270 deletions(-) commit e56ff423ee7af9e648e16b132f0d835d2cb4db26 Author: Lasse Collin Date: 2020-03-16 17:30:39 +0200 Translations: Add partial Danish translation. I made a few minor white space changes without getting them approved by the Danish translation team. po/LINGUAS | 1 + po/da.po | 896 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 897 insertions(+) commit 43e09c62e77cb8807d932c81de4decbdb956e184 Author: Lasse Collin Date: 2020-03-11 22:37:54 +0200 Translations: Add hu, zh_CN, and zh_TW. I made a few white space changes to these without getting them approved by the translation teams. (I tried to contact the hu and zh_TW teams but didn't succeed. I didn't contact the zh_CN team.) po/LINGUAS | 3 + po/hu.po | 985 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ po/zh_CN.po | 963 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ po/zh_TW.po | 956 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 2907 insertions(+) commit 982b6b198ae1ffe6093236dd8a3d03d1415b912b Author: Lasse Collin Date: 2020-03-11 14:33:30 +0200 Translations: Update vi.po to match the file from the TP. The translated strings haven't been updated but word wrapping is different. po/vi.po | 407 ++++++++++++++++++++++++++++----------------------------------- 1 file changed, 179 insertions(+), 228 deletions(-) commit 801f39691fc4abc6dd33d7653d498781b593f3eb Author: Lasse Collin Date: 2020-03-11 14:18:03 +0200 Translations: Add fi and pt_BR, and update de, fr, it, and pl. The German translation isn't identical to the file in the Translation Project but the changes (white space changes only) were approved by the translator Mario Blättermann. po/LINGUAS | 2 + po/de.po | 476 ++++++++++++++-------------- po/fi.po | 974 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ po/fr.po | 272 ++++++++-------- po/it.po | 479 ++++++++++++---------------- po/pl.po | 239 +++++++------- po/pt_BR.po | 1001 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 2697 insertions(+), 746 deletions(-) commit 311e4f85ede5d2f0bb71f3ad70b5b7db1b5adf33 Author: Lasse Collin Date: 2022-08-22 17:27:19 +0300 xz: Try to clarify --memlimit-mt-decompress vs. --memlimit-compress. src/xz/xz.1 | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) commit df23c31000283c00e5ef1ca32a0bc3bb757bd707 Author: Lasse Collin Date: 2022-08-22 16:46:18 +0300 CMake: Add liblzma tests. Thanks to Jia Tan for the patch. CMakeLists.txt | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-) commit 02a777f9c422d3b5ec895078530bb1a2b6f7bdf5 Author: Lasse Collin Date: 2022-08-19 23:32:22 +0300 xz: Revise --info-memory output. The strings could be more descriptive but it's good to have some version of this committed now. --robot mode wasn't changed yet. src/xz/hardware.c | 32 ++++++++++++++++++++++++++------ src/xz/xz.1 | 1 + 2 files changed, 27 insertions(+), 6 deletions(-) commit f864f6d42eab57ea8ed82cc2dd19a03b51377442 Author: Lasse Collin Date: 2022-08-19 23:12:02 +0300 xz: Update the man page for threaded decompression and memlimits. This documents the changes made in commits 6c6da57ae2aa962aabde6892442227063d87e88c, cad299008cf73ec566f0662a9cf2b94f86a99659, and 898faa97287a756231c663a3ed5165672b417207. The --info-memory bit hasn't been finished yet even though it's already mentioned in this commit under --memlimit-mt-decompress and --threads. src/xz/xz.1 | 148 +++++++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 121 insertions(+), 27 deletions(-) commit d13bfcc0056617dd648f655a01653932fad7067f Author: Lasse Collin Date: 2022-08-18 17:49:16 +0300 Build: Include the CMake files in the distribution. This was supposed to be done in 2020 with 5.2.5 release already but it was noticed only today. 5.2.5 and 5.2.6 even mention experiemental CMake support in the NEWS entries. Thanks to Olivier B. for reporting the problem. Makefile.am | 2 ++ 1 file changed, 2 insertions(+) commit e66787bcfefdb93f19c974f895f65969a77937b0 Author: Lasse Collin Date: 2022-08-18 17:38:05 +0300 Windows: Fix broken liblzma.dll build with Visual Studio project files. The bug was introduced in 352ba2d69af2136bc814aa1df1a132559d445616 "Windows: Fix building of resource files when config.h isn't used." That commit fixed liblzma.dll build with CMake while keeping it working with Autotools on Windows but the VS project files were forgotten. I haven't tested these changes. Thanks to Olivier B. for reporting the bug and for the initial patch. windows/vs2013/liblzma_dll.vcxproj | 6 ++++++ windows/vs2017/liblzma_dll.vcxproj | 6 ++++++ windows/vs2019/liblzma_dll.vcxproj | 6 ++++++ 3 files changed, 18 insertions(+) commit c4e8e5fb311225b8b48d34157891a640b2535e0c Author: Lasse Collin Date: 2022-08-18 17:16:49 +0300 liblzma: Threaded decoder: Improve LZMA_FAIL_FAST when LZMA_FINISH is used. It will now return LZMA_DATA_ERROR (not LZMA_OK or LZMA_BUF_ERROR) if LZMA_FINISH is used and there isn't enough input to finish decoding the Block Header or the Block. The use of LZMA_DATA_ERROR is simpler and the less risky than LZMA_BUF_ERROR but this might be changed before 5.4.0. src/liblzma/api/lzma/container.h | 6 +++++ src/liblzma/common/stream_decoder_mt.c | 42 ++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) commit 6dcf606e7efa2b259f0262f9e2f61e00116842d3 Author: Lasse Collin Date: 2022-08-12 18:31:47 +0300 Add NEWS for 5.2.6. NEWS | 121 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) commit 413b86fcf8934fae5a004f378a9483d37d8fcaab Author: Lasse Collin Date: 2022-08-12 14:28:41 +0300 Add Jia Tan to AUTHORS. AUTHORS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 352672732c346c546ff3c26d0605bc0ed1c8b7c7 Author: Lasse Collin Date: 2022-07-25 19:28:26 +0300 Build: Start the generated ChangeLog from around 5.2.0 instead of 5.0.0. This makes ChangeLog smaller. Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 6f6d11225d6016be2bbb55d845b66f6b04d048df Author: Lasse Collin Date: 2022-07-25 19:11:05 +0300 Translations: Change the copyright comment string to use with po4a. This affects the second line in po4a/xz-man.pot. The man pages of xzdiff, xzgrep, and xzmore are from GNU gzip and under GNU GPLv2+ while the rest of the man pages are in the public domain. po4a/update-po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 61f8ec804abdb4c5dac01e8ae9b90c7be58a5c24 Author: Jia Tan Date: 2022-07-25 18:30:05 +0300 liblzma: Refactor lzma_mf_is_supported() to use a switch-statement. src/liblzma/lz/lz_encoder.c | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-) commit 4d80b463a1251aa22eabc87d2732fec13b1adda6 Author: Jia Tan Date: 2022-07-25 18:20:01 +0300 Build: Don't allow empty LIST in --enable-match-finders=LIST. It's enforced only when a match finder is needed, that is, when LZMA1 or LZMA2 encoder is enabled. configure.ac | 4 ++++ 1 file changed, 4 insertions(+) commit 9cc721af5436908f2d5a828aebbc4050a32a3487 Author: Lasse Collin Date: 2022-07-24 13:27:48 +0300 xz: Update the man page that change to --keep will be in 5.2.6. src/xz/xz.1 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit b81bf0c7d1873e52a4086a9abb494471d652cb55 Author: Lasse Collin Date: 2022-07-19 23:23:54 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 340cf1ec3927767046b8293a49da3db4e393f426 Author: Nicholas Jackson Date: 2022-07-17 17:39:23 -0700 CMake: Add missing source file to liblzma build CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) commit d796b6d7fdb8b7238b277056cf9146cce25db604 Author: Lasse Collin Date: 2022-07-19 23:19:49 +0300 xzgrep man page: Document exit statuses. src/scripts/xzgrep.1 | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) commit 923bf96b55e5216a6c8df9d8331934f54784390e Author: Lasse Collin Date: 2022-07-19 23:13:24 +0300 xzgrep: Improve error handling, especially signals. xzgrep wouldn't exit on SIGPIPE or SIGQUIT when it clearly should have. It's quite possible that it's not perfect still but at least it's much better. If multiple exit statuses compete, now it tries to pick the largest of value. Some comments were added. The exit status handling of signals is still broken if the shell uses values larger than 255 in $? to indicate that a process died due to a signal ***and*** their "exit" command doesn't take this into account. This seems to work well with the ksh and yash versions I tried. However, there is a report in gzip/zgrep that OpenSolaris 5.11 (not 5.10) has a problem with "exit" truncating the argument to 8 bits: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22900#25 Such a bug would break xzgrep but I didn't add a workaround at least for now. 5.11 is old and I don't know if the problem exists in modern descendants, or if the problem exists in other ksh implementations in use. src/scripts/xzgrep.in | 72 +++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 53 insertions(+), 19 deletions(-) commit a648978b20495b7aa4a8b029c5a810b5ad9d08ff Author: Lasse Collin Date: 2022-07-19 00:10:55 +0300 xzgrep: Make the fix for ZDI-CAN-16587 more robust. I don't know if this can make a difference in the real world but it looked kind of suspicious (what happens with sed implementations that cannot process very long lines?). At least this commit shouldn't make it worse. src/scripts/xzgrep.in | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) commit bd7b290f3fe4faeceb7d3497ed9bf2e6ed5e7dc5 Author: Lasse Collin Date: 2022-07-18 21:52:31 +0300 xzgrep: Use grep -H --label when available (GNU, *BSDs). It avoids the use of sed for prefixing filenames to output lines. Using sed for that is slower and prone to security bugs so now the sed method is only used as a fallback. This also fixes an actual bug: When grepping a binary file, GNU grep nowadays prints its diagnostics to stderr instead of stdout and thus the sed-method for prefixing the filename doesn't work. So with this commit grepping binary files gives reasonable output with GNU grep now. This was inspired by zgrep but the implementation is different. src/scripts/xzgrep.in | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) commit b56729af9f1a596e57aeefd7570d8d7dce5c9f52 Author: Lasse Collin Date: 2022-07-18 21:10:25 +0300 xzgrep: Use -e to specify the pattern to grep. Now we don't need the separate test for adding the -q option as it can be added directly in the two places where it's needed. src/scripts/xzgrep.in | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) commit bad61b5997e6647911974022bfb72f3d4818a594 Author: Lasse Collin Date: 2022-07-18 19:18:48 +0300 Scripts: Use printf instead of echo in a few places. It's a good habbit as echo has some portability corner cases when the string contents can be anything. src/scripts/xzdiff.in | 6 +++--- src/scripts/xzgrep.in | 4 ++-- src/scripts/xzless.in | 4 ++-- src/scripts/xzmore.in | 8 ++++---- 4 files changed, 11 insertions(+), 11 deletions(-) commit 6a4a4a7d2667837dc824c26fcb19ed6ca5aff645 Author: Lasse Collin Date: 2022-07-17 21:36:25 +0300 xzgrep: Add more LC_ALL=C to avoid bugs with multibyte characters. Also replace one use of expr with printf. The rationale for LC_ALL=C was already mentioned in 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 that fixed a security issue. However, unrelated uses weren't changed in that commit yet. POSIX says that with sed and such tools one should use LC_ALL=C to ensure predictable behavior when strings contain byte sequences that aren't valid multibyte characters in the current locale. See under "Application usage" in here: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html With GNU sed invalid multibyte strings would work without this; it's documented in its Texinfo manual. Some other implementations aren't so forgiving. src/scripts/xzgrep.in | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) commit b48f9d615f2c2e8d2f6e253d0e48ee66d0652b68 Author: Lasse Collin Date: 2022-07-17 20:55:16 +0300 xzgrep: Fix parsing of certain options. Fix handling of "xzgrep -25 foo" (in GNU grep "grep -25 foo" is an alias for "grep -C25 foo"). xzgrep would treat "foo" as filename instead of as a pattern. This bug was fixed in zgrep in gzip in 2012. Add -E, -F, -G, and -P to the "no argument required" list. Add -X to "argument required" list. It is an intentionally-undocumented GNU grep option so this isn't an important option for xzgrep but it seems that other grep implementations (well, those that I checked) don't support -X so I hope this change is an improvement still. grep -d (grep --directories=ACTION) requires an argument. In contrast to zgrep, I kept -d in the "no argument required" list because it's not supported in xzgrep (or zgrep). This way "xzgrep -d" gives an error about option being unsupported instead of telling that it requires an argument. Both zgrep and xzgrep tell that it's unsupported if an argument is specified. Add comments. src/scripts/xzgrep.in | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) commit 2d2201bc6315deda4d43625aa510972467bd51d4 Author: Lasse Collin Date: 2022-07-14 20:33:05 +0300 Tests: Add the .lzma files to test_files.sh. tests/test_files.sh | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) commit ce5549a591bf126300618879f5b24023351baff6 Author: Lasse Collin Date: 2022-07-14 19:37:42 +0300 Tests: Add .lzma test files. tests/files/README | 63 ++++++++++++++++----- tests/files/bad-too_big_size-with_eopm.lzma | Bin 0 -> 37 bytes tests/files/bad-too_small_size-without_eopm-1.lzma | Bin 0 -> 31 bytes tests/files/bad-too_small_size-without_eopm-2.lzma | Bin 0 -> 31 bytes tests/files/bad-too_small_size-without_eopm-3.lzma | Bin 0 -> 36 bytes tests/files/bad-unknown_size-without_eopm.lzma | Bin 0 -> 31 bytes tests/files/good-known_size-with_eopm.lzma | Bin 0 -> 37 bytes tests/files/good-known_size-without_eopm.lzma | Bin 0 -> 31 bytes tests/files/good-unknown_size-with_eopm.lzma | Bin 0 -> 37 bytes 9 files changed, 50 insertions(+), 13 deletions(-) commit 107c93ee5cad51a6ea0cee471209bfe8d76deaa3 Author: Lasse Collin Date: 2022-07-14 18:12:38 +0300 liblzma: Rename a variable and improve a comment. src/liblzma/lzma/lzma_decoder.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) commit 511feb5eadb988d641b025d597f4fac7502003b8 Author: Lasse Collin Date: 2022-07-13 22:24:41 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 9595a3119b9faf0ce01375329cad8bbf85c35ea2 Author: Lasse Collin Date: 2022-07-13 22:24:07 +0300 liblzma: Add optional autodetection of LZMA end marker. Turns out that this is needed for .lzma files as the spec in LZMA SDK says that end marker may be present even if the size is stored in the header. Such files are rare but exist in the real world. The code in liblzma is so old that the spec didn't exist in LZMA SDK back then and I had understood that such files weren't possible (the lzma tool in LZMA SDK didn't create such files). This modifies the internal API so that LZMA decoder can be told if EOPM is allowed even when the uncompressed size is known. It's allowed with .lzma and not with other uses. Thanks to Karl Beldan for reporting the problem. doc/lzma-file-format.txt | 11 +++- src/liblzma/common/alone_decoder.c | 2 +- src/liblzma/common/microlzma_decoder.c | 2 +- src/liblzma/lz/lz_decoder.c | 10 +++- src/liblzma/lz/lz_decoder.h | 8 +-- src/liblzma/lzma/lzma2_decoder.c | 2 +- src/liblzma/lzma/lzma_decoder.c | 99 ++++++++++++++++++++++++---------- 7 files changed, 95 insertions(+), 39 deletions(-) commit 0c0f8e9761eb6eaf199082cf144db7ac5f9d8cb2 Author: Lasse Collin Date: 2022-07-12 18:53:04 +0300 xz: Document the special memlimit case of 2000 MiB on MIPS32. See commit fc3d3a7296ef58bb799a73943636b8bfd95339f7. src/xz/xz.1 | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) commit d1bfa3dc703325ecd974167e864a8712fdfe936e Author: Jia Tan Date: 2022-07-01 21:19:26 +0800 Created script to generate code coverage reports. The script uses lcov and genhtml after running the tests to show the code coverage statistics. The script will create a coverage directory where it is run. It can be run both in and out of the source directory. .gitignore | 4 +++ tests/code_coverage.sh | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+) commit 86a30b0255d8064169fabfd213d907016d2f9f2a Author: Jia Tan Date: 2022-06-16 17:32:19 +0300 Tests: Add more tests into test_check. tests/test_check.c | 279 +++++++++++++++++++++++++++++++++++++++++++++++++++-- tests/tests.h | 23 +++++ 2 files changed, 295 insertions(+), 7 deletions(-) commit 82e30fed66a89706388a8c15dc954d84e63f38fa Author: Lasse Collin Date: 2022-06-16 15:02:57 +0300 Tests: Use char[][24] array for enum_strings_lzma_ret. Array of pointers to short strings is a bit pointless here and now it's fully const. tests/tests.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 5ba9459e6c4a29f6870ca78ce8ac6e519d59c41e Author: Lasse Collin Date: 2022-06-16 14:12:14 +0300 Tests: tuktest.h: Add tuktest_error_impl to help with error conditions. tests/tuktest.h | 72 +++++++++++++++++++++++++-------------------------------- 1 file changed, 32 insertions(+), 40 deletions(-) commit b339892668da20aea22a93668c82b87a38e4a97f Author: Lasse Collin Date: 2022-06-16 13:29:59 +0300 Tests: tuktest.h: Rename file_from_* and use tuktest_malloc there. tests/test_bcj_exact_size.c | 4 +--- tests/tuktest.h | 52 +++++++++++++++++++++------------------------ 2 files changed, 25 insertions(+), 31 deletions(-) commit d8b63a0ad68d1c461eb373466679ebc41fbc207d Author: Lasse Collin Date: 2022-06-16 13:08:19 +0300 Tests: tuktest.h: Add malloc wrapper with automatic freeing. tests/tuktest.h | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) commit 1d51536a4b19a8fae768f8eb462fc2238cb36d53 Author: Lasse Collin Date: 2022-06-16 11:47:37 +0300 Tests: tuktest.h: Move a function. tests/tuktest.h | 44 ++++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 22 deletions(-) commit 70c7555f6403553ee35539e869de0025592d8564 Author: Lasse Collin Date: 2022-06-14 22:21:15 +0300 Tests: test_vli: Remove an invalid test-assertion. lzma_vli is unsigned so trying a signed value results in a compiler warning from -Wsign-conversion. (lzma_vli)-1 equals to LZMA_VLI_UNKNOWN anyway which is the next assertion. tests/test_vli.c | 2 -- 1 file changed, 2 deletions(-) commit 154b73c5a1092c3f785e01666b564ad7ff1be555 Author: Lasse Collin Date: 2022-06-14 22:10:10 +0300 Tests: test_vli: Add const where appropriate. tests/test_vli.c | 53 ++++++++++++++++++++++++++++------------------------- 1 file changed, 28 insertions(+), 25 deletions(-) commit 0354d6cce3ff98ea6f927107baf216253f6ce2bb Author: Jia Tan Date: 2022-06-13 20:27:03 +0800 Added vli tests to .gitignore .gitignore | 1 + 1 file changed, 1 insertion(+) commit a08f5ccf6bdc20ef70e41f6f3321618ef146f96e Author: Jia Tan Date: 2022-06-12 11:31:40 +0800 Created tests for all functions exported in vli.h Achieved 100% code coverage vli_encoder.c, vli_decoder.c, and vli_size.c tests/Makefile.am | 4 +- tests/test_vli.c | 308 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 311 insertions(+), 1 deletion(-) commit 1e3eb61815a91c0a1bfbb802e2d95593f523491f Author: jiat75 Date: 2022-06-03 21:24:54 +0800 Added parallel test artifacts to .gitignore .gitignore | 2 ++ 1 file changed, 2 insertions(+) commit 00e3613f1212eaa84e721585fdb4de6967cf2476 Author: Lasse Collin Date: 2022-06-14 21:29:21 +0300 Tests: Use good-1-empty-bcj-lzma2.xz in test_bcj_exact_size. It's much nicer this way so that the test data isn't a hardcoded table inside the C file. tests/test_bcj_exact_size.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) commit 86bab755be252bfd3e0a9aee8e7b83a9bbb23ed0 Author: Lasse Collin Date: 2022-06-14 21:26:13 +0300 Tests: Add file reading helpers to tuktest.h. tests/tuktest.h | 183 +++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 176 insertions(+), 7 deletions(-) commit 83d2337b72dbf391c6f3b41889eea99e51679105 Author: Lasse Collin Date: 2022-06-14 18:21:57 +0300 Tests: tuktest.h: Move a printf from a macro to a helper function. tests/tuktest.h | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) commit f9e8176ea7d520797a2db2d49a5a632c285674a8 Author: Lasse Collin Date: 2022-06-14 17:20:49 +0300 Tests: Add test file good-1-empty-bcj-lzma2.xz. This is from test_bcj_exact_size.c. It's good to have it as a standalone file. tests/files/README | 5 +++++ tests/files/good-1-empty-bcj-lzma2.xz | Bin 0 -> 52 bytes 2 files changed, 5 insertions(+) commit aa75c5563a760aea3aa23d997d519e702e82726b Author: Jia Tan Date: 2022-06-10 21:35:18 +0800 Tests: Created tests for hardware functions. Created tests for all API functions exported in src/liblzma/api/lzma/hardware.h. The tests are fairly trivial but are helpful because they will inform users if their machines cannot support these functions. They also improve the code coverage metrics. .gitignore | 1 + tests/Makefile.am | 2 ++ tests/test_hardware.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 48 insertions(+) commit 5c8ffdca20813939818843476fb212dfae8838a3 Author: Lasse Collin Date: 2022-06-02 21:01:45 +0300 Tests: Convert test_check to tuktest. Thanks to Jia Tan for help with all the tests. tests/test_check.c | 67 +++++++++++++++++++++++++++--------------------------- 1 file changed, 33 insertions(+), 34 deletions(-) commit faf5ff8899d539b4dcd2a7e5280cb820a4746c86 Author: Lasse Collin Date: 2022-06-02 20:31:03 +0300 Tests: Convert test_block_header to tuktest. tests/test_block_header.c | 89 +++++++++++++++++++++++++++-------------------- 1 file changed, 52 insertions(+), 37 deletions(-) commit 754d39fbebee3782258d42f154a223d3c5770ec7 Author: Lasse Collin Date: 2022-06-02 20:28:23 +0300 Tests: Convert test_bcj_exact_size to tuktest. The compress() and decompress() functions were merged because the later depends on the former so they need to be a single test case. tests/test_bcj_exact_size.c | 75 +++++++++++++++++++++++++-------------------- 1 file changed, 41 insertions(+), 34 deletions(-) commit 96da21470f9570cd08286906a050a7c22631775b Author: Lasse Collin Date: 2022-06-02 20:27:00 +0300 Tests: Include tuktest.h in tests.h. This breaks -Werror because none of the tests so far use tuktest.h and thus there are warnings about unused variables and functions. tests/tests.h | 47 +++++++++++++++++++++++------------------------ 1 file changed, 23 insertions(+), 24 deletions(-) commit df71ba1c991f60c3269aaadd398247e632714626 Author: Lasse Collin Date: 2022-06-02 20:25:21 +0300 Tests: Add tuktest.h mini-test-framework. tests/Makefile.am | 1 + tests/tuktest.h | 752 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 753 insertions(+) commit 4773608554d1b684a05ff9c1d879cf5c42266d33 Author: Lasse Collin Date: 2022-05-23 21:31:36 +0300 Build: Enable Automake's parallel test harness. It has been the default for quite some time already and the old serial harness isn't discouraged. The downside is that with parallel tests one cannot print progress info or other diagnostics to the terminal; all output from the tests will be in the log files only. But now that the compression tests are separated the parallel tests will speed things up. configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 9a6dd6d46f7f256a5573e5d874c1052033ed7b05 Author: Lasse Collin Date: 2022-05-23 21:17:47 +0300 Tests: Split test_compress.sh into separate test unit for each file. test_compress.sh now takes one command line argument: a filename to be tested. If it begins with "compress_generated_" the file will be created with create_compress_files. This will allow parallel execution of the slow tests. tests/Makefile.am | 11 +++- tests/test_compress.sh | 91 +++++++++++++++++++--------------- tests/test_compress_generated_abc | 3 ++ tests/test_compress_generated_random | 3 ++ tests/test_compress_generated_text | 3 ++ tests/test_compress_prepared_bcj_sparc | 3 ++ tests/test_compress_prepared_bcj_x86 | 3 ++ 7 files changed, 77 insertions(+), 40 deletions(-) commit c7758ac9c734707514dd34f254173ebac5eea7f8 Author: Lasse Collin Date: 2022-05-23 20:32:49 +0300 Test: Make create_compress_files.c a little more flexible. If a command line argument is given, then only the test file of that type is created. It's quite dumb in sense that unknown names don't give an error but it's good enough here. Also use EXIT_FAILURE instead of 1 as exit status for errors. tests/create_compress_files.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) commit 4a8e4a7b0ad4b03c0ac6862716c3457452cdaf8c Author: Lasse Collin Date: 2022-05-23 20:17:42 +0300 Tests: Remove unneeded commented lines from test_compress.sh. tests/test_compress.sh | 13 ------------- 1 file changed, 13 deletions(-) commit 2ee50d150ee009f36135540b459e6ff328549725 Author: Lasse Collin Date: 2022-05-23 20:16:00 +0300 Tests: Remove progress indicator from test_compress.sh. It will be useless with Automake's parallel tests. tests/test_compress.sh | 9 --------- 1 file changed, 9 deletions(-) commit 2ce4f36f179a81d0c6e182a409f363df759d1ad0 Author: Lasse Collin Date: 2022-05-23 19:37:18 +0300 liblzma: Silence a warning. The actual initialization is done via mythread_sync and seems that GCC doesn't necessarily see that it gets initialized there. src/liblzma/common/stream_decoder_mt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 5d8f3764ef43c35910e6d7003c0900a961ef6544 Author: Lasse Collin Date: 2022-04-14 20:53:16 +0300 xz: Fix build with --disable-threads. src/xz/hardware.c | 4 ++++ 1 file changed, 4 insertions(+) commit 1d592897278b172d8549aa29c3a1f3a4f432a9b9 Author: Lasse Collin Date: 2022-04-14 14:50:17 +0300 xz: Change the cap of the default -T0 memlimit for 32-bit xz. The SIZE_MAX / 3 was 1365 MiB. 1400 MiB gives little more room and it looks like a round (artificial) number in --info-memory once --info-memory is made to display it. Also, using #if avoids useless code on 64-bit builds. src/xz/hardware.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) commit c77fe55ddb7752ed0fec46967c5ec9a72632ea0c Author: Lasse Collin Date: 2022-04-14 14:20:46 +0300 xz: Add a default soft memory usage limit for --threads=0. This is a soft limit in sense that it only affects the number of threads. It never makes xz fail and it never makes xz change settings that would affect the compressed output. The idea is to make -T0 have more reasonable behavior when the system has very many cores or when a memory-hungry compression options are used. This also helps with 32-bit xz, preventing it from running out of address space. The downside of this commit is that now the number of threads might become too low compared to what the user expected. I hope this to be an acceptable compromise as the old behavior has been a source of well-argued complaints for a long time. src/xz/coder.c | 28 ++++++++++++++++++++++++++-- src/xz/hardware.c | 38 +++++++++++++++++++++++++++++--------- src/xz/hardware.h | 27 +++++++++++++++++++++++++++ 3 files changed, 82 insertions(+), 11 deletions(-) commit 0adc13bfe32c14f3e4c6ce9f2d4fdf4112ab53f4 Author: Lasse Collin Date: 2022-04-14 12:59:09 +0300 xz: Make -T0 use multithreaded mode on single-core systems. The main problem withi the old behavior is that the compressed output is different on single-core systems vs. multicore systems. This commit fixes it by making -T0 one thread in multithreaded mode on single-core systems. The downside of this is that it uses more memory. However, if --memlimit-compress is used, xz can (thanks to the previous commit) drop to the single-threaded mode still. src/xz/coder.c | 18 +++++++++--------- src/xz/hardware.c | 14 ++++++++++++++ src/xz/hardware.h | 4 ++++ 3 files changed, 27 insertions(+), 9 deletions(-) commit 898faa97287a756231c663a3ed5165672b417207 Author: Lasse Collin Date: 2022-04-14 12:38:00 +0300 xz: Changes to --memlimit-compress and --no-adjust. In single-threaded mode, --memlimit-compress can make xz scale down the LZMA2 dictionary size to meet the memory usage limit. This obviously affects the compressed output. However, if xz was in threaded mode, --memlimit-compress could make xz reduce the number of threads but it wouldn't make xz switch from multithreaded mode to single-threaded mode or scale down the LZMA2 dictionary size. This seemed illogical and there was even a "FIXME?" about it. Now --memlimit-compress can make xz switch to single-threaded mode if one thread in multithreaded mode uses too much memory. If memory usage is still too high, then the LZMA2 dictionary size can be scaled down too. The option --no-adjust was also changed so that it no longer prevents xz from scaling down the number of threads as that doesn't affect compressed output (only performance). After this commit --no-adjust only prevents adjustments that affect compressed output, that is, with --no-adjust xz won't switch from multithreaded mode to single-threaded mode and won't scale down the LZMA2 dictionary size. The man page wasn't updated yet. src/xz/coder.c | 63 +++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 43 insertions(+), 20 deletions(-) commit cad299008cf73ec566f0662a9cf2b94f86a99659 Author: Lasse Collin Date: 2022-04-11 22:20:49 +0300 xz: Add --memlimit-mt-decompress along with a default limit value. --memlimit-mt-decompress allows specifying the limit for multithreaded decompression. This matches memlimit_threading in liblzma. This limit can only affect the number of threads being used; it will never prevent xz from decompressing a file. The old --memlimit-decompress option is still used at the same time. If the value of --memlimit-decompress (the default value or one specified by the user) is less than the value of --memlimit-mt-decompress , then --memlimit-mt-decompress is reduced to match --memlimit-decompress. Man page wasn't updated yet. src/xz/args.c | 24 +++++++++++++++------- src/xz/coder.c | 34 ++++++++++--------------------- src/xz/hardware.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++----- src/xz/hardware.h | 17 ++++++++++------ src/xz/message.c | 4 +++- 5 files changed, 97 insertions(+), 42 deletions(-) commit fe87b4cd5364f5bbb6a75a0299f1500c852d7c9a Author: Lasse Collin Date: 2022-04-06 23:11:59 +0300 liblzma: Threaded decoder: Improve setting of pending_error. It doesn't need to be done conditionally. The comments try to explain it. src/liblzma/common/stream_decoder_mt.c | 51 +++++++++++++++++++++++----------- 1 file changed, 35 insertions(+), 16 deletions(-) commit 90621da7f6e1bfd6d91d60415eae04b2bca274c2 Author: Lasse Collin Date: 2022-04-06 12:39:49 +0300 liblzma: Add a new flag LZMA_FAIL_FAST for threaded decoder. In most cases if the input file is corrupt the application won't care about the uncompressed content at all. With this new flag the threaded decoder will return an error as soon as any thread has detected an error; it won't wait to copy out the data before the location of the error. I don't plan to use this in xz to keep the behavior consistent between single-threaded and multi-threaded modes. src/liblzma/api/lzma/container.h | 25 ++++++++++++++++- src/liblzma/common/common.h | 7 +++-- src/liblzma/common/stream_decoder_mt.c | 50 +++++++++++++++++++--------------- 3 files changed, 56 insertions(+), 26 deletions(-) commit 64b6d496dc815a176d8307f418f6834a26783484 Author: Lasse Collin Date: 2022-04-05 12:24:57 +0300 liblzma: Threaded decoder: Always wait for output if LZMA_FINISH is used. This makes the behavior consistent with the single-threaded decoder when handling truncated .xz files. Thanks to Jia Tan for finding this issue. src/liblzma/common/stream_decoder_mt.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) commit e671bc8828b9c0c5406c3a22c541301d0eb54518 Author: Lasse Collin Date: 2022-04-02 21:49:59 +0300 liblzma: Threaded decoder: Support zpipe.c-style decoding loop. This makes it possible to call lzma_code() in a loop that only reads new input when lzma_code() didn't fill the output buffer completely. That isn't the calling style suggested by the liblzma example program 02_decompress.c so perhaps the usefulness of this feature is limited. Also, it is possible to write such a loop so that it works with the single-threaded decoder but not with the threaded decoder even after this commit, or so that it works only if lzma_mt.timeout = 0. The zlib tutorial is a well-known example of a loop where more input is read only when output isn't full. Porting this as is to liblzma would work with the single-threaded decoder (if LZMA_CONCATENATED isn't used) but it wouldn't work with threaded decoder even after this commit because the loop assumes that no more output is possible when it cannot read more input ("if (strm.avail_in == 0) break;"). This cannot be fixed at liblzma side; the loop has to be modified at least a little. I'm adding this in any case because the actual code is simple and short and should have no harmful side-effects in other situations. src/liblzma/common/stream_decoder_mt.c | 77 +++++++++++++++++++++++++++++----- 1 file changed, 67 insertions(+), 10 deletions(-) commit 2ba8173e27be4793edb46497e499ac2ae753a316 Author: Lasse Collin Date: 2022-03-31 00:05:07 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 Author: Lasse Collin Date: 2022-03-29 19:19:12 +0300 xzgrep: Fix escaping of malicious filenames (ZDI-CAN-16587). Malicious filenames can make xzgrep to write to arbitrary files or (with a GNU sed extension) lead to arbitrary code execution. xzgrep from XZ Utils versions up to and including 5.2.5 are affected. 5.3.1alpha and 5.3.2alpha are affected as well. This patch works for all of them. This bug was inherited from gzip's zgrep. gzip 1.12 includes a fix for zgrep. The issue with the old sed script is that with multiple newlines, the N-command will read the second line of input, then the s-commands will be skipped because it's not the end of the file yet, then a new sed cycle starts and the pattern space is printed and emptied. So only the last line or two get escaped. One way to fix this would be to read all lines into the pattern space first. However, the included fix is even simpler: All lines except the last line get a backslash appended at the end. To ensure that shell command substitution doesn't eat a possible trailing newline, a colon is appended to the filename before escaping. The colon is later used to separate the filename from the grep output so it is fine to add it here instead of a few lines later. The old code also wasn't POSIX compliant as it used \n in the replacement section of the s-command. Using \ is the POSIX compatible method. LC_ALL=C was added to the two critical sed commands. POSIX sed manual recommends it when using sed to manipulate pathnames because in other locales invalid multibyte sequences might cause issues with some sed implementations. In case of GNU sed, these particular sed scripts wouldn't have such problems but some other scripts could have, see: info '(sed)Locale Considerations' This vulnerability was discovered by: cleemy desu wayo working with Trend Micro Zero Day Initiative Thanks to Jim Meyering and Paul Eggert discussing the different ways to fix this and for coordinating the patch release schedule with gzip. src/scripts/xzgrep.in | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) commit bd93b776c1bd15e90661033c918cdeb354dbcc38 Author: Lasse Collin Date: 2022-03-26 01:02:44 +0200 liblzma: Fix a deadlock in threaded decoder. If a worker thread has consumed all input so far and it's waiting on thr->cond and then the main thread enables partial update for that thread, the code used to deadlock. This commit allows one dummy decoding pass to occur in this situation which then also does the partial update. As part of the fix, this moves thr->progress_* updates to avoid the second thr->mutex locking. Thanks to Jia Tan for finding, debugging, and reporting the bug. src/liblzma/common/stream_decoder_mt.c | 71 +++++++++++++++++++++++++--------- 1 file changed, 52 insertions(+), 19 deletions(-) commit e0394e94230f208682ac1e1f4c41f22f9ad79916 Author: Lasse Collin Date: 2022-03-23 16:34:00 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 487c77d48760564b1949c5067630b675b87be4de Author: Lasse Collin Date: 2022-03-23 16:28:55 +0200 liblzma: Threaded decoder: Don't stop threads on LZMA_TIMED_OUT. LZMA_TIMED_OUT is not an error and thus stopping threads on LZMA_TIMED_OUT breaks the decoder badly. Thanks to Jia Tan for finding the bug and for the patch. src/liblzma/common/stream_decoder_mt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 6c6da57ae2aa962aabde6892442227063d87e88c Author: Lasse Collin Date: 2022-03-07 00:36:16 +0200 xz: Add initial support for threaded decompression. If threading support is enabled at build time, this will use lzma_stream_decoder_mt() even for single-threaded mode. With memlimit_threading=0 the behavior should be identical. This needs some work like adding --memlimit-threading=LIMIT. The original patch from Sebastian Andrzej Siewior included a method to get currently available RAM on Linux. It might be one way to go but as it is Linux-only, the available-RAM approach needs work for portability or using a fallback method on other OSes. The man page wasn't updated yet. src/xz/coder.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) commit 4cce3e27f529af33e0e7749a8cbcec59954946b5 Author: Lasse Collin Date: 2022-03-06 23:36:20 +0200 liblzma: Add threaded .xz decompressor. I realize that this is about a decade late. Big thanks to Sebastian Andrzej Siewior for the original patch. I made a bunch of smaller changes but after a while quite a few things got rewritten. So any bugs in the commit were created by me. src/liblzma/api/lzma/container.h | 90 +- src/liblzma/common/Makefile.inc | 5 + src/liblzma/common/common.h | 4 + src/liblzma/common/stream_decoder_mt.c | 1814 ++++++++++++++++++++++++++++++++ src/liblzma/liblzma.map | 1 + 5 files changed, 1907 insertions(+), 7 deletions(-) commit 717631b9788dc9c100ee0c87d3c14a2782638ff4 Author: Lasse Collin Date: 2022-03-06 16:54:23 +0200 liblzma: Fix docs: lzma_block_decoder() cannot return LZMA_UNSUPPORTED_CHECK. If Check is unsupported, it will be silently ignored. It's the caller's job to handle it. src/liblzma/api/lzma/block.h | 3 --- 1 file changed, 3 deletions(-) commit 1a4bb97a00936535e30ac61945aeee38882b5d1a Author: Lasse Collin Date: 2022-03-06 16:41:19 +0200 liblzma: Add new output queue (lzma_outq) features. Add lzma_outq_clear_cache2() which may leave one buffer allocated in the cache. Add lzma_outq_outbuf_memusage() to get the memory needed for a single lzma_outbuf. This is now used internally in outqueue.c too. Track both the total amount of memory allocated and the amount of memory that is in active use (not in cache). In lzma_outbuf, allow storing the current input position that matches the current output position. This way the main thread can notice when no more output is possible without first providing more input. Allow specifying return code for lzma_outq_read() in a finished lzma_outbuf. src/liblzma/common/outqueue.c | 43 +++++++++++++++++++++++++++++++++++------- src/liblzma/common/outqueue.h | 44 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 79 insertions(+), 8 deletions(-) commit ddbc6f58c2de388eed24cd7ea91b523d397da5f4 Author: Lasse Collin Date: 2022-03-06 15:18:58 +0200 liblzma: Index hash: Change return value type of hash_append() to void. src/liblzma/common/index_hash.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) commit 20e7a33e2d59c6a814447d3991f21e2702174b20 Author: Lasse Collin Date: 2022-02-22 03:42:57 +0200 liblzma: Minor addition to lzma_vli_size() API doc. Thanks to Jia Tan. src/liblzma/api/lzma/vli.h | 2 ++ 1 file changed, 2 insertions(+) commit 4f78f5fcf63592f2d77e921cfe0d5de300867374 Author: Lasse Collin Date: 2022-02-22 02:04:18 +0200 liblzma: Check the return value of lzma_index_append() in threaded encoder. If lzma_index_append() failed (most likely memory allocation failure) it could have gone unnoticed and the resulting .xz file would have an incorrect Index. Decompressing such a file would produce the correct uncompressed data but then an error would occur when verifying the Index field. src/liblzma/common/stream_encoder_mt.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) commit 5313ad66b40aab822ddca3e9905254cb99a4080d Author: Lasse Collin Date: 2022-02-22 01:37:39 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 865e0a3689a25a7ee8eecae1a34c1775e3aa676e Author: Ed Maste Date: 2022-02-11 15:25:46 +0000 liblzma: Use non-executable stack on FreeBSD as on Linux src/liblzma/check/crc32_x86.S | 4 ++-- src/liblzma/check/crc64_x86.S | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) commit 1c9a5786d206b4abc8e427326651c8174baea753 Author: Lasse Collin Date: 2022-02-20 20:36:27 +0200 liblzma: Make Block decoder catch certain types of errors better. Now it limits the input and output buffer sizes that are passed to a raw decoder. This way there's no need to check if the sizes can grow too big or overflow when updating Compressed Size and Uncompressed Size counts. This also means that a corrupt file cannot cause the raw decoder to process useless extra input or output that would exceed the size info in Block Header (and thus cause LZMA_DATA_ERROR anyway). More importantly, now the size information is verified more carefully in case raw decoder returns LZMA_OK. This doesn't really matter with the current single-threaded .xz decoder as the errors would be detected slightly later anyway. But this helps avoiding corner cases in the upcoming threaded decompressor, and it might help other Block decoder uses outside liblzma too. The test files bad-1-lzma2-{9,10,11}.xz test these conditions. With the single-threaded .xz decoder the only difference is that LZMA_DATA_ERROR is detected in a difference place now. src/liblzma/common/block_decoder.c | 79 ++++++++++++++++++++++++++------------ 1 file changed, 54 insertions(+), 25 deletions(-) commit 555de11873eb00c9b94a8be70645db502e5a9dbd Author: Lasse Collin Date: 2022-02-20 19:38:55 +0200 Tests: Add bad-1-lzma2-11.xz. tests/files/README | 5 +++++ tests/files/bad-1-lzma2-11.xz | Bin 0 -> 64 bytes 2 files changed, 5 insertions(+) commit f0da507f22e7f4e3edb75b45b74d344244ca03fb Author: Lasse Collin Date: 2022-02-18 18:51:10 +0200 Translations: Fix po4a failure with the French man page translations. Thanks to Mario Blättermann for the patch. po4a/fr_FR.po | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) commit f7711d228c3c32395460c82498c60a9f730d0239 Author: Lasse Collin Date: 2022-02-07 01:14:37 +0200 Translations: Add French translation of man pages. This matches xz-utils 5.2.5-2 in Debian. The translation was done by "bubu", proofread by the debian-l10n-french mailing list contributors, and submitted to me on the xz-devel mailing list by Jean-Pierre Giraud. Thanks to everyone! po4a/fr_FR.po | 3541 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ po4a/po4a.conf | 2 +- 2 files changed, 3542 insertions(+), 1 deletion(-) commit 6468f7e41a8e9c611e4ba8d34e2175c5dacdbeb4 Author: jiat75 Date: 2022-01-28 20:47:55 +0800 liblzma: Add NULL checks to LZMA and LZMA2 properties encoders. Previously lzma_lzma_props_encode() and lzma_lzma2_props_encode() assumed that the options pointers must be non-NULL because the with these filters the API says it must never be NULL. It is good to do these checks anyway. src/liblzma/lzma/lzma2_encoder.c | 3 +++ src/liblzma/lzma/lzma_encoder.c | 3 +++ 2 files changed, 6 insertions(+) commit 2523c30705f49eabd27b854aa656ae87cc224808 Author: Lasse Collin Date: 2022-02-06 23:19:32 +0200 liblzma: Fix uint64_t vs. size_t confusion. This broke 32-bit builds due to a pointer type mismatch. This bug was introduced with the output-size-limited encoding in 625f4c7c99b2fcc4db9e7ab2deb4884790e2e17c. Thanks to huangqinjin for the bug report. src/liblzma/rangecoder/range_encoder.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) commit 2bd36c91d03e03b31a4f12fd0afc100ae32d66e2 Author: huangqinjin Date: 2021-12-13 20:49:21 +0800 CMake: Keep compatible with Windows 95 for 32-bit build. CMakeLists.txt | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) commit 2024fbf2794885277d05378d40b2b8015a7c3b40 Author: Lasse Collin Date: 2021-11-13 21:04:05 +0200 xzgrep: Update man page timestamp. src/scripts/xzgrep.1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 400e7a239a53282cedaad927a41f3463d7f542e5 Author: Lasse Collin Date: 2021-11-13 18:23:24 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 3a512c7787b2642ca946f4adc6e9a0a5d9b0d5a0 Author: Ville Skyttä Date: 2021-11-13 10:11:57 +0200 xzgrep: use `grep -E/-F` instead of `egrep` and `fgrep` `egrep` and `fgrep` have been deprecated in GNU grep since 2007, and in current post 3.7 Git they have been made to emit obsolescence warnings: https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1 src/scripts/xzgrep.1 | 8 ++++---- src/scripts/xzgrep.in | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) commit edf525e2b1840dcaf377df472c67d8f11f8ace1b Author: Lasse Collin Date: 2021-10-28 23:02:11 +0300 Bump the version number for 5.3.2alpha. src/liblzma/api/lzma/version.h | 2 +- src/liblzma/liblzma.map | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit ea8c948655a86290524efe59cff067e06a886709 Author: Lasse Collin Date: 2021-10-28 22:59:52 +0300 Add NEWS for 5.3.2alpha. NEWS | 86 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) commit 52435f302f4724157ec50b4210cbe42b285c3cbc Author: Lasse Collin Date: 2021-10-27 23:27:48 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit f2aea1d5a504b2021bf47a238390e4f12bdd518d Author: Lasse Collin Date: 2021-10-27 23:23:11 +0300 xz: Change the coding style of the previous commit. It isn't any better now but it's consistent with the rest of the code base. src/xz/file_io.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) commit 892b16cc282f5b4e1c49871b326f4db25c5b4d81 Author: Alexander Bluhm Date: 2021-10-05 23:33:16 +0200 xz: Avoid fchown(2) failure. OpenBSD does not allow to change the group of a file if the user does not belong to this group. In contrast to Linux, OpenBSD also fails if the new group is the same as the old one. Do not call fchown(2) in this case, it would change nothing anyway. This fixes an issue with Perl Alien::Build module. https://github.com/PerlAlien/Alien-Build/issues/62 src/xz/file_io.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) commit 2b509c868cae3988bf21cd693fbf9021cdc85628 Author: Lasse Collin Date: 2021-09-17 17:31:11 +0300 liblzma: Fix liblzma.map for the lzma_microlzma_* symbols. This should have been part of d267d109c370a40b502e73f8664b154b15e4f253. Thanks to Gao Xiang. src/liblzma/liblzma.map | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit cacb06a954b58255dfc084a0bc9708f43a0fd6d6 Author: Lasse Collin Date: 2021-09-09 22:21:07 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 6928aac9da6ba612780b9f72ba1d6ecbe1e8b54e Author: Lasse Collin Date: 2021-09-09 21:41:51 +0300 liblzma: Use _MSVC_LANG to detect when "noexcept" can be used with MSVC. By default, MSVC always sets __cplusplus to 199711L. The real C++ standard version is available in _MSVC_LANG (or one could use /Zc:__cplusplus to set __cplusplus correctly). Fixes . Thanks to Dan Weiss. src/liblzma/api/lzma.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit d267d109c370a40b502e73f8664b154b15e4f253 Author: Lasse Collin Date: 2021-09-05 20:38:12 +0300 liblzma: Rename EROFS LZMA to MicroLZMA. It still exists primarily for EROFS but MicroLZMA is a more generic name (that hopefully doesn't clash with something that already exists). src/liblzma/api/lzma/container.h | 33 +++++++++++++--------- src/liblzma/common/Makefile.inc | 4 +-- .../{erofs_decoder.c => microlzma_decoder.c} | 32 ++++++++++----------- .../{erofs_encoder.c => microlzma_encoder.c} | 30 ++++++++++---------- 4 files changed, 52 insertions(+), 47 deletions(-) commit 3247e95115acb95bc27f41e8cf4501db5b0b4309 Author: Lasse Collin Date: 2021-06-04 19:02:38 +0300 xzdiff: Update the man page about the exit status. This was forgotten from 194029ffaf74282a81f0c299c07f73caca3232ca. src/scripts/xzdiff.1 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 96f5a28a46fc93ac4e296808ac0f8631d05498bc Author: Lasse Collin Date: 2021-06-04 18:52:48 +0300 xzless: Fix less(1) version detection when it contains a dot. Sometimes the version number from "less -V" contains a dot, sometimes not. xzless failed detect the version number when it does contain a dot. This fixes it. Thanks to nick87720z for reporting this. Apparently it had been reported here in 2013. src/scripts/xzless.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 5fb5212d816addbc523d0798cb482fdd0484f8fa Author: Lasse Collin Date: 2021-04-11 19:58:10 +0300 Update THANKS. THANKS | 2 ++ 1 file changed, 2 insertions(+) commit fc3d3a7296ef58bb799a73943636b8bfd95339f7 Author: Ivan A. Melnikov Date: 2021-04-09 11:45:10 +0300 Reduce maximum possible memory limit on MIPS32 Due to architectural limitations, address space available to a single userspace process on MIPS32 is limited to 2 GiB, not 4, even on systems that have more physical RAM -- e.g. 64-bit systems with 32-bit userspace, or systems that use XPA (an extension similar to x86's PAE). So, for MIPS32, we have to impose stronger memory limits. I've chosen 2000MiB to give the process some headroom. src/xz/hardware.c | 6 ++++++ 1 file changed, 6 insertions(+) commit e7da44d5151e21f153925781ad29334ae0786101 Author: Lasse Collin Date: 2021-02-13 23:31:27 +0200 CMake: Use interface library for better FindLibLZMA compatibility. https://www.mail-archive.com/xz-devel@tukaani.org/msg00446.html Thanks to Markus Rickert. CMakeLists.txt | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) commit a61dd82ada39030f41b4ffca9ea551714908bedc Author: Lasse Collin Date: 2021-01-30 18:36:04 +0200 CMake: Try to improve compatibility with the FindLibLZMA module. The naming conflict with FindLibLZMA module gets worse. Not avoiding it in the first place was stupid. Normally find_package(LibLZMA) will use the module and find_package(liblzma 5.2.5 REQUIRED CONFIG) will use the config file even with a case insensitive file system. However, if CMAKE_FIND_PACKAGE_PREFER_CONFIG is TRUE and the file system is case insensitive, find_package(LibLZMA) will find our liblzma config file instead of using FindLibLZMA module. One big problem with this is that FindLibLZMA uses LibLZMA::LibLZMA and we use liblzma::liblzma as the target name. With target names CMake happens to be case sensitive. To workaround this, this commit adds add_library(LibLZMA::LibLZMA ALIAS liblzma::liblzma) to the config file. Then both spellings work. To make the behavior consistent between case sensitive and insensitive file systems, the config and related files are renamed from liblzmaConfig.cmake to liblzma-config.cmake style. With this style CMake looks for lowercase version of the package name so find_package(LiBLzmA 5.2.5 REQUIRED CONFIG) will work to find our config file. There are other differences between our config file and FindLibLZMA so it's still possible that things break for reasons other than the spelling of the target name. Hopefully those situations aren't too common. When the config file is available, it should always give as good or better results as FindLibLZMA so this commit doesn't affect the recommendation to use find_package(liblzma 5.2.5 REQUIRED CONFIG) which explicitly avoids FindLibLZMA. Thanks to Markus Rickert. CMakeLists.txt | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) commit 5b7bc1b8ae766a76710ca1b99f909cf52c697f05 Author: Lasse Collin Date: 2021-01-29 21:19:43 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 6c6f0db340dcb8bb424411cedba713405d55f6b8 Author: Lasse Collin Date: 2021-01-29 21:19:08 +0200 liblzma: Fix unitialized variable. This was introduced two weeks ago in the commit 625f4c7c99b2fcc4db9e7ab2deb4884790e2e17c. Thanks to Nathan Moinvaziri. src/liblzma/lzma/lzma_encoder.c | 1 + 1 file changed, 1 insertion(+) commit bb1d5c1fdd30550d4221ecd336e0be1206132a5c Author: Lasse Collin Date: 2021-01-24 22:32:41 +0200 Tests: Add bad-1-lzma2-10.xz and also modify -9.xz. tests/files/README | 11 +++++++++-- tests/files/bad-1-lzma2-10.xz | Bin 0 -> 60 bytes tests/files/bad-1-lzma2-9.xz | Bin 72 -> 72 bytes 3 files changed, 9 insertions(+), 2 deletions(-) commit 6b8abc84a5469792e0355d0bfc0784d41cfdfef7 Author: Lasse Collin Date: 2021-01-24 19:22:35 +0200 liblzma: Fix a wrong comment in stream_encoder_mt.c. src/liblzma/common/stream_encoder_mt.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) commit 939fc5ed654aac25fe0c8684b2df8dbeadb2de1e Author: Lasse Collin Date: 2021-01-24 18:51:51 +0200 Tests: Add bad-1-lzma2-9.xz. tests/files/README | 4 ++++ tests/files/bad-1-lzma2-9.xz | Bin 0 -> 72 bytes 2 files changed, 4 insertions(+) commit fdd30032f8531ac89519b48c21d810ecf06825f6 Author: Lasse Collin Date: 2021-01-24 17:02:00 +0200 Tests: Add bad-1-check-crc32-2.xz. tests/files/README | 7 +++++++ tests/files/bad-1-check-crc32-2.xz | Bin 0 -> 72 bytes 2 files changed, 7 insertions(+) commit db465419ae26ec7fb9b9472183911ff521620c77 Author: Lasse Collin Date: 2021-01-17 19:20:50 +0200 liblzma: In EROFS LZMA decoder, verify that comp_size matches at the end. When the uncompressed size is known to be exact, after decompressing the stream exactly comp_size bytes of input must have been consumed. This is a minor improvement to error detection. src/liblzma/common/erofs_decoder.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) commit 774cc0118ba2496581cb2621505a04bb6598cc75 Author: Lasse Collin Date: 2021-01-17 18:53:34 +0200 liblzma: Make EROFS LZMA decoder work when exact uncomp_size isn't known. The caller must still not specify an uncompressed size bigger than the actual uncompressed size. As a downside, this now needs the exact compressed size. src/liblzma/api/lzma/container.h | 23 ++++++++--- src/liblzma/common/erofs_decoder.c | 80 ++++++++++++++++++++++++++++++++++---- 2 files changed, 91 insertions(+), 12 deletions(-) commit 421b0aa352da244075db10205cf33712f91b9835 Author: Lasse Collin Date: 2021-01-14 20:57:11 +0200 liblzma: Fix missing normalization in rc_encode_dummy(). Without this fix it could attempt to create too much output. src/liblzma/rangecoder/range_encoder.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) commit 601ec0311e769fc704daaaa7dac0ca840aff080e Author: Lasse Collin Date: 2021-01-14 20:07:01 +0200 liblzma: Add EROFS LZMA encoder and decoder. Right now this is just a planned extra-compact format for use in the EROFS file system in Linux. At this point it's possible that the format will either change or be abandoned and removed completely. The special thing about the encoder is that it uses the output-size-limited encoding added in the previous commit. EROFS uses fixed-sized blocks (e.g. 4 KiB) to hold compressed data so the compressors must be able to create valid streams that fill the given block size. src/liblzma/api/lzma/container.h | 76 +++++++++++++++++++ src/liblzma/common/Makefile.inc | 2 + src/liblzma/common/erofs_decoder.c | 148 +++++++++++++++++++++++++++++++++++++ src/liblzma/common/erofs_encoder.c | 139 ++++++++++++++++++++++++++++++++++ src/liblzma/liblzma.map | 2 + 5 files changed, 367 insertions(+) commit 625f4c7c99b2fcc4db9e7ab2deb4884790e2e17c Author: Lasse Collin Date: 2021-01-13 19:16:32 +0200 liblzma: Add rough support for output-size-limited encoding in LZMA1. With this it is possible to encode LZMA1 data without EOPM so that the encoder will encode as much input as it can without exceeding the specified output size limit. The resulting LZMA1 stream will be a normal LZMA1 stream without EOPM. The actual uncompressed size will be available to the caller via the uncomp_size pointer. One missing thing is that the LZMA layer doesn't inform the LZ layer when the encoding is finished and thus the LZ may read more input when it won't be used. However, this doesn't matter if encoding is done with a single call (which is the planned use case for now). For proper multi-call encoding this should be improved. This commit only adds the functionality for internal use. Nothing uses it yet. src/liblzma/common/common.h | 11 +++ src/liblzma/lz/lz_encoder.c | 16 ++++ src/liblzma/lz/lz_encoder.h | 4 + src/liblzma/lzma/lzma_encoder.c | 127 +++++++++++++++++++++++--------- src/liblzma/lzma/lzma_encoder_private.h | 12 +++ src/liblzma/rangecoder/range_encoder.h | 111 ++++++++++++++++++++++++++++ 6 files changed, 246 insertions(+), 35 deletions(-) commit 9cdabbeea891e8f1e7741b076f7db6ac05ae392a Author: Lasse Collin Date: 2021-01-11 23:57:11 +0200 Scripts: Add zstd support to xzdiff. src/scripts/xzdiff.1 | 6 ++++-- src/scripts/xzdiff.in | 16 +++++++++++----- 2 files changed, 15 insertions(+), 7 deletions(-) commit d9ec3add97cf4c999a7f594c6529680227b6c274 Author: Lasse Collin Date: 2021-01-11 23:41:30 +0200 Update THANKS. THANKS | 2 ++ 1 file changed, 2 insertions(+) commit 074259f4f3966aeac6edb205fecbc1a8d2b58bb2 Author: Lasse Collin Date: 2021-01-11 23:41:16 +0200 xz: Make --keep accept symlinks, hardlinks, and setuid/setgid/sticky. Previously this required using --force but that has other effects too which might be undesirable. Changing the behavior of --keep has a small risk of breaking existing scripts but since this is a fairly special corner case I expect the likehood of breakage to be low enough. I think the new behavior is more logical. The only reason for the old behavior was to be consistent with gzip and bzip2. Thanks to Vincent Lefevre and Sebastian Andrzej Siewior. src/xz/file_io.c | 9 +++++---- src/xz/xz.1 | 16 +++++++++++++++- 2 files changed, 20 insertions(+), 5 deletions(-) commit 73c555b3077c19dda29b6f4592ced2af876f8333 Author: Lasse Collin Date: 2021-01-11 23:28:52 +0200 Scripts: Fix exit status of xzgrep. Omit the -q option from xz, gzip, and bzip2. With xz this shouldn't matter. With gzip it's important because -q makes gzip replace SIGPIPE with exit status 2. With bzip2 it's important because with -q bzip2 is completely silent if input is corrupt while other decompressors still give an error message. Avoiding exit status 2 from gzip is important because bzip2 uses exit status 2 to indicate corrupt input. Before this commit xzgrep didn't recognize corrupt .bz2 files because xzgrep was treating exit status 2 as SIGPIPE for gzip compatibility. zstd still needs -q because otherwise it is noisy in normal operation. The code to detect real SIGPIPE didn't check if the exit status was due to a signal (>= 128) and so could ignore some other exit status too. src/scripts/xzgrep.in | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) commit 194029ffaf74282a81f0c299c07f73caca3232ca Author: Lasse Collin Date: 2021-01-11 22:01:51 +0200 Scripts: Fix exit status of xzdiff/xzcmp. This is a minor fix since this affects only the situation when the files differ and the exit status is something else than 0. In such case there could be SIGPIPE from a decompression tool and that would result in exit status of 2 from xzdiff/xzcmp while the correct behavior would be to return 1 or whatever else diff or cmp may have returned. This commit omits the -q option from xz/gzip/bzip2/lzop arguments. I'm not sure why the -q was used in the first place, perhaps it hides warnings in some situation that I cannot see at the moment. Hopefully the removal won't introduce a new bug. With gzip the -q option was harmful because it made gzip return 2 instead of >= 128 with SIGPIPE. Ignoring exit status 2 (warning from gzip) isn't practical because bzip2 uses exit status 2 to indicate corrupt input file. It's better if SIGPIPE results in exit status >= 128. With bzip2 the removal of -q seems to be good because with -q it prints nothing if input is corrupt. The other tools aren't silent in this situation even with -q. On the other hand, if zstd support is added, it will need -q since otherwise it's noisy in normal situations. Thanks to Étienne Mollier and Sebastian Andrzej Siewior. src/scripts/xzdiff.in | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) commit f7fa309e1f7178d04c7bedc03b73077639371e97 Author: Lasse Collin Date: 2021-01-09 21:14:36 +0200 liblzma: Make lzma_outq usable for threaded decompression too. Before this commit all output queue buffers were allocated as a single big allocation. Now each buffer is allocated separately when needed. Used buffers are cached to avoid reallocation overhead but the cache will keep only one buffer size at a time. This should make things work OK in the decompression where most of the time the buffer sizes will be the same but with some less common files the buffer sizes may vary. While this should work fine, it's still a bit preliminary and may even get reverted if it turns out to be useless for decompression. src/liblzma/common/outqueue.c | 268 +++++++++++++++++++++------------ src/liblzma/common/outqueue.h | 138 ++++++++++++----- src/liblzma/common/stream_encoder_mt.c | 52 ++++--- 3 files changed, 301 insertions(+), 157 deletions(-) commit a35a69d693ce37d4ba7c1855bda7d9cfa13d1778 Author: Lasse Collin Date: 2020-12-23 17:15:49 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 4fd79b90c52396d70e0b1206ceb1a873a0ad2589 Author: H.J. Lu Date: 2020-12-23 06:49:04 -0800 liblzma: Enable Intel CET in x86 CRC assembly codes When Intel CET is enabled, we need to include in assembly codes to mark Intel CET support and add _CET_ENDBR to indirect jump targets. Tested on Intel Tiger Lake under CET enabled Linux. src/liblzma/check/crc32_x86.S | 9 +++++++++ src/liblzma/check/crc64_x86.S | 9 +++++++++ 2 files changed, 18 insertions(+) commit bb3b8c6a23e25db79f862b1de325c56052e0354b Author: Lasse Collin Date: 2020-12-16 18:33:29 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 21588ca34af98738954fc12ded1b89d7294ef646 Author: Lasse Collin Date: 2020-12-16 18:30:14 +0200 Build: Don't build bundles on Apple OSes. Thanks to Daniel Packard. CMakeLists.txt | 3 +++ 1 file changed, 3 insertions(+) commit d05b0c42dd8b38d8c6b8193c8af50e9bd3d16f28 Author: Lasse Collin Date: 2020-12-05 22:44:03 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 1890351f3423627ba5c4c495402f32d7e9ed90b7 Author: Adam Borowski Date: 2020-09-25 03:35:18 +0200 Scripts: Add zstd support to xzgrep. Thanks to Adam Borowski. src/scripts/xzgrep.1 | 9 ++++++--- src/scripts/xzgrep.in | 1 + 2 files changed, 7 insertions(+), 3 deletions(-) commit 2f108abb3d82e4e2313b438dae9c0c7c7a6366f2 Author: Lasse Collin Date: 2020-11-17 20:51:48 +0200 CMake: Fix compatibility with CMake 3.13. The syntax "if(DEFINED CACHE{FOO})" requires CMake 3.14. In some other places the code treats the cache variables like normal variables already (${FOO} or if(FOO) is used, not ${CACHE{FOO}). Thanks to ygrek for reporting the bug on IRC. CMakeLists.txt | 2 +- cmake/tuklib_cpucores.cmake | 4 ++-- cmake/tuklib_physmem.cmake | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) commit 5af726a79273fafa5de5745b117e567f21c90e49 Author: Lasse Collin Date: 2020-11-01 22:56:43 +0200 Update THANKS. THANKS | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) commit 4575d9d365c756ec189899f9f743e0b3515ce72d Author: Lasse Collin Date: 2020-11-01 22:34:25 +0200 xz: Avoid unneeded \f escapes on the man page. I don't want to use \c in macro arguments but groff_man(7) suggests that \f has better portability. \f would be needed for the .TP strings for portability reasons anyway. Thanks to Bjarni Ingi Gislason. src/xz/xz.1 | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) commit 620b32f5339f86710cb4435e01ecdac972ccac73 Author: Lasse Collin Date: 2020-11-01 19:09:53 +0200 xz: Use non-breaking spaces when intentionally using more than one space. This silences some style checker warnings. Seems that spaces in the beginning of a line don't need this treatment. Thanks to Bjarni Ingi Gislason. src/xz/xz.1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit cb1f34988c8a4130485091b2f8b641303d8f701b Author: Lasse Collin Date: 2020-11-01 18:49:37 +0200 xz: Protect the ellipsis (...) on the man page with \&. This does it only when ... appears outside macro calls. Thanks to Bjarni Ingi Gislason. src/xz/xz.1 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 5d224da3da87400f2fab313abbd7c710e7169ef9 Author: Lasse Collin Date: 2020-11-01 18:41:21 +0200 xz: Avoid the abbreviation "e.g." on the man page. A few are simply omitted, most are converted to "for example" and surrounded with commas. Sounds like that this is better style, for example, man-pages(7) recommends avoiding such abbreviations except in parenthesis. Thanks to Bjarni Ingi Gislason. src/xz/xz.1 | 66 ++++++++++++++++++++++++++++++------------------------------- 1 file changed, 33 insertions(+), 33 deletions(-) commit 90457dbe3e5717660f5b81f8c604860fc5137c0c Author: Lasse Collin Date: 2020-07-12 23:10:03 +0300 xz man page: Change \- (minus) to \(en (en-dash) for a numeric range. Docs of ancient troff/nroff mention \(em (em-dash) but not \(en and \- was used for both minus and en-dash. I don't know how portable \(en is nowadays but it can be changed back if someone complains. At least GNU groff and OpenBSD's mandoc support it. Thanks to Bjarni Ingi Gislason for the patch. src/xz/xz.1 | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) commit 352ba2d69af2136bc814aa1df1a132559d445616 Author: Lasse Collin Date: 2020-07-12 20:46:24 +0300 Windows: Fix building of resource files when config.h isn't used. Now CMake + Visual Studio works for building liblzma.dll. Thanks to Markus Rickert. src/common/common_w32res.rc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) commit a9e2a87f1d61dcf684d809bf08c8ebea93f8a480 Author: Lasse Collin Date: 2020-04-06 19:31:50 +0300 src/scripts/xzgrep.1: Filenames to xzgrep are optional. xzgrep --help was correct already. src/scripts/xzgrep.1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit a7ba275d9b855d186abb29eb7a4f4cb6d9ca6fe0 Author: Bjarni Ingi Gislason Date: 2020-03-26 22:17:31 +0000 src/script/xzgrep.1: Remove superfluous '.RB' Output is from: test-groff -b -e -mandoc -T utf8 -rF0 -t -w w -z [ "test-groff" is a developmental version of "groff" ] Input file is ./src/scripts/xzgrep.1 :20 (macro RB): only 1 argument, but more are expected :23 (macro RB): only 1 argument, but more are expected :26 (macro RB): only 1 argument, but more are expected :29 (macro RB): only 1 argument, but more are expected :32 (macro RB): only 1 argument, but more are expected "abc..." does not mean the same as "abc ...". The output from nroff and troff is unchanged except for the space between "file" and "...". Signed-off-by: Bjarni Ingi Gislason src/scripts/xzgrep.1 | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) commit 133d498db0f4b14f066d192d64dbcade45deae6b Author: Bjarni Ingi Gislason Date: 2020-03-30 21:56:36 +0000 xzgrep.1: Delete superfluous '.PP' Summary: mandoc -T lint xzgrep.1 : mandoc: xzgrep.1:79:2: WARNING: skipping paragraph macro: PP empty There is no change in the output of "nroff" and "troff". Signed-off-by: Bjarni Ingi Gislason src/scripts/xzgrep.1 | 1 - 1 file changed, 1 deletion(-) commit 057839ca982f886387b66746bffe749cb14fd8cd Author: Bjarni Ingi Gislason Date: 2020-03-26 21:16:18 +0000 src/xz/xz.1: Correct misused two-fonts macros Output is from: test-groff -b -e -mandoc -T utf8 -rF0 -t -w w -z [ "test-groff" is a developmental version of "groff" ] Input file is ./src/xz/xz.1 :408 (macro BR): only 1 argument, but more are expected :1009 (macro BR): only 1 argument, but more are expected :1743 (macro BR): only 1 argument, but more are expected :1920 (macro BR): only 1 argument, but more are expected :2213 (macro BR): only 1 argument, but more are expected Output from nroff and troff is unchanged, except for a font change of a full stop (.). Signed-off-by: Bjarni Ingi Gislason src/xz/xz.1 | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) commit b8e12f5ab4c9fd3cb09a4330b2861f6b979ababd Author: Lasse Collin Date: 2020-03-23 18:07:50 +0200 Typo fixes from fossies.org. https://fossies.org/linux/misc/xz-5.2.5.tar.xz/codespell.html Makefile.am | 2 +- doc/examples/01_compress_easy.c | 2 +- src/liblzma/api/lzma/base.h | 2 +- src/liblzma/check/crc32_x86.S | 2 +- src/liblzma/common/index.c | 2 +- src/xz/xz.1 | 4 ++-- 6 files changed, 7 insertions(+), 7 deletions(-) commit 869b9d1b4edd6df07f819d360d306251f8147353 Author: Lasse Collin Date: 2020-03-17 16:24:28 +0200 Update NEWS for 5.2.5. NEWS | 105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) commit a048e3a92d238c65f050a765174d9c75417231d4 Author: Lasse Collin Date: 2020-03-16 20:01:37 +0200 README: Update outdated sections. README | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) commit 29aed815ad4f98f3e4d355faa76a244ecd8ce716 Author: Lasse Collin Date: 2020-03-16 19:39:45 +0200 README: Mention that man pages can be translated. README | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) commit 7fa7653940cc9dcfcbce2fbc5166ea343ad4e3c1 Author: Lasse Collin Date: 2020-03-16 16:43:29 +0200 Update INSTALL.generic from Automake 1.16.1. INSTALL.generic | 321 ++++++++++++++++++++++++++++---------------------------- 1 file changed, 162 insertions(+), 159 deletions(-) commit 9bd317ef03ab9b3e6a927c27c2e9c4ac041182f0 Author: Lasse Collin Date: 2020-03-15 15:27:22 +0200 Update INSTALL for Windows and DOS and add preliminary info for z/OS. INSTALL | 51 +++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 41 insertions(+), 10 deletions(-) commit a3148c0446dc7fa96363752df414d22539c9007b Author: Lasse Collin Date: 2020-03-15 15:26:20 +0200 Build: Update m4/ax_pthread.m4 from Autoconf Archive (again). m4/ax_pthread.m4 | 219 +++++++++++++++++++++++++++++-------------------------- 1 file changed, 117 insertions(+), 102 deletions(-) commit 7812002dd3ed319e42a14662a8531802cca8ca67 Author: Lasse Collin Date: 2020-03-11 21:15:35 +0200 xz: Never use thousand separators in DJGPP builds. DJGPP 2.05 added support for thousands separators but it's broken at least under WinXP with Finnish locale that uses a non-breaking space as the thousands separator. Workaround by disabling thousands separators for DJGPP builds. src/xz/util.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) commit 7c8f688bf7fccd65d396e0130cbf4ea5dff5c56f Author: Lasse Collin Date: 2020-03-11 19:38:08 +0200 DOS: Update dos/Makefile for DJGPP 2.05. It doesn't need -fgnu89-inline like 2.04beta did. dos/Makefile | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) commit 319ca928d73de87940c54e30bffe69f9fa65efdf Author: Lasse Collin Date: 2020-03-11 19:36:07 +0200 DOS: Update instructions in dos/INSTALL.txt. dos/INSTALL.txt | 59 ++++++++++++++++++++++++++++----------------------------- 1 file changed, 29 insertions(+), 30 deletions(-) commit cb6b227ce39932824812ccd8a0647bd968de27d2 Author: Lasse Collin Date: 2020-03-11 17:58:51 +0200 DOS: Update config.h. The added defines assume GCC >= 4.8. dos/config.h | 8 ++++++++ 1 file changed, 8 insertions(+) commit 4572d53e16e87eee375bc5624de2fd59bb0ae9cd Author: Lasse Collin Date: 2020-03-02 13:54:33 +0200 liblzma: Fix a comment and RC_SYMBOLS_MAX. The comment didn't match the value of RC_SYMBOLS_MAX and the value itself was slightly larger than actually needed. The only harm about this was that memory usage was a few bytes larger. src/liblzma/rangecoder/range_encoder.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 265daa873c0d871f5f23f9b56e133a6f20045a0a Author: Lasse Collin Date: 2020-02-27 20:58:52 +0200 Build: Make CMake build fail if tuklib_cpucores or tuklib_physmem fails. CMakeLists.txt | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) commit 7c8b904527cdbe61248c80edcc2e20d840c4fef9 Author: Lasse Collin Date: 2020-02-27 20:24:27 +0200 Build: Add support for --no-po4a option to autogen.sh. Normally, if po4a isn't available, autogen.sh will return with non-zero exit status. The option --no-po4a can be useful when one knows that po4a isn't available but wants autogen.sh to still return with zero exit status. autogen.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) commit 292a5c0f9c9b3a66f5a5c652dc46381836d4537f Author: Lasse Collin Date: 2020-02-25 21:35:14 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 474320e9908786ba2021035f9013191e16cde08a Author: Lasse Collin Date: 2020-02-25 20:42:31 +0200 Build: Fix bugs in the CMake files. Seems that the phrase "add more quotes" from sh/bash scripting applies to CMake as well. E.g. passing an unquoted list ${FOO} to a function that expects one argument results in only the first element of the list being passed as an argument and the rest get ignored. Adding quotes helps ("${FOO}"). list(INSERT ...) is weird. Inserting an empty string to an empty variable results in empty list, but inserting it to a non-empty variable does insert an empty element to the list. Since INSERT requires at least one element, "${CMAKE_THREAD_LIBS_INIT}" needs to be quoted in CMakeLists.txt. It might result in an empty element in the list. It seems to not matter as empty elements consistently get ignored in that variable. In fact, calling cmake_check_push_state() and cmake_check_pop_state() will strip the empty elements from CMAKE_REQUIRED_LIBRARIES! In addition to quoting fixes, this fixes checks for the cache variables in tuklib_cpucores.cmake and tuklib_physmem.cmake. Thanks to Martin Matuška for testing and reporting the problems. These fixes aren't tested yet but hopefully they soon will be. CMakeLists.txt | 52 ++++++++++++++++++++++----------------------- cmake/tuklib_common.cmake | 8 ++++--- cmake/tuklib_cpucores.cmake | 30 ++++++++++++++------------ cmake/tuklib_integer.cmake | 34 +++++++++++++++-------------- cmake/tuklib_mbstr.cmake | 6 +++--- cmake/tuklib_physmem.cmake | 29 +++++++++++++------------ cmake/tuklib_progname.cmake | 4 ++-- 7 files changed, 85 insertions(+), 78 deletions(-) commit 7e3493d40eac0c3fa3d5124097745a70e15c41f6 Author: Lasse Collin Date: 2020-02-24 23:38:16 +0200 Build: Add very limited experimental CMake support. This does *NOT* replace the Autotools-based build system in the foreseeable future. See the comment in the beginning of CMakeLists.txt. So far this has been tested only on GNU/Linux but I commit it anyway to make it easier for others to test. Since I haven't played much with CMake before, it's likely that there are things that have been done in a silly or wrong way and need to be fixed. CMakeLists.txt | 643 ++++++++++++++++++++++++++++++++++++++++++++ cmake/tuklib_common.cmake | 47 ++++ cmake/tuklib_cpucores.cmake | 173 ++++++++++++ cmake/tuklib_integer.cmake | 100 +++++++ cmake/tuklib_mbstr.cmake | 20 ++ cmake/tuklib_physmem.cmake | 149 ++++++++++ cmake/tuklib_progname.cmake | 19 ++ 7 files changed, 1151 insertions(+) commit 21bd4701fca3e9002ce78bc135debca369ed8545 Author: Lasse Collin Date: 2020-02-24 23:37:07 +0200 Update m4/.gitignore. m4/.gitignore | 1 + 1 file changed, 1 insertion(+) commit e094d1d0f196a91ec703e8d0055948feef349ae8 Author: Lasse Collin Date: 2020-02-24 23:29:35 +0200 tuklib: Omit an unneeded from a tests. tuklib_cpucores.c and tuklib_physmem.c don't include even via other files in this package, so clearly that header isn't needed in the tests either (no one has reported build problems due to a missing header in a .c file). m4/tuklib_cpucores.m4 | 1 - m4/tuklib_physmem.m4 | 1 - 2 files changed, 2 deletions(-) commit b3ed19a55fe99a45bd77614e149d39d18498075c Author: Lasse Collin Date: 2020-02-24 23:01:00 +0200 liblzma: Remove unneeded from fastpos_tablegen.c. This file only generates fastpos_table.c. It isn't built as a part of liblzma. src/liblzma/lzma/fastpos_tablegen.c | 1 - 1 file changed, 1 deletion(-) commit 7b8982b29179b3c586e0456dc9ecbd4f58dcea59 Author: Lasse Collin Date: 2020-02-22 14:15:07 +0200 Use defined(__GNUC__) before __GNUC__ in preprocessor lines. This should silence the equivalent of -Wundef in compilers that don't define __GNUC__. src/common/sysdefs.h | 3 ++- src/liblzma/api/lzma.h | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) commit 43dfe04e6209c691cf4fbe3072d4ee91271748f1 Author: Lasse Collin Date: 2020-02-21 17:40:02 +0200 liblzma: Add more uses of lzma_memcmplen() to the normal mode of LZMA. This gives a tiny encoder speed improvement. This could have been done in 2014 after the commit 544aaa3d13554e8640f9caf7db717a96360ec0f6 but it was forgotten. src/liblzma/lzma/lzma_encoder_optimum_normal.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) commit 59e6eb4840b9f52fa3a61544974017279b448216 Author: Lasse Collin Date: 2020-02-21 17:01:15 +0200 Build: Add visibility.m4 from gnulib. Appears that this file used to get included as a side effect of gettext. After the change to gettext version requirements this file no longer got copied to the package and so the build was broken. m4/.gitignore | 1 - m4/visibility.m4 | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+), 1 deletion(-) commit 7fe3ef2eaa53d439cec043727ea1998f4ff0e22a Author: Lasse Collin Date: 2020-02-21 16:10:44 +0200 xz: Silence a warning when sig_atomic_t is long int. It can be true at least on z/OS. src/xz/signals.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit b0a2a77d10940c42b449d47a005bfc2e50ab5db8 Author: Lasse Collin Date: 2020-02-21 15:59:26 +0200 xz: Avoid unneeded access of a volatile variable. src/xz/signals.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 524c2f12c762032b819757aeda8af7c47c4cabce Author: Lasse Collin Date: 2020-02-21 01:24:18 +0200 tuklib_integer.m4: Optimize the check order. The __builtin byteswapping is the preferred one so check for it first. m4/tuklib_integer.m4 | 56 +++++++++++++++++++++++++++------------------------- 1 file changed, 29 insertions(+), 27 deletions(-) commit 57360bb4fd79b358b36d2877db26ac828d1fdfcb Author: Lasse Collin Date: 2020-02-20 18:54:04 +0200 tuklib_exit: Add missing header. strerror() needs which happened to be included via tuklib_common.h -> tuklib_config.h -> sysdefs.h if HAVE_CONFIG_H was defined. This wasn't tested without config.h before so it had worked fine. src/common/tuklib_exit.c | 1 + 1 file changed, 1 insertion(+) commit fddd31175e74a538997a939d930462fde17d2dd4 Author: Lasse Collin Date: 2020-02-18 19:12:35 +0200 Revert the previous commit and add a comment. The previous commit broke crc32_tablegen.c. If the whole package is built without config.h (with defines set on the compiler command line) this should still work fine as long as these headers conform to C99 well enough. src/common/tuklib_config.h | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) commit 4e4e9fbb7e66d45319525ac224bff48fbdd0cf6e Author: Lasse Collin Date: 2020-02-17 23:37:20 +0200 Do not check for HAVE_CONFIG_H in tuklib_config.h. In XZ Utils sysdefs.h takes care of it and the required headers. src/common/tuklib_config.h | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) commit 2d4cef954feba82073951358466a1d614141cf33 Author: Lasse Collin Date: 2020-02-16 11:18:28 +0200 sysdefs.h: Omit the conditionals around string.h and limits.h. string.h is used unconditionally elsewhere in the project and configure has always stopped if limits.h is missing, so these headers must have been always available even on the weirdest systems. src/common/sysdefs.h | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) commit feb9c1969bc3eb33d4ecb72cfa897f92dae84939 Author: Lasse Collin Date: 2020-02-15 15:07:11 +0200 Build: Bump Autoconf and Libtool version requirements. There is no specific reason for this other than blocking the most ancient versions. These are still old: Autoconf 2.69 (2012) Automake 1.12 (2012) gettext 0.19.6 (2015) Libtool 2.4 (2010) configure.ac | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 3d576cf92158d62790017ad7f2dd6dc1dd6b42bb Author: Lasse Collin Date: 2020-02-15 03:08:32 +0200 Build: Use AM_GNU_GETTEXT_REQUIRE_VERSION and require 0.19.6. This bumps the version requirement from 0.19 (from 2014) to 0.19.6 (2015). Using only the old AM_GNU_GETTEXT_VERSION results in old gettext infrastructure being placed in the package. By using both macros we get the latest gettext files while the other programs in the Autotools family can still see the old macro. configure.ac | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) commit fa792b8befaf7cb3960b655e0a9410da866d756f Author: Lasse Collin Date: 2020-02-14 20:42:06 +0200 Translations: Add German translation of the man pages. Thanks to Mario Blättermann. po4a/de.po | 5532 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ po4a/po4a.conf | 2 +- 2 files changed, 5533 insertions(+), 1 deletion(-) commit 6f7211b6bb47a895b47f533282dba9ee9a1b0c8b Author: Lasse Collin Date: 2020-02-07 15:32:21 +0200 Build: Add support for translated man pages using po4a. The dependency on po4a is optional. It's never required to install the translated man pages when xz is built from a release tarball. If po4a is missing when building from xz.git, the translated man pages won't be generated but otherwise the build will work normally. The translations are only updated automatically by autogen.sh and by "make mydist". This makes it easy to keep po4a as an optional dependency and ensures that I won't forget to put updated translations to a release tarball. The translated man pages aren't installed if --disable-nls is used. The installation of translated man pages abuses Automake internals by calling "install-man" with redefined dist_man_MANS and man_MANS. This makes the hairy script code slightly less hairy. If it breaks some day, this code needs to be fixed; don't blame Automake developers. Also, this adds more quotes to the existing shell script code in the Makefile.am "-hook"s. Makefile.am | 4 ++++ autogen.sh | 8 ++++--- po4a/.gitignore | 2 ++ po4a/po4a.conf | 14 +++++++++++ po4a/update-po | 45 ++++++++++++++++++++++++++++++++++ src/scripts/Makefile.am | 64 +++++++++++++++++++++++++++++++++++++------------ src/xz/Makefile.am | 50 +++++++++++++++++++++++++++----------- src/xzdec/Makefile.am | 55 ++++++++++++++++++++++++++++++++---------- 8 files changed, 197 insertions(+), 45 deletions(-) commit 426f9e5819ff7710a5ff573a96c02940be65d52f Author: Lasse Collin Date: 2020-02-06 17:31:38 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit e3a4481d020e4de89efa037f335cf50f3ca55592 Author: Lasse Collin Date: 2020-02-05 22:35:06 +0200 Update tests/.gitignore. .gitignore | 4 ++++ 1 file changed, 4 insertions(+) commit 07208de92f2d5bca764f6d0ebe9d9866051dc4ef Author: Lasse Collin Date: 2020-02-05 22:28:51 +0200 Update m4/.gitignore. m4/.gitignore | 1 + 1 file changed, 1 insertion(+) commit c91fbf223db46c3b3cb9df769863a1a60cd9c908 Author: Lasse Collin Date: 2020-02-05 20:47:38 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 15a133b6d1a3eab4faf6eb52a71fdc56bd65846f Author: Lasse Collin Date: 2020-02-05 20:40:14 +0200 xz: Make it a fatal error if enabling the sandbox fails. Perhaps it's too drastic but on the other hand it will let me learn about possible problems if people report the errors. This won't be backported to the v5.2 branch. src/xz/file_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit af0fb386ef55db66654ae39e2deec6e04190c4ff Author: Lasse Collin Date: 2020-02-05 20:33:50 +0200 xz: Comment out annoying sandboxing messages. src/xz/file_io.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) commit 986d8c9b52b824474088e5bb3b6940651660f0e2 Author: Lasse Collin Date: 2020-02-05 19:33:37 +0200 Build: Workaround a POSIX shell detection problem on Solaris. I don't know if the problem is in gnulib's gl_POSIX_SHELL macro or if xzgrep does something that isn't in POSIX. The workaround adds a special case for Solaris: if /usr/xpg4/bin/sh exists and gl_cv_posix_shell wasn't overriden on the configure command line, use that shell for xzgrep and other scripts. That shell is known to work and exists on most Solaris systems. configure.ac | 10 ++++++++++ 1 file changed, 10 insertions(+) commit 6629ed929cc7d45a11e385f357ab58ec15e7e4ad Author: Lasse Collin Date: 2020-02-03 22:03:50 +0200 Build: Update m4/ax_pthread.m4 from Autoconf Archive. m4/ax_pthread.m4 | 398 ++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 279 insertions(+), 119 deletions(-) commit 353970510895f6a80adfe60cf71b70a95adfa8bc Author: Lasse Collin Date: 2020-02-01 19:56:18 +0200 xz: Limit --memlimit-compress to at most 4020 MiB for 32-bit xz. See the code comment for reasoning. It's far from perfect but hopefully good enough for certain cases while hopefully doing nothing bad in other situations. At presets -5 ... -9, 4020 MiB vs. 4096 MiB makes no difference on how xz scales down the number of threads. The limit has to be a few MiB below 4096 MiB because otherwise things like "xz --lzma2=dict=500MiB" won't scale down the dict size enough and xz cannot allocate enough memory. With "ulimit -v $((4096 * 1024))" on x86-64, the limit in xz had to be no more than 4085 MiB. Some safety margin is good though. This is hack but it should be useful when running 32-bit xz on a 64-bit kernel that gives full 4 GiB address space to xz. Hopefully this is enough to solve this: https://bugzilla.redhat.com/show_bug.cgi?id=1196786 FreeBSD has a patch that limits the result in tuklib_physmem() to SIZE_MAX on 32-bit systems. While I think it's not the way to do it, the results on --memlimit-compress have been good. This commit should achieve practically identical results for compression while leaving decompression and tuklib_physmem() and thus lzma_physmem() unaffected. src/xz/hardware.c | 32 +++++++++++++++++++++++++++++++- src/xz/xz.1 | 21 ++++++++++++++++++++- 2 files changed, 51 insertions(+), 2 deletions(-) commit ba76d67585f88677af9f48b48e7bdc3bb7687def Author: Lasse Collin Date: 2020-01-26 20:53:25 +0200 xz: Set the --flush-timeout deadline when the first input byte arrives. xz --flush-timeout=2000, old version: 1. xz is started. The next flush will happen after two seconds. 2. No input for one second. 3. A burst of a few kilobytes of input. 4. No input for one second. 5. Two seconds have passed and flushing starts. The first second counted towards the flush-timeout even though there was no pending data. This can cause flushing to occur more often than needed. xz --flush-timeout=2000, after this commit: 1. xz is started. 2. No input for one second. 3. A burst of a few kilobytes of input. The next flush will happen after two seconds counted from the time when the first bytes of the burst were read. 4. No input for one second. 5. No input for another second. 6. Two seconds have passed and flushing starts. src/xz/coder.c | 6 +----- src/xz/file_io.c | 6 +++++- src/xz/mytime.c | 1 - 3 files changed, 6 insertions(+), 7 deletions(-) commit fd47fd62bbb1bfd13ab63869137971d8b390025f Author: Lasse Collin Date: 2020-01-26 20:19:19 +0200 xz: Move flush_needed from mytime.h to file_pair struct in file_io.h. src/xz/coder.c | 3 ++- src/xz/file_io.c | 3 ++- src/xz/file_io.h | 3 +++ src/xz/mytime.c | 3 --- src/xz/mytime.h | 4 ---- 5 files changed, 7 insertions(+), 9 deletions(-) commit 815035681063d5774d3640fc20b8ede783dd574e Author: Lasse Collin Date: 2020-01-26 14:49:22 +0200 xz: coder.c: Make writing output a separate function. The same code sequence repeats so it's nicer as a separate function. Note that in one case there was no test for opt_mode != MODE_TEST, but that was only because that condition would always be true, so this commit doesn't change the behavior there. src/xz/coder.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) commit 5a49e081a098455bcdbd95cefb90e9b18780fe58 Author: Lasse Collin Date: 2020-01-26 14:13:42 +0200 xz: Fix semi-busy-waiting in xz --flush-timeout. When input blocked, xz --flush-timeout=1 would wake up every millisecond and initiate flushing which would have nothing to flush and thus would just waste CPU time. The fix disables the timeout when no input has been seen since the previous flush. src/xz/coder.c | 4 ++++ src/xz/file_io.c | 15 +++++++++++---- src/xz/file_io.h | 4 ++++ 3 files changed, 19 insertions(+), 4 deletions(-) commit dcca70fe9fa3c4bec56cf9c79e966166c9a9cf6a Author: Lasse Collin Date: 2020-01-26 13:47:31 +0200 xz: Refactor io_read() a bit. src/xz/file_io.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) commit 4ae9ab70cd3214395756435d13d8d000368ca2cb Author: Lasse Collin Date: 2020-01-26 13:37:08 +0200 xz: Update a comment in file_io.h. src/xz/file_io.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) commit 3333ba4a6795a55cf0375329ba08152bd7fcbd46 Author: Lasse Collin Date: 2020-01-26 13:27:51 +0200 xz: Move the setting of flush_needed in file_io.c to a nicer location. src/xz/file_io.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) commit cf2df0f05ac98c1158c6e48145900b773223605d Author: Lasse Collin Date: 2020-01-19 21:54:33 +0200 Use $(LIB_FUZZING_ENGINE) in tests/ossfuzz/Makefile. https://github.com/google/oss-fuzz/pull/3219#issuecomment-573751048 Thanks to Bhargava Shastry for sending the patch. tests/ossfuzz/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 7136f1735c60ac6967c4b8e277fcde53d485234f Author: Lasse Collin Date: 2019-12-31 00:41:28 +0200 Rename unaligned_read32ne to read32ne, and similarly for the others. src/common/tuklib_integer.h | 64 +++++++++++++++---------------- src/liblzma/common/alone_encoder.c | 2 +- src/liblzma/common/block_header_decoder.c | 2 +- src/liblzma/common/block_header_encoder.c | 2 +- src/liblzma/common/memcmplen.h | 9 ++--- src/liblzma/common/stream_flags_decoder.c | 6 +-- src/liblzma/common/stream_flags_encoder.c | 8 ++-- src/liblzma/lz/lz_encoder_hash.h | 2 +- src/liblzma/lzma/lzma_decoder.c | 2 +- src/liblzma/lzma/lzma_encoder.c | 2 +- src/liblzma/lzma/lzma_encoder_private.h | 3 +- src/liblzma/simple/simple_decoder.c | 2 +- src/liblzma/simple/simple_encoder.c | 2 +- tests/test_block_header.c | 4 +- tests/test_stream_flags.c | 6 +-- 15 files changed, 54 insertions(+), 62 deletions(-) commit 5e78fcbf2eb21936022c9c5c3625d4da76f4b241 Author: Lasse Collin Date: 2019-12-31 00:29:48 +0200 Rename read32ne to aligned_read32ne, and similarly for the others. Using the aligned methods requires more care to ensure that the address really is aligned, so it's nicer if the aligned methods are prefixed. The next commit will remove the unaligned_ prefix from the unaligned methods which in liblzma are used in more places than the aligned ones. src/common/tuklib_integer.h | 56 +++++++++++++++++++++--------------------- src/liblzma/check/crc32_fast.c | 4 +-- src/liblzma/check/crc64_fast.c | 4 +-- 3 files changed, 32 insertions(+), 32 deletions(-) commit 77bc5bc6dd67056cfd5888520ac930cfc57b4516 Author: Lasse Collin Date: 2019-12-31 00:18:24 +0200 Revise tuklib_integer.h and .m4. Add a configure option --enable-unsafe-type-punning to get the old non-conforming memory access methods. It can be useful with old compilers or in some other less typical situations but shouldn't normally be used. Omit the packed struct trick for unaligned access. While it's best in some cases, this is simpler. If the memcpy trick doesn't work, one can request unsafe type punning from configure. Because CRC32/CRC64 code needs fast aligned reads, if no very safe way to do it is found, type punning is used as a fallback. This sucks but since it currently works in practice, it seems to be the least bad option. It's never needed with GCC >= 4.7 or Clang >= 3.6 since these support __builtin_assume_aligned and thus fast aligned access can be done with the memcpy trick. Other things: - Support GCC/Clang __builtin_bswapXX - Cleaner bswap fallback macros - Minor cleanups m4/tuklib_integer.m4 | 46 ++++- src/common/tuklib_integer.h | 488 ++++++++++++++++++++++++-------------------- 2 files changed, 316 insertions(+), 218 deletions(-) commit 8b72950a6b2e2a36c2d8fdc8857564b57191b088 Author: Lasse Collin Date: 2019-12-29 22:51:58 +0200 Tests: Hopefully fix test_check.c to work on EBCDIC systems. Thanks to Daniel Richard G. tests/test_check.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) commit 43ce4ea7c762238d3df9717b34126d3e0d7cd51c Author: Lasse Collin Date: 2019-09-24 23:02:40 +0300 Scripts: Put /usr/xpg4/bin to the beginning of PATH on Solaris. This adds a configure option --enable-path-for-scripts=PREFIX which defaults to empty except on Solaris it is /usr/xpg4/bin to make POSIX grep and others available. The Solaris case had been documented in INSTALL with a manual fix but it's better to do this automatically since it is needed on most Solaris systems anyway. Thanks to Daniel Richard G. INSTALL | 43 +++++++++++++++++++++++++++++++++++-------- configure.ac | 26 ++++++++++++++++++++++++++ src/scripts/xzdiff.in | 1 + src/scripts/xzgrep.in | 1 + src/scripts/xzless.in | 1 + src/scripts/xzmore.in | 1 + 6 files changed, 65 insertions(+), 8 deletions(-) commit 6a89e656ebedd53a10cd1a063a32a9e4ade0da1f Author: Lasse Collin Date: 2019-07-12 18:57:43 +0300 Fix comment typos in tuklib_mbstr* files. src/common/tuklib_mbstr.h | 2 +- src/common/tuklib_mbstr_fw.c | 2 +- src/common/tuklib_mbstr_width.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) commit ac0b4212656a48ef0c187c0c941d40ac9489ae36 Author: Lasse Collin Date: 2019-07-12 18:30:46 +0300 Add missing include to tuklib_mbstr_width.c. It didn't matter in XZ Utils because sysdefs.h includes string.h anyway. src/common/tuklib_mbstr_width.c | 1 + 1 file changed, 1 insertion(+) commit 72a443281fb0b91aebf8cdff2ab1f7c07b081240 Author: Lasse Collin Date: 2019-07-12 18:10:57 +0300 Update tuklib base headers to include stdbool.h. src/common/tuklib_common.h | 2 +- src/common/tuklib_config.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) commit de1f47b2b40e960b7bc3acba754f66dd19705921 Author: Lasse Collin Date: 2019-06-28 00:54:31 +0300 xz: Automatically align the strings in --info-memory. This makes it easier to translate the strings. Also, the string for amount of RAM was shortened. src/xz/hardware.c | 45 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 34 insertions(+), 11 deletions(-) commit 8ce679125dbd0e2058d8f886e738d7f19a45cab5 Author: Lasse Collin Date: 2019-06-25 23:15:21 +0300 liblzma: Fix a buggy comment. src/liblzma/lz/lz_encoder_mf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit fc77929e92e869f6869bf88931066103fd75f376 Author: Lasse Collin Date: 2019-06-25 00:16:06 +0300 configure.ac: Fix a typo in a comment. configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit e873902641794210ad7db59743f98e3e1cd6139f Author: Lasse Collin Date: 2019-06-25 00:08:13 +0300 Tests: Silence warnings from clang -Wassign-enum. Also changed 999 to 99 so it fits even if lzma_check happened to be 8 bits wide. tests/test_block_header.c | 3 ++- tests/test_stream_flags.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) commit d499e467d99efeaae688564eedc4548837c1416a Author: Lasse Collin Date: 2019-06-24 23:52:17 +0300 liblzma: Add a comment. src/liblzma/common/stream_encoder_mt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit a12b13c5f0d54c684fa8446f93fdac08ab2a716b Author: Lasse Collin Date: 2019-06-24 23:45:21 +0300 liblzma: Silence clang -Wmissing-variable-declarations. src/liblzma/check/crc32_table.c | 3 +++ src/liblzma/check/crc64_table.c | 3 +++ 2 files changed, 6 insertions(+) commit 1b4675cebf7471f7cc9b7072c950e3de97147063 Author: Lasse Collin Date: 2019-06-24 23:25:41 +0300 Add LZMA_RET_INTERNAL1..8 to lzma_ret and use one for LZMA_TIMED_OUT. LZMA_TIMED_OUT is *internally* used as a value for lzma_ret enumeration. Previously it was #defined to 32 and cast to lzma_ret. That way it wasn't visible in the public API, but this was hackish. Now the public API has eight LZMA_RET_INTERNALx members and LZMA_TIMED_OUT is #defined to LZMA_RET_INTERNAL1. This way the code is cleaner overall although the public API has a few extra mysterious enum members. src/liblzma/api/lzma/base.h | 15 ++++++++++++++- src/liblzma/common/common.c | 4 +--- src/liblzma/common/common.h | 5 ++--- src/xz/message.c | 8 ++++++++ 4 files changed, 25 insertions(+), 7 deletions(-) commit 159c43875eb25deea626ed651274464bae3e32ef Author: Lasse Collin Date: 2019-06-24 22:57:43 +0300 xz: Silence a warning from clang -Wsign-conversion in main.c. src/xz/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 466cfcd3e52f6750ce28a635997f3dd84fb18515 Author: Lasse Collin Date: 2019-06-24 22:52:20 +0300 xz: Make "headings" static in list.c. Caught by clang -Wmissing-variable-declarations. src/xz/list.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 608517b9b76c41fac6613dbda1193d6f41338e19 Author: Lasse Collin Date: 2019-06-24 22:47:39 +0300 liblzma: Remove incorrect uses of lzma_attribute((__unused__)). Caught by clang -Wused-but-marked-unused. src/liblzma/common/alone_decoder.c | 3 +-- src/liblzma/common/alone_encoder.c | 3 +-- src/liblzma/lz/lz_decoder.c | 3 +-- 3 files changed, 3 insertions(+), 6 deletions(-) commit c2d2ab6a9d41a2b55d047c5b710aacf80d219255 Author: Lasse Collin Date: 2019-06-24 20:53:55 +0300 Tests: Silence a warning from -Wsign-conversion. tests/create_compress_files.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) commit 2402f7873dcae719d0ebddd23bb579074519ac52 Author: Lasse Collin Date: 2019-06-24 20:45:49 +0300 xz: Fix an integer overflow with 32-bit off_t. Or any off_t which isn't very big (like signed 64 bit integer that most system have). A small off_t could overflow if the file being decompressed had long enough run of zero bytes, which would result in corrupt output. src/xz/file_io.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) commit 4fd3a8dd0b60f029e1c66a0ee634f9e9fda3caa9 Author: Lasse Collin Date: 2019-06-24 01:24:17 +0300 xz: Cleanup io_seek_src() a bit. lseek() returns -1 on error and checking for -1 is nicer. src/xz/file_io.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) commit dfda7cf6afa486e10df035327d68753896dfb48a Author: Lasse Collin Date: 2019-06-24 00:57:23 +0300 Tests: Remove a duplicate branch from tests/tests.h. The duplication was introduced about eleven years ago and should have been cleaned up back then already. This was caught by -Wduplicated-branches. tests/tests.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) commit 1d4a904d8fb634bd5a04f7fbdd17d3739f3d8866 Author: Lasse Collin Date: 2019-06-24 00:40:45 +0300 xz: Change io_seek_src and io_pread arguments from off_t to uint64_t. This helps fixing warnings from -Wsign-conversion and makes the code look better too. src/xz/file_io.c | 16 ++++++++++++---- src/xz/file_io.h | 4 ++-- src/xz/list.c | 9 ++++----- 3 files changed, 18 insertions(+), 11 deletions(-) commit 50120deb0159fcb53ee1a6caffb2bb81a1ecd990 Author: Lasse Collin Date: 2019-06-24 00:12:38 +0300 xz: list.c: Fix some warnings from -Wsign-conversion. src/xz/list.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) commit d0a78751eb54fb1572002746c533936a118e4e42 Author: Lasse Collin Date: 2019-06-23 23:22:45 +0300 tuklib_mbstr_width: Fix a warning from -Wsign-conversion. src/common/tuklib_mbstr_width.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 7883d73530b4b2a701ddd7d50c35676cbc158039 Author: Lasse Collin Date: 2019-06-23 23:19:34 +0300 xz: Fix some of the warnings from -Wsign-conversion. src/xz/args.c | 4 ++-- src/xz/coder.c | 4 ++-- src/xz/file_io.c | 5 +++-- src/xz/message.c | 4 ++-- src/xz/mytime.c | 4 ++-- src/xz/options.c | 2 +- src/xz/util.c | 4 ++-- 7 files changed, 14 insertions(+), 13 deletions(-) commit c2b994fe3d35e9e575c28869a2f7f534f2495d05 Author: Lasse Collin Date: 2019-06-23 22:27:45 +0300 tuklib_cpucores: Silence warnings from -Wsign-conversion. src/common/tuklib_cpucores.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) commit 07c4fa9e1a195e0543f271380c8de22a3ab145ff Author: Lasse Collin Date: 2019-06-23 21:40:47 +0300 xzdec: Fix warnings from -Wsign-conversion. src/xzdec/xzdec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit dfac2c9a1d7d4a2b8a5d7c9c6d567dee48318bcf Author: Lasse Collin Date: 2019-06-23 21:38:56 +0300 liblzma: Fix warnings from -Wsign-conversion. Also, more parentheses were added to the literal_subcoder macro in lzma_comon.h (better style but no functional change in the current usage). src/liblzma/common/block_header_decoder.c | 2 +- src/liblzma/delta/delta_decoder.c | 2 +- src/liblzma/lzma/fastpos.h | 2 +- src/liblzma/lzma/lzma2_decoder.c | 8 ++++---- src/liblzma/lzma/lzma_common.h | 3 ++- src/liblzma/lzma/lzma_decoder.c | 16 ++++++++-------- src/liblzma/simple/arm.c | 6 +++--- src/liblzma/simple/armthumb.c | 8 ++++---- src/liblzma/simple/ia64.c | 2 +- src/liblzma/simple/powerpc.c | 9 +++++---- src/liblzma/simple/x86.c | 2 +- 11 files changed, 31 insertions(+), 29 deletions(-) commit 41838dcc26375f6aa393a63e4d81e2f4d223de07 Author: Lasse Collin Date: 2019-06-23 19:33:55 +0300 tuklib_integer: Silence warnings from -Wsign-conversion. src/common/tuklib_integer.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit 3ce05d235f736d392347a05086b8033416874b87 Author: Lasse Collin Date: 2019-06-20 19:40:30 +0300 tuklib_integer: Fix usage of conv macros. Use a temporary variable instead of e.g. conv32le(unaligned_read32ne(buf)) because the macro can evaluate its argument multiple times. src/common/tuklib_integer.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) commit b525b0c0ef40cd89b69294c9b8d57f4a8db58e1f Author: Lasse Collin Date: 2019-06-03 20:44:19 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 039a168e8cf201d5104a25ec41f0cf25eda6cc53 Author: Lasse Collin Date: 2019-06-03 20:41:54 +0300 liblzma: Fix comments. Thanks to Bruce Stark. src/liblzma/common/alone_encoder.c | 4 ++-- src/liblzma/common/block_util.c | 2 +- src/liblzma/common/common.c | 2 +- src/liblzma/common/filter_common.h | 2 +- src/liblzma/common/filter_decoder.h | 2 +- src/liblzma/common/filter_flags_encoder.c | 2 +- 6 files changed, 7 insertions(+), 7 deletions(-) commit c460f6defebc5a81bbca90adc2476154ca244f69 Author: Lasse Collin Date: 2019-06-02 00:50:59 +0300 liblzma: Fix one more unaligned read to use unaligned_read16ne(). src/liblzma/lz/lz_encoder_hash.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit c81d77c537f0b8c8672868e1dc6cf7290ce4a25b Author: Lasse Collin Date: 2019-06-01 21:41:55 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 386394fc9fcde2615391f804eaa466749f96f4ef Author: Lasse Collin Date: 2019-06-01 21:36:13 +0300 liblzma: memcmplen: Use ctz32() from tuklib_integer.h. The same compiler-specific #ifdefs are already in tuklib_integer.h src/liblzma/common/memcmplen.h | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) commit 264ab971ce2994baac41b1579c9c35aba7743fc8 Author: Lasse Collin Date: 2019-06-01 21:30:03 +0300 tuklib_integer: Cleanup MSVC-specific code. src/common/tuklib_integer.h | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) commit 33773c6f2a8711d4aa6656795db52c59a28580ec Author: Lasse Collin Date: 2019-06-01 19:01:21 +0300 liblzma: Use unaligned_readXXne functions instead of type punning. Now gcc -fsanitize=undefined should be clean. Thanks to Jeffrey Walton. src/liblzma/common/memcmplen.h | 12 ++++++------ src/liblzma/lzma/lzma_encoder_private.h | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) commit e5f13a66567b1987e0aae42c6fdcd277bb5810ba Author: Lasse Collin Date: 2019-06-01 18:46:54 +0300 tuklib_integer: Autodetect support for unaligned access on ARM. The result is used as the default for --enable-unaligned-access. The test should work with GCC and Clang. m4/tuklib_integer.m4 | 11 +++++++++++ 1 file changed, 11 insertions(+) commit 3bc112c2d38d5f348bce7bc2422286b1692c7490 Author: Lasse Collin Date: 2019-06-01 18:41:16 +0300 tuklib_integer: Improve unaligned memory access. Now memcpy() or GNU C packed structs for unaligned access instead of type punning. See the comment in this commit for details. Avoiding type punning with unaligned access is needed to silence gcc -fsanitize=undefined. New functions: unaliged_readXXne and unaligned_writeXXne where XX is 16, 32, or 64. src/common/tuklib_integer.h | 180 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 168 insertions(+), 12 deletions(-) commit 2a22de439ec63da1927b640eda309296a1e8dce5 Author: Lasse Collin Date: 2019-05-13 20:05:17 +0300 liblzma: Avoid memcpy(NULL, foo, 0) because it is undefined behavior. I should have always known this but I didn't. Here is an example as a reminder to myself: int mycopy(void *dest, void *src, size_t n) { memcpy(dest, src, n); return dest == NULL; } In the example, a compiler may assume that dest != NULL because passing NULL to memcpy() would be undefined behavior. Testing with GCC 8.2.1, mycopy(NULL, NULL, 0) returns 1 with -O0 and -O1. With -O2 the return value is 0 because the compiler infers that dest cannot be NULL because it was already used with memcpy() and thus the test for NULL gets optimized out. In liblzma, if a null-pointer was passed to memcpy(), there were no checks for NULL *after* the memcpy() call, so I cautiously suspect that it shouldn't have caused bad behavior in practice, but it's hard to be sure, and the problematic cases had to be fixed anyway. Thanks to Jeffrey Walton. src/liblzma/common/common.c | 6 +++++- src/liblzma/lz/lz_decoder.c | 12 +++++++++--- src/liblzma/simple/simple_coder.c | 10 +++++++++- 3 files changed, 23 insertions(+), 5 deletions(-) commit d3fc850cfedc058247d9e334ce59bbc8f2286d8a Author: Lasse Collin Date: 2019-05-11 20:56:08 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 4adb8288ab61d5f14e212007b8742df0710baf73 Author: Lasse Collin Date: 2019-05-11 20:54:12 +0300 xz: Update xz man page date. src/xz/xz.1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 2fb0ddaa557ce86e38fe06439930fa8665f092fd Author: Antoine Cœur Date: 2019-05-08 13:30:57 +0800 spelling Doxyfile.in | 2 +- NEWS | 2 +- doc/examples/11_file_info.c | 2 +- src/liblzma/api/lzma/block.h | 2 +- src/liblzma/api/lzma/hardware.h | 2 +- src/liblzma/api/lzma/lzma12.h | 2 +- src/liblzma/api/lzma/vli.h | 2 +- src/liblzma/common/file_info.c | 4 ++-- src/liblzma/common/hardware_physmem.c | 2 +- src/liblzma/common/index.c | 4 ++-- src/liblzma/common/stream_encoder_mt.c | 2 +- src/liblzma/common/vli_decoder.c | 2 +- src/liblzma/lz/lz_decoder.c | 2 +- src/scripts/xzgrep.in | 2 +- src/xz/args.c | 2 +- src/xz/coder.c | 4 ++-- src/xz/list.c | 4 ++-- src/xz/main.c | 2 +- src/xz/mytime.h | 2 +- src/xz/private.h | 2 +- src/xz/xz.1 | 2 +- windows/build.bash | 2 +- 22 files changed, 26 insertions(+), 26 deletions(-) commit 4ed339606156bd313ed99237485cb8ed0362d64f Author: Lasse Collin Date: 2019-05-01 18:43:10 +0300 xz: In xz -lvv look at the widths of the check names too. Now the widths of the check names is used to adjust the width of the Check column. This way there no longer is a need to restrict the widths of the check names to be at most ten terminal-columns. src/xz/list.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) commit 2f4281a1001dcf7fdf1418c0c0d246c16561fb65 Author: Lasse Collin Date: 2019-05-01 18:33:25 +0300 xz: Fix xz -lvv column alignment to look at the translated strings. src/xz/list.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 01d01b7c7c0b8eaf7f780a5584ec52c22d10fa4a Author: Lasse Collin Date: 2019-05-01 16:52:36 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 64030c6b17f7743df03a9948a0ccfcdf40c6b97c Author: Lasse Collin Date: 2019-05-01 16:43:16 +0300 Windows: Update VS version in windows/vs2019/config.h. windows/vs2019/config.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 2dc9117f5fbfab31444a3ca1e55795ccfa8a9f51 Author: Julien Marrec Date: 2019-04-25 17:44:06 +0200 Windows: Upgrade solution itself windows/vs2019/xz_win.sln | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) commit ac31413916fa9b11bab17f0f0aa63e2869360f6f Author: Julien Marrec Date: 2019-04-25 17:40:24 +0200 Windows: Upgrade solution with VS2019 windows/vs2019/liblzma.vcxproj | 15 ++++++++------- windows/vs2019/liblzma_dll.vcxproj | 15 ++++++++------- 2 files changed, 16 insertions(+), 14 deletions(-) commit be25a0c37ba92a20c390b4d17fe502457fe96b71 Author: Julien Marrec Date: 2019-04-25 17:39:32 +0200 Windows: Duplicate windows/vs2017 before upgrading windows/vs2019/config.h | 148 ++++++++++++++ windows/vs2019/liblzma.vcxproj | 356 ++++++++++++++++++++++++++++++++++ windows/vs2019/liblzma_dll.vcxproj | 385 +++++++++++++++++++++++++++++++++++++ windows/vs2019/xz_win.sln | 48 +++++ 4 files changed, 937 insertions(+) commit d0e58b3a51e8e616f3dc26ec7b7e4aa0fa6991ad Author: Lasse Collin Date: 2019-03-04 22:49:04 +0200 README: Update translation instructions. XZ Utils is now part of the Translation Project . README | 32 +++++++++++++------------------- 1 file changed, 13 insertions(+), 19 deletions(-) commit a750c35a7d45a16c11c1d40fecee8443c32a9996 Author: Lasse Collin Date: 2019-03-04 21:20:39 +0200 xz: Automatically align column headings in xz -lvv. src/xz/list.c | 263 ++++++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 212 insertions(+), 51 deletions(-) commit 6cb42e8aa1dc37bf403a9f5acbd07e86036b7e77 Author: Lasse Collin Date: 2019-03-04 01:07:59 +0200 xz: Automatically align strings ending in a colon in --list output. This should avoid alignment errors in translations with these strings. src/xz/list.c | 114 +++++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 102 insertions(+), 12 deletions(-) commit 1e3f29b62f2c03e50fc9ebea7b83c1497dd35484 Author: Lasse Collin Date: 2019-01-13 17:29:23 +0200 Windows/VS2017: Omit WindowsTargetPlatformVersion from project files. I understood that if a WTPV is specified, it's often wrong because different VS installations have different SDK version installed. Omitting the WTPV tag makes VS2017 default to Windows SDK 8.1 which often is also missing, so in any case people may need to specify the WTPV before building. But some day in the future a missing WTPV tag will start to default to the latest installed SDK which sounds reasonable: https://developercommunity.visualstudio.com/content/problem/140294/windowstargetplatformversion-makes-it-impossible-t.html Thanks to "dom". windows/INSTALL-MSVC.txt | 4 ++++ windows/vs2017/liblzma.vcxproj | 1 - windows/vs2017/liblzma_dll.vcxproj | 1 - 3 files changed, 4 insertions(+), 2 deletions(-) commit 4d86076332aece6314063d3712a5f364172bbb0f Author: Lasse Collin Date: 2018-12-20 20:42:29 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit b55d79461d1f6aeaac03c7dae84481e5eb8bea4c Author: Lasse Collin Date: 2018-12-14 20:34:30 +0200 xz: Fix a crash in progress indicator when in passthru mode. "xz -dcfv not_an_xz_file" crashed (all four options are required to trigger it). It caused xz to call lzma_get_progress(&strm, ...) when no coder was initialized in strm. In this situation strm.internal is NULL which leads to a crash in lzma_get_progress(). The bug was introduced when xz started using lzma_get_progress() to get progress info for multi-threaded compression, so the bug is present in versions 5.1.3alpha and higher. Thanks to Filip Palian for the bug report. src/xz/coder.c | 11 +++++++---- src/xz/message.c | 18 ++++++++++++++++-- src/xz/message.h | 3 ++- 3 files changed, 25 insertions(+), 7 deletions(-) commit 4ae5526de013efd1021686fa80bdd10cf1cb9c56 Author: Lasse Collin Date: 2018-11-22 17:20:31 +0200 xz: Update man page timestamp. src/xz/xz.1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 6a36d0d5f49e0080ff64dd9ef250abd489bea2ad Author: Pavel Raiskup Date: 2018-11-22 15:14:34 +0100 'have have' typos src/xz/signals.c | 2 +- src/xz/xz.1 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit 9eca51ef805ed8002a851df1b4995d71826c8b6e Author: Lasse Collin Date: 2018-11-02 20:40:48 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 106d1a663d4ba42b63231caa289f531548df81c1 Author: Lasse Collin Date: 2018-11-02 20:18:45 +0200 Tests: Add a fuzz test program and a config file for OSS-Fuzz. Thanks to Bhargava Shastry and Github user pdknsk. tests/Makefile.am | 1 + tests/ossfuzz/Makefile | 7 ++++ tests/ossfuzz/config/fuzz.dict | 2 + tests/ossfuzz/config/fuzz.options | 2 + tests/ossfuzz/fuzz.c | 82 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 94 insertions(+) commit a18ae42a79a19b1394b41eb3e238139fd28012ec Author: Lasse Collin Date: 2018-10-26 22:49:10 +0300 liblzma: Don't verify header CRC32s if building for fuzz testing. FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION is #defined when liblzma is being built for fuzz testing. Most fuzzed inputs would normally get rejected because of incorrect CRC32 and the actual header decoding code wouldn't get fuzzed. Disabling CRC32 checks avoids this problem. The fuzzer program must still use LZMA_IGNORE_CHECK flag to disable verification of integrity checks of uncompressed data. src/liblzma/common/block_header_decoder.c | 5 ++++- src/liblzma/common/index_decoder.c | 5 ++++- src/liblzma/common/index_hash.c | 5 ++++- src/liblzma/common/stream_flags_decoder.c | 10 ++++++++-- 4 files changed, 20 insertions(+), 5 deletions(-) commit f76f7516d6a1c832f61810c82e92d151cc80966c Author: Lasse Collin Date: 2018-07-27 18:10:44 +0300 xzless: Rename unused variables to silence static analysers. In this particular case I don't see this affecting readability of the code. Thanks to Pavel Raiskup. src/scripts/xzless.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 3cbcaeb07eb7543735befd6f507fdb5fa4363cff Author: Lasse Collin Date: 2018-07-27 16:02:58 +0300 liblzma: Remove an always-true condition from lzma_index_cat(). This should help static analysis tools to see that newg isn't leaked. Thanks to Pavel Raiskup. src/liblzma/common/index.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit 76762ae6098ec55c326f4b4b4a42e8c1918ee81f Author: Lasse Collin Date: 2018-05-19 21:23:25 +0300 liblzma: Improve lzma_properties_decode() API documentation. src/liblzma/api/lzma/filter.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) commit 2267f5b0d20a5d24e93fcd9f72ea7eeb0d89708c Author: Lasse Collin Date: 2018-04-29 18:58:19 +0300 Bump the version number to 5.3.1alpha. src/liblzma/api/lzma/version.h | 2 +- src/liblzma/liblzma.map | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit cee3021d30704858e4bdd22240e7d28e570d7451 Author: Lasse Collin Date: 2018-04-29 18:48:00 +0300 extra/scanlzma: Fix compiler warnings. extra/scanlzma/scanlzma.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) commit c5c7ceb08a011b97d261798033e2c39613a69eb7 Author: Lasse Collin Date: 2018-04-29 18:44:47 +0300 DOS: Add file_info.c to the list of files to build. dos/Makefile | 1 + 1 file changed, 1 insertion(+) commit 114cab97af766b21e0fc8620479202fb1e7a5e41 Author: Lasse Collin Date: 2018-04-29 18:33:10 +0300 Update NEWS for 5.3.1alpha. NEWS | 11 +++++++++++ 1 file changed, 11 insertions(+) commit b8139e11c512bbf32bf58ab0689f9bb6c52819da Author: Lasse Collin Date: 2018-04-29 18:15:37 +0300 Add NEWS for 5.2.4. NEWS | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) commit 47b59d47cfd904a420fbd45629d168ca1973721d Author: Lasse Collin Date: 2018-02-06 19:36:30 +0200 Update THANKS. THANKS | 2 ++ 1 file changed, 2 insertions(+) commit bc197991690ede24ab143665b5b0f0f9cb35cc46 Author: Ben Boeckel Date: 2018-01-29 13:58:18 -0500 nothrow: use noexcept for C++11 and newer In C++11, the `throw()` specifier is deprecated and `noexcept` is preffered instead. src/liblzma/api/lzma.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) commit fb6d4f83cb6e144734f2a4216bb117bd56dc3cb5 Author: Lasse Collin Date: 2018-02-06 18:02:48 +0200 liblzma: Remove incorrect #ifdef from range_common.h. In most cases it was harmless but it could affect some custom build systems. Thanks to Pippijn van Steenhoven. src/liblzma/rangecoder/range_common.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) commit bc577d35c2d0ed17f554d2d8107b2a2a9abbac76 Author: Lasse Collin Date: 2018-01-10 22:10:39 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 713bbc1a80f26d34c96ed3dbb9887362204de3a1 Author: Lasse Collin Date: 2018-01-10 21:54:27 +0200 tuklib_integer: New Intel C compiler needs immintrin.h. Thanks to Melanie Blower (Intel) for the patch. src/common/tuklib_integer.h | 11 +++++++++++ 1 file changed, 11 insertions(+) commit a0ee1afbd99da138b559cb27fa2022e7f1ab44f3 Author: Lasse Collin Date: 2017-09-24 20:04:24 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit a1e2c568de29c0b57d873eab40a2879b749da429 Author: Lasse Collin Date: 2017-09-16 20:36:20 +0300 Windows: Fix paths in VS project files. Some paths use slashes instead of backslashes as directory separators... now it should work (I tested VS2013 version). windows/vs2013/liblzma.vcxproj | 12 ++++++------ windows/vs2013/liblzma_dll.vcxproj | 24 ++++++++++++------------ windows/vs2017/liblzma.vcxproj | 12 ++++++------ windows/vs2017/liblzma_dll.vcxproj | 24 ++++++++++++------------ 4 files changed, 36 insertions(+), 36 deletions(-) commit cea5cf8d26c9d1dc30a808614d79c0b25640e15e Author: Lasse Collin Date: 2017-09-16 12:56:20 +0300 Windows: Update VS2017 project files to include file info decoder. windows/vs2017/liblzma.vcxproj | 2 ++ windows/vs2017/liblzma_dll.vcxproj | 2 ++ 2 files changed, 4 insertions(+) commit 95d563db3ee497b223e522b699c4d4c29943eef0 Author: Lasse Collin Date: 2017-09-16 12:54:23 +0300 Windows: Add project files for VS2017. These files match the v5.2 branch (no file info decoder). windows/vs2017/config.h | 148 ++++++++++++++ windows/vs2017/liblzma.vcxproj | 355 ++++++++++++++++++++++++++++++++++ windows/vs2017/liblzma_dll.vcxproj | 384 +++++++++++++++++++++++++++++++++++++ windows/vs2017/xz_win.sln | 48 +++++ 4 files changed, 935 insertions(+) commit ab72416d62ea8f50ad31d5b8545fcb6a2bf96b73 Author: Lasse Collin Date: 2017-09-16 12:45:50 +0300 Windows: Update VS2013 project files to include file info decoder. windows/vs2013/liblzma.vcxproj | 2 ++ windows/vs2013/liblzma_dll.vcxproj | 2 ++ 2 files changed, 4 insertions(+) commit 82388980187b0e3794d187762054200bbdcc9a53 Author: Lasse Collin Date: 2017-09-16 12:39:43 +0300 Windows: Move VS2013 files into windows/vs2013 directory. windows/{ => vs2013}/config.h | 0 windows/{ => vs2013}/liblzma.vcxproj | 278 +++++++++++++++--------------- windows/{ => vs2013}/liblzma_dll.vcxproj | 280 +++++++++++++++---------------- windows/{ => vs2013}/xz_win.sln | 0 4 files changed, 279 insertions(+), 279 deletions(-) commit 94e3f986aa4e14b4ff01ac24857f499630d6d180 Author: Lasse Collin Date: 2017-08-14 20:08:33 +0300 Fix or hide warnings from GCC 7's -Wimplicit-fallthrough. src/liblzma/lzma/lzma_decoder.c | 6 ++++++ src/xz/list.c | 2 ++ 2 files changed, 8 insertions(+) commit 0b0e1e6803456aac641a59332200f8e95e2b7ea8 Author: Alexey Tourbin Date: 2017-05-16 23:56:35 +0300 Docs: Fix a typo in a comment in doc/examples/02_decompress.c. doc/examples/02_decompress.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit a015cd1f90116e655be4eaf4aad42c4c911c2807 Author: Lasse Collin Date: 2017-05-23 18:34:43 +0300 xz: Fix "xz --list --robot missing_or_bad_file.xz". It ended up printing an uninitialized char-array when trying to print the check names (column 7) on the "totals" line. This also changes the column 12 (minimum xz version) to 50000002 (xz 5.0.0) instead of 0 when there are no valid input files. Thanks to kidmin for the bug report. src/xz/list.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) commit c2e29f06a7d1e3ba242ac2fafc69f5d6e92f62cd Author: Lasse Collin Date: 2017-04-24 20:20:11 +0300 Docs: Add doc/examples/11_file_info.c. doc/examples/11_file_info.c | 206 ++++++++++++++++++++++++++++++++++++++++++++ doc/examples/Makefile | 3 +- 2 files changed, 208 insertions(+), 1 deletion(-) commit 1520f6ec808896375ac7bf778c449e0f7dea5f46 Author: Lasse Collin Date: 2017-04-24 19:48:47 +0300 Build: Omit pre-5.0.0 entries from the generated ChangeLog. It makes ChangeLog significantly smaller. Makefile.am | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 8269782283806c90a8509c2ac2a308344f70e171 Author: Lasse Collin Date: 2017-04-24 19:48:23 +0300 xz: Use lzma_file_info_decoder() for --list. src/xz/list.c | 254 ++++++++++------------------------------------------------ 1 file changed, 44 insertions(+), 210 deletions(-) commit e353d0b1cc0d3997ae5048faa8e6786414953e06 Author: Lasse Collin Date: 2017-04-24 19:35:50 +0300 liblzma: Add lzma_file_info_decoder(). src/liblzma/api/lzma/index.h | 66 ++++ src/liblzma/common/Makefile.inc | 1 + src/liblzma/common/file_info.c | 855 ++++++++++++++++++++++++++++++++++++++++ src/liblzma/liblzma.map | 7 +- 4 files changed, 928 insertions(+), 1 deletion(-) commit 144ef9e19e9496c995b21505dd1e111c442968d1 Author: Lasse Collin Date: 2017-04-24 19:30:22 +0300 Update the Git repository URL to HTTPS in ChangeLog. ChangeLog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit 8c9842c265993d7dd4039f732d3546267fb5ecc4 Author: Lasse Collin Date: 2017-04-21 15:05:16 +0300 liblzma: Rename LZMA_SEEK to LZMA_SEEK_NEEDED and seek_in to seek_pos. src/liblzma/api/lzma/base.h | 18 +++++++++--------- src/liblzma/common/common.c | 2 +- src/xz/message.c | 2 +- 3 files changed, 11 insertions(+), 11 deletions(-) commit 662b27c417cab248cb365dd7682121bdec4d5ae7 Author: Lasse Collin Date: 2017-04-19 22:17:35 +0300 Update the home page URLs to HTTPS. COPYING | 2 +- README | 2 +- configure.ac | 2 +- doc/faq.txt | 4 ++-- dos/config.h | 2 +- src/common/common_w32res.rc | 2 +- src/xz/xz.1 | 6 +++--- src/xzdec/xzdec.1 | 4 ++-- windows/README-Windows.txt | 2 +- windows/config.h | 2 +- 10 files changed, 14 insertions(+), 14 deletions(-) commit c28f0b3d00af87b92dda229831548d8eb0067d1d Author: Lasse Collin Date: 2017-04-05 18:47:22 +0300 xz: Add io_seek_src(). src/xz/file_io.c | 20 +++++++++++++++++--- src/xz/file_io.h | 13 +++++++++++++ 2 files changed, 30 insertions(+), 3 deletions(-) commit bba477257d7319c8764890f3669175b866d24944 Author: Lasse Collin Date: 2017-03-30 22:01:54 +0300 xz: Use POSIX_FADV_RANDOM for in "xz --list" mode. xz --list is random access so POSIX_FADV_SEQUENTIAL was clearly wrong. src/xz/file_io.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) commit 310d19816d1652b0c8bb1b82574d46345d924752 Author: Lasse Collin Date: 2017-03-30 20:03:05 +0300 liblzma: Make lzma_index_decoder_init() visible to other liblzma funcs. This is to allow other functions to use it without going via the public API (lzma_index_decoder()). src/liblzma/common/Makefile.inc | 1 + src/liblzma/common/index_decoder.c | 10 +++++----- src/liblzma/common/index_decoder.h | 24 ++++++++++++++++++++++++ 3 files changed, 30 insertions(+), 5 deletions(-) commit a27920002dbc469f778a134fc665b7c3ea73701b Author: Lasse Collin Date: 2017-03-30 20:00:09 +0300 liblzma: Add generic support for input seeking (LZMA_SEEK). Also mention LZMA_SEEK in xz/message.c to silence a warning. src/liblzma/api/lzma/base.h | 31 ++++++++++++++++++++++++++++++- src/liblzma/common/common.c | 12 +++++++++++- src/xz/message.c | 1 + 3 files changed, 42 insertions(+), 2 deletions(-) commit a0b1dda409bc3e6e2957a2651663fc411d2caf2d Author: Lasse Collin Date: 2017-03-30 19:47:45 +0300 liblzma: Fix lzma_memlimit_set(strm, 0). The 0 got treated specially in a buggy way and as a result the function did nothing. The API doc said that 0 was supposed to return LZMA_PROG_ERROR but it didn't. Now 0 is treated as if 1 had been specified. This is done because 0 is already used to indicate an error from lzma_memlimit_get() and lzma_memusage(). In addition, lzma_memlimit_set() no longer checks that the new limit is at least LZMA_MEMUSAGE_BASE. It's counter-productive for the Index decoder and was actually needed only by the auto decoder. Auto decoder has now been modified to check for LZMA_MEMUSAGE_BASE. src/liblzma/api/lzma/base.h | 7 ++++++- src/liblzma/common/auto_decoder.c | 3 +++ src/liblzma/common/common.c | 6 ++++-- 3 files changed, 13 insertions(+), 3 deletions(-) commit 84462afaada61379f5878e46f8f00e25a1cdcf29 Author: Lasse Collin Date: 2017-03-30 19:16:55 +0300 liblzma: Similar memlimit fix for stream_, alone_, and auto_decoder. src/liblzma/api/lzma/container.h | 21 +++++++++++++++++---- src/liblzma/common/alone_decoder.c | 5 +---- src/liblzma/common/auto_decoder.c | 5 +---- src/liblzma/common/stream_decoder.c | 5 +---- 4 files changed, 20 insertions(+), 16 deletions(-) commit cbc74017939690d13441b8926bb743fb03211b83 Author: Lasse Collin Date: 2017-03-30 18:58:18 +0300 liblzma: Fix handling of memlimit == 0 in lzma_index_decoder(). It returned LZMA_PROG_ERROR, which was done to avoid zero as the limit (because it's a special value elsewhere), but using LZMA_PROG_ERROR is simply inconvenient and can cause bugs. The fix/workaround is to treat 0 as if it were 1 byte. It's effectively the same thing. The only weird consequence is that then lzma_memlimit_get() will return 1 even when 0 was specified as the limit. This fixes a very rare corner case in xz --list where a specific memory usage limit and a multi-stream file could print the error message "Internal error (bug)" instead of saying that the memory usage limit is too low. src/liblzma/api/lzma/index.h | 18 +++++++++++------- src/liblzma/common/index_decoder.c | 4 ++-- 2 files changed, 13 insertions(+), 9 deletions(-) commit 78ae13bced912b1b92ae927992c99cbcc463cae7 Author: Lasse Collin Date: 2016-12-30 13:25:10 +0200 Update NEWS for 5.2.3. NEWS | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) commit 0297863fdb453aed1a25eb025f3ba7bacbbb1357 Author: Lasse Collin Date: 2016-12-26 20:55:52 +0200 Document --enable-sandbox configure option in INSTALL. INSTALL | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) commit d4a0462abe5478193521c14625e1c81fead87f9f Author: Lasse Collin Date: 2016-11-21 20:24:50 +0200 liblzma: Avoid multiple definitions of lzma_coder structures. Only one definition was visible in a translation unit. It avoided a few casts and temp variables but seems that this hack doesn't work with link-time optimizations in compilers as it's not C99/C11 compliant. Fixes: http://www.mail-archive.com/xz-devel@tukaani.org/msg00279.html src/liblzma/common/alone_decoder.c | 44 +++++---- src/liblzma/common/alone_encoder.c | 34 ++++--- src/liblzma/common/auto_decoder.c | 35 ++++--- src/liblzma/common/block_decoder.c | 41 ++++---- src/liblzma/common/block_encoder.c | 40 ++++---- src/liblzma/common/common.h | 18 ++-- src/liblzma/common/index_decoder.c | 33 ++++--- src/liblzma/common/index_encoder.c | 16 ++-- src/liblzma/common/stream_decoder.c | 50 +++++----- src/liblzma/common/stream_encoder.c | 56 ++++++----- src/liblzma/common/stream_encoder_mt.c | 124 ++++++++++++++----------- src/liblzma/delta/delta_common.c | 25 ++--- src/liblzma/delta/delta_decoder.c | 6 +- src/liblzma/delta/delta_encoder.c | 12 ++- src/liblzma/delta/delta_private.h | 4 +- src/liblzma/lz/lz_decoder.c | 60 ++++++------ src/liblzma/lz/lz_decoder.h | 13 ++- src/liblzma/lz/lz_encoder.c | 57 +++++++----- src/liblzma/lz/lz_encoder.h | 9 +- src/liblzma/lzma/lzma2_decoder.c | 32 ++++--- src/liblzma/lzma/lzma2_encoder.c | 51 +++++----- src/liblzma/lzma/lzma_decoder.c | 27 +++--- src/liblzma/lzma/lzma_encoder.c | 29 +++--- src/liblzma/lzma/lzma_encoder.h | 9 +- src/liblzma/lzma/lzma_encoder_optimum_fast.c | 3 +- src/liblzma/lzma/lzma_encoder_optimum_normal.c | 23 ++--- src/liblzma/lzma/lzma_encoder_private.h | 6 +- src/liblzma/simple/arm.c | 2 +- src/liblzma/simple/armthumb.c | 2 +- src/liblzma/simple/ia64.c | 2 +- src/liblzma/simple/powerpc.c | 2 +- src/liblzma/simple/simple_coder.c | 61 ++++++------ src/liblzma/simple/simple_private.h | 12 +-- src/liblzma/simple/sparc.c | 2 +- src/liblzma/simple/x86.c | 15 +-- 35 files changed, 532 insertions(+), 423 deletions(-) commit a01794c52add98263b49119842c3e7141d1b9ced Author: Lasse Collin Date: 2016-10-24 18:53:25 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit df8f446e3ad47e5148b8c8d8b6e519d3ce29cb9d Author: Lasse Collin Date: 2016-10-24 18:51:36 +0300 tuklib_cpucores: Add support for sched_getaffinity(). It's available in glibc (GNU/Linux, GNU/kFreeBSD). It's better than sysconf(_SC_NPROCESSORS_ONLN) because sched_getaffinity() gives the number of cores available to the process instead of the total number of cores online. As a side effect, this commit fixes a bug on GNU/kFreeBSD where configure would detect the FreeBSD-specific cpuset_getaffinity() but it wouldn't actually work because on GNU/kFreeBSD it requires using -lfreebsd-glue when linking. Now the glibc-specific function will be used instead. Thanks to Sebastian Andrzej Siewior for the original patch and testing. m4/tuklib_cpucores.m4 | 30 +++++++++++++++++++++++++++++- src/common/tuklib_cpucores.c | 9 +++++++++ 2 files changed, 38 insertions(+), 1 deletion(-) commit 446e4318fa79788e09299d5953b5dd428953d14b Author: Lasse Collin Date: 2016-06-30 20:27:36 +0300 xz: Fix copying of timestamps on Windows. xz used to call utime() on Windows, but its result gets lost on close(). Using _futime() seems to work. Thanks to Martok for reporting the bug: http://www.mail-archive.com/xz-devel@tukaani.org/msg00261.html configure.ac | 2 +- src/xz/file_io.c | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) commit 1b0ac0c53c761263e91e34195cb21dfdcfeac0bd Author: Lasse Collin Date: 2016-06-16 22:46:02 +0300 xz: Silence warnings from -Wlogical-op. Thanks to Evan Nemerson. src/xz/file_io.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) commit c83b7a03342c3325ff10400b22ee21edfcd1e026 Author: Lasse Collin Date: 2016-04-10 20:55:49 +0300 Build: Fix = to += for xz_SOURCES in src/xz/Makefile.am. Thanks to Christian Kujau. src/xz/Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit ade31a2bfb95c94d05fbfc0ecbba5d6377f2506e Author: Lasse Collin Date: 2016-04-10 20:54:17 +0300 Build: Bump GNU Gettext version requirement to 0.19. It silences a few warnings and most people probably have 0.19 even on stable distributions. Thanks to Christian Kujau. configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit ac398c3bafa6e4c80e20571373a96947db863b3d Author: Lasse Collin Date: 2016-03-13 20:21:49 +0200 liblzma: Disable external SHA-256 by default. This is the sane thing to do. The conflict with OpenSSL on some OSes and especially that the OS-provided versions can be significantly slower makes it clear that it was a mistake to have the external SHA-256 support enabled by default. Those who want it can now pass --enable-external-sha256 to configure. INSTALL was updated with notes about OSes where this can be a bad idea. The SHA-256 detection code in configure.ac had some bugs that could lead to a build failure in some situations. These were fixed, although it doesn't matter that much now that the external SHA-256 is disabled by default. MINIX >= 3.2.0 uses NetBSD's libc and thus has SHA256_Init in libc instead of libutil. Support for the libutil version was removed. INSTALL | 36 ++++++++++++++++++++++ configure.ac | 76 +++++++++++++++++++++++------------------------ src/liblzma/check/check.h | 16 ++++------ 3 files changed, 79 insertions(+), 49 deletions(-) commit 6fd5ecb589a9fdd7a576ea48c4850d496bab9ce5 Author: Lasse Collin Date: 2016-03-10 20:27:05 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 473ef0dc69a30e64d5fa0d34aca02f7309faa3e9 Author: Lasse Collin Date: 2016-03-10 20:26:49 +0200 Build: Avoid SHA256_Init on FreeBSD and MINIX 3. On FreeBSD 10 and older, SHA256_Init from libmd conflicts with libcrypto from OpenSSL. The OpenSSL version has different sizeof(SHA256_CTX) and it can cause weird problems if wrong SHA256_Init gets used. Looking at the source, MINIX 3 seems to have a similar issue but I'm not sure. To be safe, I disabled SHA256_Init on MINIX 3 too. NetBSD has SHA256_Init in libc and they had a similar problem, but they already fixed it in 2009. Thanks to Jim Wilcoxson for the bug report that helped in finding the problem. configure.ac | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) commit faf302137e54d605b44ecf0373cb51a6403a2de1 Author: Lasse Collin Date: 2015-11-08 20:16:10 +0200 tuklib_physmem: Hopefully silence a warning on Windows. src/common/tuklib_physmem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit e52e9151cf8613022d1de4712ff39dbcb666e991 Author: Lasse Collin Date: 2015-11-04 23:17:43 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 14115f84a38161d55eaa2d070f08739bde37e966 Author: Lasse Collin Date: 2015-11-04 23:14:00 +0200 liblzma: Make Valgrind happier with optimized (gcc -O2) liblzma. When optimizing, GCC can reorder code so that an uninitialized value gets used in a comparison, which makes Valgrind unhappy. It doesn't happen when compiled with -O0, which I tend to use when running Valgrind. Thanks to Rich Prohaska. I remember this being mentioned long ago by someone else but nothing was done back then. src/liblzma/lz/lz_encoder.c | 4 ++++ 1 file changed, 4 insertions(+) commit f4c95ba94beb71a608eb6eadbf82b44f53a0260e Author: Lasse Collin Date: 2015-11-03 20:55:45 +0200 liblzma: Rename lzma_presets.c back to lzma_encoder_presets.c. It would be too annoying to update other build systems just because of this. src/liblzma/lzma/Makefile.inc | 2 +- src/liblzma/lzma/{lzma_presets.c => lzma_encoder_presets.c} | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit c7bc20a6f3e71920871d48db31a79ab58b5a0a4b Author: Lasse Collin Date: 2015-11-03 20:47:07 +0200 Build: Disable xzdec, lzmadec, and lzmainfo when they cannot be built. They all need decoder support and if that isn't available, there's no point trying to build them. configure.ac | 3 +++ 1 file changed, 3 insertions(+) commit 5cbca1205deeb6fb7afe7a864fa68a57466d928a Author: Lasse Collin Date: 2015-11-03 20:35:19 +0200 Build: Simplify $enable_{encoders,decoders} usage a bit. configure.ac | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) commit af13781886c8e7a0aabebb5141ea282dc364f5c6 Author: Lasse Collin Date: 2015-11-03 20:31:31 +0200 Windows/MSVC: Update config.h. windows/config.h | 6 ++++++ 1 file changed, 6 insertions(+) commit 9fa5949330f162c5a2f6653f83025327837e8f39 Author: Lasse Collin Date: 2015-11-03 20:29:58 +0200 DOS: Update config.h. dos/config.h | 6 ++++++ 1 file changed, 6 insertions(+) commit cb3111e3ed84152912b5138d690c8d9f00c6ef02 Author: Lasse Collin Date: 2015-11-03 20:29:33 +0200 xz: Make xz buildable even when encoders or decoders are disabled. The patch is quite long but it's mostly about adding new #ifdefs to omit code when encoders or decoders have been disabled. This adds two new #defines to config.h: HAVE_ENCODERS and HAVE_DECODERS. configure.ac | 4 ++++ src/xz/Makefile.am | 8 ++++++-- src/xz/args.c | 16 ++++++++++++++++ src/xz/coder.c | 33 +++++++++++++++++++++++++-------- src/xz/main.c | 9 +++++++-- src/xz/private.h | 5 ++++- 6 files changed, 62 insertions(+), 13 deletions(-) commit 4cc584985c0b7a13901da1b7a64ef9f7cc36e8ab Author: Lasse Collin Date: 2015-11-03 18:06:40 +0200 Build: Build LZMA1/2 presets also when only decoder is wanted. People shouldn't rely on the presets when decoding raw streams, but xz uses the presets as the starting point for raw decoder options anyway. lzma_encocder_presets.c was renamed to lzma_presets.c to make it clear it's not used solely by the encoder code. src/liblzma/lzma/Makefile.inc | 6 +++++- src/liblzma/lzma/{lzma_encoder_presets.c => lzma_presets.c} | 3 ++- 2 files changed, 7 insertions(+), 2 deletions(-) commit 23ed1d41489f632155bbc9660f323d57e09da180 Author: Lasse Collin Date: 2015-11-03 17:54:48 +0200 Build: Fix configure to handle LZMA1 dependency with LZMA2. Now it gives an error if LZMA1 encoder/decoder is missing when LZMA2 encoder/decoder was requested. Even better would be LZMA2 implicitly enabling LZMA1 but it would need more code. configure.ac | 5 ----- 1 file changed, 5 deletions(-) commit b0bc3e03852af13419ea2960881824258d451117 Author: Lasse Collin Date: 2015-11-03 17:41:54 +0200 Build: Don't omit lzma_cputhreads() unless using --disable-threads. Previously it was omitted if encoders were disabled with --disable-encoders. It didn't make sense and it also broke the build. src/liblzma/common/Makefile.inc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) commit c6bf438ab39e0fb4a47d3c81725c227919502726 Author: Lasse Collin Date: 2015-11-02 18:16:51 +0200 liblzma: Fix a build failure related to external SHA-256 support. If an appropriate header and structure were found by configure, but a library with a usable SHA-256 functions wasn't, the build failed. src/liblzma/check/check.h | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-) commit e18adc56f2262aa9394d46681e9e4b9981ed5e97 Author: Lasse Collin Date: 2015-11-02 15:19:10 +0200 xz: Always close the file before trying to delete it. unlink() can return EBUSY in errno for open files on some operating systems and file systems. src/xz/file_io.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) commit 282e768a1484e88c8b7ec35655ee4959954ec87a Author: Lasse Collin Date: 2015-10-12 21:08:42 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 372e402713a1d4337ffce5f56d5c5c9ed99a66d0 Author: Lasse Collin Date: 2015-10-12 21:07:41 +0300 Tests: Add tests for the two bugs fixed in index.c. tests/test_index.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) commit 21515d79d778b8730a434f151b07202d52a04611 Author: Lasse Collin Date: 2015-10-12 20:45:15 +0300 liblzma: Fix lzma_index_dup() for empty Streams. Stream Flags and Stream Padding weren't copied from empty Streams. src/liblzma/common/index.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) commit 09f395b6b360c0b13e8559eece1d179b908ebd3a Author: Lasse Collin Date: 2015-10-12 20:31:44 +0300 liblzma: Add a note to index.c for those using static analyzers. src/liblzma/common/index.c | 3 +++ 1 file changed, 3 insertions(+) commit 3bf857edfef51374f6f3fffae3d817f57d3264a0 Author: Lasse Collin Date: 2015-10-12 20:29:09 +0300 liblzma: Fix a memory leak in error path of lzma_index_dup(). lzma_index_dup() calls index_dup_stream() which, in case of an error, calls index_stream_end() to free memory allocated by index_stream_init(). However, it illogically didn't actually free the memory. To make it logical, the tree handling code was modified a bit in addition to changing index_stream_end(). Thanks to Evan Nemerson for the bug report. src/liblzma/common/index.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) commit 7f05803979b4b79642d5be4218a79da7a0b12c47 Author: Lasse Collin Date: 2015-09-29 13:57:28 +0300 Update NEWS for 5.2.2. NEWS | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) commit 397fcc0946315b55c3c6d80e37e82a2a78bc15c1 Author: Hauke Henningsen Date: 2015-08-17 04:59:54 +0200 Update German translation, mostly wrt orthography Provide an update of the German translation. * A lot of compound words were previously written with spaces, while German orthography is relatively clear in that the components should not be separated. * When referring to the actual process of (de)compression rather than the concept, replace “(De-)Kompression” with “(De-)Komprimierung”. Previously, both forms were used in this context and are now used in a manner consistent with “Komprimierung” being more likely to refer to a process. * Consistently translate “standard input”/“output” * Use “Zeichen” instead of false friend “Charakter” for “character” * Insert commas around relative clauses (as required in German) * Some other minor corrections * Capitalize “ß” as “ẞ” * Consistently start option descriptions in --help with capital letters Acked-By: Andre Noll * Update after msgmerge po/de.po | 383 ++++++++++++++++++++++++++++++++------------------------------- 1 file changed, 196 insertions(+), 187 deletions(-) commit cbc9e39bae715accb44168930a71888480aad569 Author: Lasse Collin Date: 2015-08-11 13:23:04 +0300 Build: Minor Cygwin cleanup. Some tests used "cygwin*" and some used "cygwin". I changed them all to use "cygwin". Shouldn't affect anything in practice. configure.ac | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) commit bcacd8ce7a031566858e5e03c1009064c3f1c89e Author: Lasse Collin Date: 2015-08-11 13:21:52 +0300 Build: Support building of MSYS2 binaries. configure.ac | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) commit 0275a5398c01d57b724dec7fea52dec3bd6edc6c Author: Lasse Collin Date: 2015-08-09 21:06:26 +0300 Windows: Define DLL_EXPORT when building liblzma.dll with MSVC. src/liblzma/common/common.h uses it to set __declspec(dllexport) for the API symbols. Thanks to Adam Walling. windows/liblzma_dll.vcxproj | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) commit a74525cf9b945fb0b370e64cf406104beb31729b Author: Lasse Collin Date: 2015-08-09 21:02:20 +0300 Windows: Omit unneeded header files from MSVC project files. windows/liblzma.vcxproj | 5 ----- windows/liblzma_dll.vcxproj | 5 ----- 2 files changed, 10 deletions(-) commit fbbb295a91caf39faf8838c8c39526e4cb4dc121 Author: Lasse Collin Date: 2015-07-12 20:48:19 +0300 liblzma: A MSVC-specific hack isn't needed with MSVC 2013 and newer. src/liblzma/api/lzma.h | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) commit 713dbe5c230fe00865a54f5c32358ea30f9a1156 Author: Lasse Collin Date: 2015-06-19 20:38:55 +0300 Update THANKS. THANKS | 2 ++ 1 file changed, 2 insertions(+) commit 3a5d755d055d51f99c523b4c2952727e1e69cfa1 Author: Lasse Collin Date: 2015-06-19 20:21:30 +0300 Windows: Update the docs. INSTALL | 29 ++++++++----- windows/INSTALL-MSVC.txt | 47 ++++++++++++++++++++++ windows/{INSTALL-Windows.txt => INSTALL-MinGW.txt} | 2 +- 3 files changed, 67 insertions(+), 11 deletions(-) commit b0798c6aa6184efcefd0bdcca20f96121a13feda Author: Lasse Collin Date: 2015-06-19 17:25:31 +0300 Windows: Add MSVC project files for building liblzma. Thanks to Adam Walling for creating these files. windows/liblzma.vcxproj | 359 ++++++++++++++++++++++++++++++++++++++++ windows/liblzma_dll.vcxproj | 388 ++++++++++++++++++++++++++++++++++++++++++++ windows/xz_win.sln | 48 ++++++ 3 files changed, 795 insertions(+) commit 9b02a4ffdac1b9f066658ec4c95c0834f4cd2fb7 Author: Andre Noll Date: 2015-05-28 15:50:00 +0200 Fix typo in German translation. As pointed out by Robert Pollak, there's a typo in the German translation of the compression preset option (-0 ... -9) help text. "The compressor" translates to "der Komprimierer", and the genitive form is "des Komprimierers". The old word makes no sense at all. po/de.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit c7f4041f6b8f4729f88d3bc888b2a4080ae51f72 Author: Lasse Collin Date: 2015-05-13 20:57:55 +0300 Tests: Fix a memory leak in test_bcj_exact_size. Thanks to Cristian Rodríguez. tests/test_bcj_exact_size.c | 1 + 1 file changed, 1 insertion(+) commit 17b29d4f0ae0f780fbd69e15a398dc478d8492f8 Author: Lasse Collin Date: 2015-05-12 18:08:24 +0300 Fix NEWS about threading in 5.2.0. Thanks to Andy Hochhaus. NEWS | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) commit 49c26920d6e2d85e5c6123e34958aed2e77485ad Author: Lasse Collin Date: 2015-05-11 21:26:16 +0300 xz: Document that threaded decompression hasn't been implemented yet. src/xz/xz.1 | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) commit 5b2458cb244ed237efe4de1ebcf06e1b3a1f4256 Author: Lasse Collin Date: 2015-04-20 20:20:29 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 6bd0349c58451b13442e8f463e35de83548bf985 Author: Lasse Collin Date: 2015-04-20 19:59:18 +0300 Revert "xz: Use pipe2() if available." This reverts commit 7a11c4a8e5e15f13d5fa59233b3172e65428efdd. It is a problem when libc has pipe2() but the kernel is too old to have pipe2() and thus pipe2() fails. In xz it's pointless to have a fallback for non-functioning pipe2(); it's better to avoid pipe2() completely. Thanks to Michael Fox for the bug report. configure.ac | 4 ++-- src/xz/file_io.c | 9 +-------- 2 files changed, 3 insertions(+), 10 deletions(-) commit fc0df0f8db87dff45543708a711e17d29c37f632 Author: Lasse Collin Date: 2015-04-01 14:45:25 +0300 xz: Fix the Capsicum rights on user_abort_pipe. src/xz/file_io.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) commit 57393615b31b3788dd77280452d845bcc12d33af Author: Lasse Collin Date: 2015-03-31 22:20:11 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 1238381143a9a7ce84839c2582ccd56ff750a440 Author: Lasse Collin Date: 2015-03-31 22:19:34 +0300 xz: Add support for sandboxing with Capsicum. The sandboxing is used conditionally as described in main.c. This isn't optimal but it was much easier to implement than a full sandboxing solution and it still covers the most common use cases where xz is writing to standard output. This should have practically no effect on performance even with small files as fork() isn't needed. C and locale libraries can open files as needed. This has been fine in the past, but it's a problem with things like Capsicum. io_sandbox_enter() tries to ensure that various locale-related files have been loaded before cap_enter() is called, but it's possible that there are other similar problems which haven't been seen yet. Currently Capsicum is available on FreeBSD 10 and later and there is a port to Linux too. Thanks to Loganaden Velvindron for help. configure.ac | 41 +++++++++++++++++++++++++++ src/xz/Makefile.am | 2 +- src/xz/file_io.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/xz/file_io.h | 6 ++++ src/xz/main.c | 18 ++++++++++++ src/xz/private.h | 4 +++ 6 files changed, 151 insertions(+), 1 deletion(-) commit 29a087fb5a0c879f0b1bc4c6b989f7b87bacdf9e Author: Lasse Collin Date: 2015-03-31 21:12:30 +0300 Fix bugs and otherwise improve ax_check_capsicum.m4. AU_ALIAS was removed because the new version is incompatible with the old version. It no longer checks for separately. It's enough to test for it as part of AC_CHECK_DECL. The defines HAVE_CAPSICUM_SYS_CAPSICUM_H and HAVE_CAPSICUM_SYS_CAPABILITY_H were removed as unneeded. HAVE_SYS_CAPSICUM_H from AC_CHECK_HEADERS is enough. It no longer does a useless search for the Capsicum library if the header wasn't found. Fixed a bug in ACTION-IF-FOUND (the first argument). Specifying the argument omitted the default action but the given action wasn't used instead. AC_DEFINE([HAVE_CAPSICUM]) is now always called when Capsicum support is found. Previously it was part of the default ACTION-IF-FOUND which a custom action would override. Now the default action only prepends ${CAPSICUM_LIB} to LIBS. The documentation was updated. Since there as no serial number, "#serial 2" was added. m4/ax_check_capsicum.m4 | 103 ++++++++++++++++++++++++------------------------ 1 file changed, 51 insertions(+), 52 deletions(-) commit 6e845c6a3eddf2fde9db5a29950421dff60a43ac Author: Lasse Collin Date: 2015-03-31 19:20:24 +0300 Add m4/ax_check_capsicum.m4 for detecting Capsicum support. The file was loaded from this web page: https://github.com/google/capsicum-test/blob/dev/autoconf/m4/ax_check_capsicum.m4 Thanks to Loganaden Velvindron for pointing it out for me. m4/ax_check_capsicum.m4 | 86 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) commit 3717885f9ef2c06f1bcbad9f4c2ed2d5695f844e Author: Lasse Collin Date: 2015-03-30 22:44:02 +0300 Bump version to 5.3.0alpha and soname to 5.3.99. The idea of 99 is that it looks a bit weird in this context. For new features there's no API/ABI stability in devel versions. src/liblzma/Makefile.am | 2 +- src/liblzma/api/lzma/version.h | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) commit eccd8155e107c5ada03d13e7730675cdf1a44ddc Author: Lasse Collin Date: 2015-03-29 22:14:47 +0300 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 25263fd9e7a8a913395cb93d7c104cd48c2b4a00 Author: Lasse Collin Date: 2015-03-29 22:13:48 +0300 Fix the detection of installed RAM on QNX. The earlier version compiled but didn't actually work since sysconf(_SC_PHYS_PAGES) always fails (or so I was told). Thanks to Ole André Vadla Ravnås for the patch and testing. m4/tuklib_physmem.m4 | 6 +++--- src/common/tuklib_physmem.c | 14 +++++++++++++- 2 files changed, 16 insertions(+), 4 deletions(-) commit 4c544d2410903d38402221cb783ed85585b6a007 Author: Lasse Collin Date: 2015-03-27 22:39:07 +0200 Fix CPU core count detection on QNX. It tried to use sysctl() on QNX but - it broke the build because sysctl() needs -lsocket on QNX; - sysctl() doesn't work for detecting the core count on QNX even if it compiled. sysconf() works. An alternative would have been to use QNX-specific SYSPAGE_ENTRY(num_cpu) from . Thanks to Ole André Vadla Ravnås. m4/tuklib_cpucores.m4 | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) commit e0ea6737b03e83ccaff4514d00e31bb926f8f0f3 Author: Lasse Collin Date: 2015-03-07 22:05:57 +0200 xz: size_t/uint32_t cleanup in options.c. src/xz/options.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) commit 8bcca29a65335fd679c13814b70b35b68fa5daed Author: Lasse Collin Date: 2015-03-07 22:04:23 +0200 xz: Fix a comment and silence a warning in message.c. src/xz/message.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) commit f243f5f44c6b19a7c289a0ec73a03ee08364cb5b Author: Lasse Collin Date: 2015-03-07 22:01:00 +0200 liblzma: Silence more uint32_t vs. size_t warnings. src/liblzma/lz/lz_encoder.c | 2 +- src/liblzma/lzma/lzma_encoder.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit 7f0a4c50f4a374c40acf4b86848f301ad1e82d34 Author: Lasse Collin Date: 2015-03-07 19:54:00 +0200 xz: Make arg_count an unsigned int to silence a warning. Actually the value of arg_count cannot exceed INT_MAX but it's nicer as an unsigned int. src/xz/args.h | 2 +- src/xz/main.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit f6ec46801588b1be29c07c9db98558b521304002 Author: Lasse Collin Date: 2015-03-07 19:33:17 +0200 liblzma: Fix a warning in index.c. src/liblzma/common/index.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) commit a24518971cc621315af142dd3bb7614fab04ad27 Author: Lasse Collin Date: 2015-02-26 20:46:14 +0200 Build: Fix a CR+LF problem when running autoreconf -fi on OS/2. build-aux/version.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit dec11497a71518423b5ff0e759100cf8aadf6c7b Author: Lasse Collin Date: 2015-02-26 16:53:44 +0200 Bump version and soname for 5.2.1. src/liblzma/Makefile.am | 2 +- src/liblzma/api/lzma/version.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) commit 29e39c79975ab89ee5dd671e97064534a9f3a649 Author: Lasse Collin Date: 2015-02-26 13:01:09 +0200 Update NEWS for 5.2.1. NEWS | 14 ++++++++++++++ 1 file changed, 14 insertions(+) commit 7a11c4a8e5e15f13d5fa59233b3172e65428efdd Author: Lasse Collin Date: 2015-02-22 19:38:48 +0200 xz: Use pipe2() if available. configure.ac | 4 ++-- src/xz/file_io.c | 9 ++++++++- 2 files changed, 10 insertions(+), 3 deletions(-) commit 117d962685c72682c63edc9bb765367189800202 Author: Lasse Collin Date: 2015-02-21 23:40:26 +0200 liblzma: Fix a compression-ratio regression in LZMA1/2 in fast mode. The bug was added in the commit f48fce093b07aeda95c18850f5e086d9f2383380 and thus affected 5.1.4beta and 5.2.0. Luckily the bug cannot cause data corruption or other nasty things. src/liblzma/lzma/lzma_encoder_optimum_fast.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit ae984e31c167d3bc52972ec422dd1ebd5f5d5719 Author: Lasse Collin Date: 2015-02-21 23:00:19 +0200 xz: Fix the fcntl() usage when creating a pipe for the self-pipe trick. Now it reads the old flags instead of blindly setting O_NONBLOCK. The old code may have worked correctly, but this is better. src/xz/file_io.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) commit 2205bb5853098aea36a56df6f5747037175f66b4 Author: Lasse Collin Date: 2015-02-10 15:29:34 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit d935b0cdf3db440269b9d952b2b281b18f8c7b08 Author: Lasse Collin Date: 2015-02-10 15:28:30 +0200 tuklib_cpucores: Use cpuset_getaffinity() on FreeBSD if available. In FreeBSD, cpuset_getaffinity() is the preferred way to get the number of available cores. Thanks to Rui Paulo for the patch. I edited it slightly, but hopefully I didn't break anything. m4/tuklib_cpucores.m4 | 23 ++++++++++++++++++++++- src/common/tuklib_cpucores.c | 18 ++++++++++++++++++ 2 files changed, 40 insertions(+), 1 deletion(-) commit eb61bc58c20769cac4d05f363b9c0e8c9c71a560 Author: Lasse Collin Date: 2015-02-09 22:08:37 +0200 xzdiff: Make the mktemp usage compatible with FreeBSD's mktemp. Thanks to Rui Paulo for the fix. src/scripts/xzdiff.in | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) commit b9a5b6b7a29029680af733082b6a46e0fc01623a Author: Lasse Collin Date: 2015-02-03 21:45:53 +0200 Add a few casts to tuklib_integer.h to silence possible warnings. I heard that Visual Studio 2013 gave warnings without the casts. Thanks to Gabi Davar. src/common/tuklib_integer.h | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) commit c45757135f40e4a0de730ba5fff0100219493982 Author: Lasse Collin Date: 2015-01-26 21:24:39 +0200 liblzma: Set LZMA_MEMCMPLEN_EXTRA depending on the compare method. src/liblzma/common/memcmplen.h | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) commit 3c500174ed5485f550972a2a6109c361e875f069 Author: Lasse Collin Date: 2015-01-26 20:40:16 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit fec88d41e672d9e197c9442aecf02bd0dfa6d516 Author: Lasse Collin Date: 2015-01-26 20:39:28 +0200 liblzma: Silence harmless Valgrind errors. Thanks to Torsten Rupp for reporting this. I had forgotten to run Valgrind before the 5.2.0 release. src/liblzma/lz/lz_encoder.c | 6 ++++++ 1 file changed, 6 insertions(+) commit a9b45badfec0928d20a27c7176c005fa637f7d1e Author: Lasse Collin Date: 2015-01-09 21:50:19 +0200 xz: Fix comments. src/xz/file_io.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) commit 541aee6dd4aa97a809aba281475a21b641bb89e2 Author: Lasse Collin Date: 2015-01-09 21:35:06 +0200 Update THANKS. THANKS | 1 + 1 file changed, 1 insertion(+) commit 4170edc914655310d2363baccf5e615e09b04911 Author: Lasse Collin Date: 2015-01-09 21:34:06 +0200 xz: Don't fail if stdout doesn't support O_NONBLOCK. This is similar to the case with stdin. Thanks to Brad Smith for the bug report and testing on OpenBSD. src/xz/file_io.c | 36 +++++++++++++++--------------------- 1 file changed, 15 insertions(+), 21 deletions(-) commit 04bbc0c2843c50c8ad1cba42b937118e38b0508d Author: Lasse Collin Date: 2015-01-07 19:18:20 +0200 xz: Fix a memory leak in DOS-specific code. src/xz/file_io.c | 2 ++ 1 file changed, 2 insertions(+) commit f0f1f6c7235ffa901cf76fe18e33749e200b3eea Author: Lasse Collin Date: 2015-01-07 19:08:06 +0200 xz: Don't fail if stdin doesn't support O_NONBLOCK. It's a problem at least on OpenBSD which doesn't support O_NONBLOCK on e.g. /dev/null. I'm not surprised if it's a problem on other OSes too since this behavior is allowed in POSIX-1.2008. The code relying on this behavior was committed in June 2013 and included in 5.1.3alpha released on 2013-10-26. Clearly the development releases only get limited testing. src/xz/file_io.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) commit d2d484647d9d9d679f03c75abb0404f67069271c Author: Lasse Collin Date: 2015-01-06 20:30:15 +0200 Tests: Don't hide unexpected error messages in test_files.sh. Hiding them makes no sense since normally there's no error when testing the "good" files. With "bad" files errors are expected and then it makes sense to keep the messages hidden. tests/test_files.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) commit aae6a6aeda51cf94a47e39ad624728f9bee75e30 Author: Lasse Collin Date: 2014-12-30 11:17:16 +0200 Update Solaris notes in INSTALL. Mention the possible "make check" failure on Solaris in the Solaris-specific section of INSTALL. It was already in section 4.5 but it is better mention it in the OS-specific section too. INSTALL | 4 ++++ 1 file changed, 4 insertions(+) commit 7815112153178800a3521b9f31960e7cdc26cfba Author: Lasse Collin Date: 2014-12-26 12:00:05 +0200 Build: POSIX shell isn't required if scripts are disabled. INSTALL | 3 ++- configure.ac | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) commit a0cd05ee71d330b79ead6eb9222e1b24e1559d3a Author: Lasse Collin Date: 2014-12-21 20:48:37 +0200 DOS: Update Makefile. dos/Makefile | 1 + 1 file changed, 1 insertion(+) commit b85ee0905ec4ab7656d22e63519fdd3bedb21f2e Author: Lasse Collin Date: 2014-12-21 19:50:38 +0200 Windows: Fix bin_i486 to bin_i686 in build.bash. windows/build.bash | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) commit cbafa710918195dbba3db02c3fab4f0538235206 Author: Lasse Collin Date: 2014-12-21 18:58:44 +0200 Docs: Use lzma_cputhreads() in 04_compress_easy_mt.c. doc/examples/04_compress_easy_mt.c | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-) commit 8dbb57238d372c7263cfeb3e7f7fd9a73173156a Author: Lasse Collin Date: 2014-12-21 18:56:44 +0200 Docs: Update docs/examples/00_README.txt. doc/examples/00_README.txt | 4 ++++ 1 file changed, 4 insertions(+) commit 6060f7dc76fd6c2a8a1f8e85d0e4d86bb78273e6 Author: Lasse Collin Date: 2014-12-21 18:11:17 +0200 Bump version and soname for 5.2.0. I know that soname != app version, but I skip AGE=1 in -version-info to make the soname match the liblzma version anyway. It doesn't hurt anything as long as it doesn't conflict with library versioning rules. src/liblzma/Makefile.am | 2 +- src/liblzma/api/lzma/version.h | 6 +++--- src/liblzma/liblzma.map | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-)