Распаковать Squashfs и упаковать обратно
здравствуйте друзья! Есть Squashfs, само собой режим read only. Надо взять распаковать, поменять пару строчек текста и запаковать обратно. Как правильно это сделать?
Меняю пару строчек, при этом удаляю столько же символов Комментов, т.е файл больше стать не должен. упаковываю так
в итоге почему-то получается объём больше на 2 мб, почему.
а как узнать какой алгоритм используется и как сжимать
Алгоритм сжатия gzip.
вот что у меня есть:
А вот что получается
Как видите оба в gzip посмотрел степень сжатия по умолчанию идёт 9. Что ещё может быть/
Хардлинков не было?
вот что пишет при сборке
Пробовал собрать squashfs сразу из тех данных, что распаковал? Размер тот же?
да конечно! размер получается такой же.. как и с изменением строки одной. не знаю что и делать ж..
«Number of duplicate files found» то же, что и в варианте с изменённым файлом?
нет ты не понял размер при просто распаковке и сборке выходит на 2 мб больше. Что вношу изменения в строку что не вношу всё равно откуда то прибавляются 2 мб.
Видимо, при создании оригинального образа использовали какой-то хитрый софт, который позволяет избавиться от дополнительной избыточности.
Похожая ситуация с базами GeoIP. Контора, которая их поставляет, публикует формат и даже перловые библиотеки для создания своих баз. Но если распаковать их базу, а потом запаковать их кодом, получится файл почти в два раза больше. Что-то около 100 МБ против 50 МБ.
Я бы начал со сравнения оригинального и перепакованного образов squashfs по байтам. По разнице можно попытаться понять, где их паковщик поступил по-другому.
снимал прошивку банально через dd
Перед упаковкой группируй файлы по типу, чтобы похожие файлы были рядом.
эмм можно понять как группировать?
Например, самое тупое — сортировка по расширению
Какая разница, как ты её снимал? Тут дело в том, что каким-то образом файлы смогли упаковать лучше, чем может упаковать mksquashfs. И при этом оставаясь в рамках формата.
Вообще, ничего удивительного. Если из GIMP’а сохраняешь PNG-файл, он не обязательно самого маленького размера. Специализированными компрессорами можно перебрать все варианты сжатия и сэкономить ещё несколько килобайт.
Тебе нужно найти способ упаковать плотнее. Для этого нужно либо найти некие волшебные утилиты, либо написать их самому.
Хотя бы расскажи откуда ты слил это squashfs.
SquashFS Windows Install Instructions? #43
Comments
Copy link Quote reply
firrae commented Jun 9, 2017
I’m not 100% sure how to install the SquashFS you link to in the prerequisites section of the readme. I get that’s it’s pre-compiled but where do we put the file? I have tried putting it into the PATH on my Windows Server machine but to no luck so far. Any further instruction would be helpful.
pmq20 commented Jun 10, 2017
Otherwise, you could try downloading https://github.com/pmq20/squashfuse/files/691217/sqfs43-win32.zip and unzipping them to C:\Windows .
weikangong commented Jun 10, 2017 •
I’m having the same issue on windows as well. tried unzipping to C:\Windows but its still asking for installation. gives me the following error:
pmq20 commented Jun 10, 2017
@thegreatkang Please try the following and tell me their output ( in order to diagnose),
First open cmd, then,
weikangong commented Jun 10, 2017
@pmq20 i managed to find the problem by using mksquashfs -version you mentioned, i had to copy out the entire content of the folder onto c:\windows (initially i left it untouched after unzipping). now im running into Failed running [«call vcbuild.bat x64»] problem, will try to resolve it. thanks for the amazing work!
firrae commented Jun 10, 2017
I will try this as soon as I get back to work on Monday. Thanks!
firrae commented Jun 21, 2017
This seems to be working now. I will update if there are any other issues I encounter.
rameres8 commented Aug 24, 2018 •
how to compress squashfs-root files with this script
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Squashfs tools for windows
SquashFS is a highly compressed, read only file system often used as a root fs on embedded devices, live systems or simply as a compressed archive format.
Think of it as a .tar.gz that you can mount (or XZ, LZO, LZ4, ZSTD).
This project originally started out as a fork of squashfs-tools 4.3, after encountering some short comings and realizing that there have been no updates on the SourceForge site or mailing list for a long time. Even before the first public release, the fork was replaced with a complete re-write after growing frustrated with the existing code base. For lack of a better name, and because the original appeared to be unmaintained at the time, the name squashfs-tools-ng was kept, although the published code base technically never had any connection to squashfs-tools.
Maintenance of the original squashfs-tools has since resumed, squashfs-tools version 4.4 was released and continues to be maintained in parallel. The utilities provided by squashfs-tools-ng offer alternative tooling and are intentionally named differently, so both packages can be installed side by side.
The actual guts of squashfs-tools-ng are encapsulated in a library with a generic API designed to make SquashFS available to other applications as an embeddable, extensible archive format (or to simply read, write or manipulate SquashFS file systems).
The utility programs are largely command line wrappers around the library. The following tools are provided:
- gensquashfs can be used to produce SquashFS images from gen_init_cpio like file listings or simply pack an input directory. Can use an SELinux contexts file (see selabel_file(5)) to generate SELinux labels.
- rdsquashfs can be used to inspect and unpack SquashFS images.
- sqfs2tar can turn a SquashFS image into a tarball, written to stdout.
- tar2sqfs can turn a tarball (read from stdin) into a SquashFS image.
- sqfsdiff can compare the contents of two SquashFS images.
The library and the tools that produce SquashFS images are designed to operate deterministically. Same input will produce byte-for-byte identical output. Failure to do so is treated as a critical bug.
A number of Linux distributions already offer squashfs-tools-ng through their package management system. Replogy maintains an up to date list:
Pre-built Windows binary Packages
Pre-compiled binary packages for Windows are available here:
Those packages contain the binaries for the tools, the SquashFS library and pre-compiled dependency libraries (zstd, lzo, lzma; others are built in).
The binary package does not contain any source code. See below on how to obtain and compile the source for squashfs-tools-ng. The corresponding source code from which the 3rd party libraries have been built is also available for download at the above location.
The headers and import libraries to build applications that use libsquashfs are included. For convenience, the pre-compiled, 3rd party dependency libraries also come with headers and import libraries.
In short: libsquashfs is LGPLv3 licensed, the utility programs are GPLv3.
Some 3rd party source code is included with more permissive licenses, some of which is actually compiled into libsquashfs. Copyright notices for those must be included when distributing either source or binaries of squashfs-tools-ng.
See COPYING.md for more detailed information.
Getting and Building the Source Code
Official release tarballs can be obtained here:
The official git tree is available at the following locations:
Those locations are kept in sync and the former is a GitHub project that also accepts and handles issues & pull requests.
If you are working on an official release tarball, you can build the package like every autotools based package:
If you work on the git tree, you need to bootstrap the build system first:
If Doxygen is available, a reference manual can be built as follows:
The pre-compiled binary packages for Windows are built using a helper script that uses a MinGW cross toolchain to build squashfs-tools-ng and any of the required dependencies:
Structure of the Source Code
The main functionality of the package is split up into a number of libraries. The actual tools are mainly wrappers around the libraries that combine their functionality in a useful way.
The headers of all the libraries can be found in the include directory, whereas the source code is in a per-library sub-directory within lib . The tools themselves are in sub-directories within bin .
The include directory has a sub-directory sqfs which contains the public headers of libsquashfs.so which are installed along with the library. All other headers are private to this package.
The following components exist:
- libfstree.a built from files in lib/fstree contains functions for manipulating a file system tree.
- libtar.a built from files in lib/tar contains data structures and functions for parsing and creating tar files.
- libsquashfs.so built from files in lib/sqfs contains all kinds of data structures for reading and writing SquashFS archives. Abstractions for data compression and so on. It contains the actual brains of this package.
- libcommon.a built from files in lib/common contains a bunch of commonly used code shared across the utilities.
- libcompat.a built from files in lib/compat contains minimal implementations of POSIX or GNU functions that are not available on some platforms.
- libutil.a contains common utilities that are used internally by both the programs and libsquashfs.so .
Optionally, libsquashfs can be compiled with builtin, custom versions of zlib and lz4. The configure options —with-builtin-zlib and —with-builtin-lz4 can be used. The respective library sources are also in the lib directory.
The tests sub-directory contains unit tests for the libraries.
The extras sub-directory contains a few demo programs that use libsquashfs .
To allow 3rd party applications to use libsquashfs.so without restricting their choice of license, the code in the lib/sqfs and lib/util directories is licensed under the LGPLv3, in contrast to the rest of this package.
A Note on LZO Support
The SquashFS format supports compression using LZO. The liblzo2 library itself is released under the GNU GPL, version 2.
To make the libsquashfs library available as an LGPL library, it cannot be linked against liblzo2 , neither statically nor dynamically.
This legal problem has been solved using the following technical measure:
- libsquashfs , as of right now, does not support LZO compression.
- The libcommon helper library has an implementation of an liblzo2 based compressor. This library and the tools that use it are released under the GPL.
This way, the tools themselves do support LZO compression seamlessly, while the libsquashfs library does not.
Automated Testing and Analysis
The GitHub project for squashfs-tools-ng is registered with Travis-CI and Coverity Scan.
The Travis-CI page shows the current build status for various system configurations for the latest commit on master, as well as pull requests on the GitHub project page.
The Coverity Scan page shows details for static analysis runs on the code, which are triggered manually and thus run less frequently.
A documentation of the SquashFS on-disk format in plain text format can be found in the documentation directory, which is based on an online version that can be found here:
The closest thing to an official web site can be found here:
This location also hosts the Doxygen reference manual for the latest release.
There is currently no official mailing list. So far I used the squashfs-tools mailing list on SourceForge for announcments and I will continue to do so until I am booted off.
About
A new set of tools and libraries for working with SquashFS images