Как установить zlib python

building Python from source with zlib support

When building Python 3.2.3 from source on Ubuntu 12.04, the zlib module is not available.

I downloaded the official source distribution from python.org, and attempted to build and install it with the following commands.

The make command output includes the following.

After running make install and starting the interpreter, the zlib module cannot be imported.

I confirmed that the zlib1g-dev package is installed on my system.

10 Answers 10

I had a similar problem on CentOS 6.3 and python 3.2.3

Edit /Modules/Setup and uncomment the line:

change to directory /Modules/zlib:

then compiled my python3.2 source.

and was then able to test import zlib and it all worked fine 🙂

Как установить zlib python. Смотреть фото Как установить zlib python. Смотреть картинку Как установить zlib python. Картинка про Как установить zlib python. Фото Как установить zlib python

I am using CentOS 6.6 and was recieving zlib errors. None of the other answers proposed here worked for me (including the fix for CentOS 6.3 of uncommenting a line in Modules/Setup). I have fixed it using the following commands.

Then configuring and installing python as follows:

I can now import zlib in /usr/local/bin/python2.7 with no problems.

The reason is explained here.

The dpkg-dev package installs the dpkg-architecture executable, which enables Python to find the necessary libraries.

The original question was about Python 3.2.3. I also downloaded Python 2.7.3 and confirmed that the same problem exists, and this solution is applicable to it as well.

I ended up going down the rabbit hole trying to figure out why our Python wasn’t building with zlib support and found out that the CPython setup.py does not look at CFLAGS for include dirs, only CPPFLAGS : https://github.com/python/cpython/blob/master/setup.py#L562

The only solution that helped me with installing python 3.5.1 was to apt-get zlib1g-dev (and other packages such as python-setuptools and python-pip) and then rebuild python 3.5.1 from source.

As I undestand new build of python is made with inclusion of previously apt-getted related packages. So when you browse the content of new Python-3.5.1/lib/site-packages there will be pip and setuptools. More importantly, they will be copied to any virtualenv you make using Python-3.5.1 AND this virtualenv will use THEM insted of system-default. This is very, very important to rememmber when installing new python version. Otherwise one might get into a black hole of errors such as:

Источник

horta / install-zlib.bat

This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters

:: Attention!
:: Please, refer to the https://github.com/horta/zlib.install repository
:: for an up-to-date version of this installer.

Omid-Asadi commented Jan 30, 2019

Hi
I execute this bat and a folder has been downloaded in the same directory. what should I do with the folder?

samsantiago commented Nov 5, 2019

Thanks for creating this batch file. Was quite helpful to me today.

horta commented Nov 6, 2019

Thanks @samsantiago You might be interested in this link: https://github.com/horta/zlib.install
That’s where I’ve been maintaining this installed more up-to-date.

@Omid-Asadi Just saw your comment. Sorry for not getting back to you sooner. If you still need the installer, that above link might sort out the problem: the zlib should be installed in your ProgramFiles folder.

sgtpeppr778 commented Dec 22, 2019

Hello @horta, thanks for your work to help make this installation easier. While I’m not new to computers, I am newer to this type of stuff. I’m installing this for use with a python script for RealTerrain by MarcoPolo to make a 3D terrain model and get this error:

Traceback (most recent call last):
File «real-terrain.py», line 11, in
from PIL import Image
ModuleNotFoundError: No module named ‘PIL’

So I figured out I need to install an imaging library and stumbled across this. I clearly don’t have everything set up properly. I think I added everything I need to windows PATH and got cmake installed, but it seems I’m still getting errors:

Fetching https://zlib.net/zlib1211.zip
Downloading https://zlib.net/zlib1211.zip. Exception calling «DownloadFile» with «2» argument(s): «An exception occurred during a WebClient request.»
At line:10 char:1

done.
— Building for: NMake Makefiles
CMake Error at CMakeLists.txt:4 (project):
Generator

does not support platform specification, but platform

CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
— Configuring incomplete, errors occurred!
See also «F:/Python/zlib.install-master/zlib-1.2.11/build/CMakeFiles/CMakeOutput.log».

Any help would be greatly appreciated.
Thanks for your time!

Источник

zlib — Compression compatible with gzip¶

For applications that require data compression, the functions in this module allow compression and decompression, using the zlib library. The zlib library has its own home page at https://www.zlib.net. There are known incompatibilities between the Python module and versions of the zlib library earlier than 1.1.3; 1.1.3 has a security vulnerability, so we recommend using 1.1.4 or later.

zlib’s functions have many options and often need to be used in a particular order. This documentation doesn’t attempt to cover all of the permutations; consult the zlib manual at http://www.zlib.net/manual.html for authoritative information.

The available exception and functions in this module are:

exception zlib. error В¶

Exception raised on compression and decompression errors.

Computes an Adler-32 checksum of data. (An Adler-32 checksum is almost as reliable as a CRC32 but can be computed much more quickly.) The result is an unsigned 32-bit integer. If value is present, it is used as the starting value of the checksum; otherwise, a default value of 1 is used. Passing in value allows computing a running checksum over the concatenation of several inputs. The algorithm is not cryptographically strong, and should not be used for authentication or digital signatures. Since the algorithm is designed for use as a checksum algorithm, it is not suitable for use as a general hash algorithm.

Changed in version 3.6: level can now be used as a keyword parameter.

Returns a compression object, to be used for compressing data streams that won’t fit into memory at once.

The wbits argument controls the size of the history buffer (or the “window size”) used when compressing data, and whether a header and trailer is included in the output. It can take several ranges of values, defaulting to 15 (MAX_WBITS):

+9 to +15: The base-two logarithm of the window size, which therefore ranges between 512 and 32768. Larger values produce better compression at the expense of greater memory usage. The resulting output will include a zlib-specific header and trailer.

в€’9 to в€’15: Uses the absolute value of wbits as the window size logarithm, while producing a raw output stream with no header or trailing checksum.

+25 to +31 = 16 + (9 to 15): Uses the low 4 bits of the value as the window size logarithm, while including a basic gzip header and trailing checksum in the output.

zdict is a predefined compression dictionary. This is a sequence of bytes (such as a bytes object) containing subsequences that are expected to occur frequently in the data that is to be compressed. Those subsequences that are expected to be most common should come at the end of the dictionary.

Changed in version 3.3: Added the zdict parameter and keyword argument support.

Computes a CRC (Cyclic Redundancy Check) checksum of data. The result is an unsigned 32-bit integer. If value is present, it is used as the starting value of the checksum; otherwise, a default value of 0 is used. Passing in value allows computing a running checksum over the concatenation of several inputs. The algorithm is not cryptographically strong, and should not be used for authentication or digital signatures. Since the algorithm is designed for use as a checksum algorithm, it is not suitable for use as a general hash algorithm.

Decompresses the bytes in data, returning a bytes object containing the uncompressed data. The wbits parameter depends on the format of data, and is discussed further below. If bufsize is given, it is used as the initial size of the output buffer. Raises the error exception if any error occurs.

+8 to +15: The base-two logarithm of the window size. The input must include a zlib header and trailer.

0: Automatically determine the window size from the zlib header. Only supported since zlib 1.2.3.5.

в€’8 to в€’15: Uses the absolute value of wbits as the window size logarithm. The input must be a raw stream with no header or trailer.

+24 to +31 = 16 + (8 to 15): Uses the low 4 bits of the value as the window size logarithm. The input must include a gzip header and trailer.

+40 to +47 = 32 + (8 to 15): Uses the low 4 bits of the value as the window size logarithm, and automatically accepts either the zlib or gzip format.

When decompressing a stream, the window size must not be smaller than the size originally used to compress the stream; using a too-small value may result in an error exception. The default wbits value corresponds to the largest window size and requires a zlib header and trailer to be included.

Changed in version 3.6: wbits and bufsize can be used as keyword arguments.

Returns a decompression object, to be used for decompressing data streams that won’t fit into memory at once.

The wbits parameter controls the size of the history buffer (or the “window size”), and what header and trailer format is expected. It has the same meaning as described for decompress().

The zdict parameter specifies a predefined compression dictionary. If provided, this must be the same dictionary as was used by the compressor that produced the data that is to be decompressed.

If zdict is a mutable object (such as a bytearray ), you must not modify its contents between the call to decompressobj() and the first call to the decompressor’s decompress() method.

Changed in version 3.3: Added the zdict parameter.

Compression objects support the following methods:

Compress. compress ( data ) В¶

Compress data, returning a bytes object containing compressed data for at least part of the data in data. This data should be concatenated to the output produced by any preceding calls to the compress() method. Some input may be kept in internal buffers for later processing.

Compress. flush ( [ mode ] ) В¶

Returns a copy of the compression object. This can be used to efficiently compress a set of data that share a common initial prefix.

Changed in version 3.8: Added copy.copy() and copy.deepcopy() support to compression objects.

Decompression objects support the following methods and attributes:

A bytes object that contains any data that was not consumed by the last decompress() call because it exceeded the limit for the uncompressed data buffer. This data has not yet been seen by the zlib machinery, so you must feed it (possibly with further data concatenated to it) back to a subsequent decompress() method call in order to get correct output.

A boolean indicating whether the end of the compressed data stream has been reached.

This makes it possible to distinguish between a properly-formed compressed stream, and an incomplete or truncated one.

Decompress data, returning a bytes object containing the uncompressed data corresponding to at least part of the data in string. This data should be concatenated to the output produced by any preceding calls to the decompress() method. Some of the input data may be preserved in internal buffers for later processing.

Changed in version 3.6: max_length can be used as a keyword argument.

The optional parameter length sets the initial size of the output buffer.

Returns a copy of the decompression object. This can be used to save the state of the decompressor midway through the data stream in order to speed up random seeks into the stream at a future point.

Changed in version 3.8: Added copy.copy() and copy.deepcopy() support to decompression objects.

Information about the version of the zlib library in use is available through the following constants:

The version string of the zlib library actually loaded by the interpreter.

Reading and writing gzip-format files.

The zlib library home page.

The zlib manual explains the semantics and usage of the library’s many functions.

Источник

Python zlib Library Tutorial

Как установить zlib python. Смотреть фото Как установить zlib python. Смотреть картинку Как установить zlib python. Картинка про Как установить zlib python. Фото Как установить zlib python

What is Python zlib

The Python zlib library provides a Python interface to the zlib C library, which is a higher-level abstraction for the DEFLATE lossless compression algorithm. The data format used by the library is specified in the RFC 1950 to 1952, which is available at http://www.ietf.org/rfc/rfc1950.txt.

The zlib compression format is free to use, and is not covered by any patent, so you can safely use it in commercial products as well. It is a lossless compression format (which means you don’t lose any data between compression and decompression), and has the advantage of being portable across different platforms. Another important benefit of this compression mechanism is that it doesn’t expand the data.

The main use of the zlib library is in applications that require compression and decompression of arbitrary data, whether it be a string, structured in-memory content, or files.

The most important functionalities included in this library are compression and decompression. Compression and decompression can both be done as a one-off operations, or by splitting the data into chunks like you’d seem from a stream of data. Both modes of operation are explained in this article.

One of the best things, in my opinion, about the zlib library is that it is compatible with the gzip file format/tool (which is also based on DEFLATE), which is one of the most widely used compression applications on Unix systems.

Compression

Compressing a String of Data

The zlib library provides us with the compress function, which can be used to compress a string of data. The syntax of this function is very simple, taking only two arguments:

An example of using the compress method on a simple string is shown below:

And the result is as follows:

If we change the level to 0 (no compression), then line 5 becomes:

And the new result is:

If you don’t format the string as hexadecimal, as I’ve done in this example, and view the output data you’ll probably notice that the input string is still readable even after being «compressed», although it has a few extra formatting characters around it.

Compressing Large Data Streams

Large data streams can be managed with the compressobj() function, which returns a compression object. The syntax is as follows:

The main difference between the arguments of this function and the compress() function is (aside from the data parameter) the wbits argument, which controls the window size, and whether or not the header and trailer are included in the output.

The possible values for wbits are:

The following code shows how to use the compressobj() function:

After running this code, the result is:

As we can see from the figure above, the phrase «Hello world» has been compressed. Typically this method is used for compressing data streams that won’t fit into memory at once. Although this example does not have a very large stream of data, it serves the purpose of showing the mechanics of the compressobj() function.

You may also be able to see how it would be useful in a larger application in which you can configure the compression and then pass around the compression object to other methods/modules. This can then be used to compress chunks of data in series.

You may also be able to see how it would be useful in a scenario where you have a data stream to compress. Instead of having to accumulate all of the data in memory, you can just call compress.compress(data) and compress.flush() on your data chunk and then move on to the next chunk while leaving the previous one to be cleaned up by garbage collection.

Compressing a File

We can also use the compress() function to compress the data in a file. The syntax is the same as in the first example.

In the example below we will compress a PNG image file named «logo.png» (which, I should note, is already a compressed version of the original raw image).

The example code is as follows:

Free eBook: Git Essentials

Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!

The result is as follows:

As we can see, the file was compressed by 13%.

The only difference between this example and our first one is the source of the data. However, I think it is important to show so you can get an idea of what kind of data can be compressed, whether it be just an ASCII string or binary image data. Simply read in your data from the file like you normally would and call the compress method.

Saving Compressed Data to a File

The compressed data can also be saved to a file for later use. The example below shows how to save some compressed text into a file:

The above example compresses our simple «Hello world» string and saves the compressed data into a file named «outfile.txt». The «outfile.txt» file, when opened with our text editor, looks as follows:

Decompression

Decompressing a String of Data

A compressed string of data can be easily decompressed by using the decompress() function. The syntax is as follows:

This function decompresses the bytes in the data argument. The wbits argument can be used to manage the size of the history buffer. The default value matches the largest window size. It also asks for the inclusion of the header and trailer of the compressed file. The possible values are:

The initial value of the buffer size is indicated in the bufsize argument. However, the important aspect about this parameter is that it doesn’t need to be exact, because if extra buffer size is needed, it will automatically be increased.

The following example shows how to decompress the string of data compressed in our previous example:

The result is as follows:

Decompressing Large Data Streams

Decompressing big data streams may require memory management due to the size or source of your data. It’s possible that you may not be able to use all of the available memory for this task (or you don’t have enough memory), so the decompressobj() method allows you to divide up a stream of data in to several chunks which you can decompress separately.

The syntax of the decompressobj() function is as follows:

This function returns a decompression object, which what you use to decompress the individual data. The wbits argument has the same characteristics as in decompress() function previously explained.

The following code shows how to decompress a big stream of data that is stored in a file. Firstly, the program creates a file named «outfile.txt», which contains the compressed data. Note that the data is compressed using a value of wbits equal to +15. This ensures the creation of a header and a trailer in the data.

The file is then decompressed using chunks of data. Again, in this example the file doesn’t contain a massive amount of data, but nevertheless, it serves the purpose of explaining the buffer concept.

The code is as follows:

After running the above code, we obtain the following results:

Decompressing Data from a File

The compressed data contained in a file can be easily decompressed, as you’ve seen in previous examples. This example is very similar to the previous one in that we’re decompressing data that originates from a file, except that in this case we’re going back to using the one-off decompress method, which decompresses the data in a single method call. This is useful for when your data is small enough to easily fit in memory.

This can be seen from the following example:

The above program opens the file «compressed.dat» created in a previous example, which contains the compressed «Hello world» string.

After running the program we get the following result:

Wrapping Up

The Python library zlib provides us with a useful set of functions for file compression using the zlib format. The functions compress() and decompress() are normally used. However, when there are memory constraints, the functions compressobj() and decompressobj() are available to provide more flexibility by supporting compression/decompression of data streams. These functions help split the data into smaller and more manageable chunks, which can be compressed or decompressed using the compress() and decompress() functions respectively.

Keep in mind that the zlib library also has quite a few more features than what we were able to cover in this article. For example you can use zlib to compute the checksum of some data to verify its integrity when decompressed. For more information on additional features like this, check out the official documentation.

Источник

Как установить zlib python

С помощью Python можно работать с архивами: сжимать и распаковывать. Сегодня расскажем вам о 7 стандартных библиотках, которые помогут вам для работы со сжатием и архивированием. Читайте в этой статье: как сжать данные извлечь из zip, tar, bz2, xz с помощью Python.

Какие библиотеки есть в Python для сжатия данных

Одно из преимуществ Python — это наличие библиотек, которые решают чуть ли не все насущные проблемы. Так вот для сжатия данных в практически любой формат Python-библиотек огромное множество. К стандартным относятся следующие модули:

1) zlib служит для сжатия данных на основе алгоритма Deflate [1].
2) gzip использует уже алгоритм сжатия zlib и служит интерфейсом, аналогичным утилитам gzip и gunzip.
3) lzma использует алгоритм LZMA [2] для сжатия, поддерживает форматы xz и lzma.
4) shutil высокоуровневый модуль для работы с архивами разных форматов. Мы, кстати, использовали его тут.
5) bz2 служит для сжатия bzip2, алгоритм сжатия эффективнее вышеупомянутого Deflate, но работает медленней.
6) zipfile предназначен для работы с zip-файлами, причем модуль предоставляет возможность создавать, читать, записывать и добавлять в архив.
7) tarfile предназначен для работы с tar, поддерживает сжатия форматов gzip, bz2 и lzma.

Сжатие и распаковка данных

Для начала создадим бинарный файл размером 1 МБ и заполним его нулям с помощью псевдоустройства /dev/zero :

Для сжатия нужно указать level (уровень сжатия), которые также берутся из модуля zlib.

Размер получившегося архива равен 1024 байт. Иными словами, мы сжали данные в 3 раза. Распаковка архива выполняется схожим образом:

В результате файл bz2 имеет размер 48 байт, что даже лучше того, как сжимает zlib.

Читайте файлы порциями, когда данные большого размера

Источник

Добавить комментарий

Ваш адрес email не будет опубликован. Обязательные поля помечены *