{ "mode": "perldoc", "parameter": "IO::Compress::FAQ", "section": "", "url": "https://www.chedong.com/phpMan.php/perldoc/IO%3A%3ACompress%3A%3AFAQ/json", "generated": "2026-06-15T23:01:22Z", "sections": { "NAME": { "content": "IO::Compress::FAQ -- Frequently Asked Questions about IO::Compress\n", "subsections": [] }, "DESCRIPTION": { "content": "Common questions answered.\n", "subsections": [] }, "GENERAL": { "content": "Compatibility with Unix compress/uncompress.\nAlthough \"Compress::Zlib\" has a pair of functions called \"compress\" and \"uncompress\", they are\n*not* related to the Unix programs of the same name. The \"Compress::Zlib\" module is not\ncompatible with Unix \"compress\".\n\nIf you have the \"uncompress\" program available, you can use this to read compressed files\n\nopen F, \"uncompress -c $filename |\";\nwhile ()\n{\n...\n\nAlternatively, if you have the \"gunzip\" program available, you can use this to read compressed\nfiles\n\nopen F, \"gunzip -c $filename |\";\nwhile ()\n{\n...\n\nand this to write compress files, if you have the \"compress\" program available\n\nopen F, \"| compress -c $filename \";\nprint F \"data\";\n...\nclose F ;\n", "subsections": [ { "name": "Accessing .tar.Z files", "content": "The \"Archive::Tar\" module can optionally use \"Compress::Zlib\" (via the \"IO::Zlib\" module) to\naccess tar files that have been compressed with \"gzip\". Unfortunately tar files compressed with\nthe Unix \"compress\" utility cannot be read by \"Compress::Zlib\" and so cannot be directly\naccessed by \"Archive::Tar\".\n\nIf the \"uncompress\" or \"gunzip\" programs are available, you can use one of these workarounds to\nread \".tar.Z\" files from \"Archive::Tar\"\n\nFirstly with \"uncompress\"\n\nuse strict;\nuse warnings;\nuse Archive::Tar;\n\nopen F, \"uncompress -c $filename |\";\nmy $tar = Archive::Tar->new(*F);\n...\n\nand this with \"gunzip\"\n\nuse strict;\nuse warnings;\nuse Archive::Tar;\n\nopen F, \"gunzip -c $filename |\";\nmy $tar = Archive::Tar->new(*F);\n...\n\nSimilarly, if the \"compress\" program is available, you can use this to write a \".tar.Z\" file\n\nuse strict;\nuse warnings;\nuse Archive::Tar;\nuse IO::File;\n\nmy $fh = IO::File->new( \"| compress -c >$filename\" );\nmy $tar = Archive::Tar->new();\n...\n$tar->write($fh);\n$fh->close ;\n\nHow do I recompress using a different compression?\nThis is easier that you might expect if you realise that all the \"IO::Compress::*\" objects are\nderived from \"IO::File\" and that all the \"IO::Uncompress::*\" modules can read from an \"IO::File\"\nfilehandle.\n\nSo, for example, say you have a file compressed with gzip that you want to recompress with\nbzip2. Here is all that is needed to carry out the recompression.\n\nuse IO::Uncompress::Gunzip ':all';\nuse IO::Compress::Bzip2 ':all';\n\nmy $gzipFile = \"somefile.gz\";\nmy $bzipFile = \"somefile.bz2\";\n\nmy $gunzip = IO::Uncompress::Gunzip->new( $gzipFile )\nor die \"Cannot gunzip $gzipFile: $GunzipError\\n\" ;\n\nbzip2 $gunzip => $bzipFile\nor die \"Cannot bzip2 to $bzipFile: $Bzip2Error\\n\" ;\n\nNote, there is a limitation of this technique. Some compression file formats store extra\ninformation along with the compressed data payload. For example, gzip can optionally store the\noriginal filename and Zip stores a lot of information about the original file. If the original\ncompressed file contains any of this extra information, it will not be transferred to the new\ncompressed file using the technique above.\n" } ] }, "ZIP": { "content": "What Compression Types do IO::Compress::Zip & IO::Uncompress::Unzip support?\nThe following compression formats are supported by \"IO::Compress::Zip\" and\n\"IO::Uncompress::Unzip\"\n\n* Store (method 0)\n\nNo compression at all.\n\n* Deflate (method 8)\n\nThis is the default compression used when creating a zip file with \"IO::Compress::Zip\".\n\n* Bzip2 (method 12)\n\nOnly supported if the \"IO-Compress-Bzip2\" module is installed.\n\n* Lzma (method 14)\n\nOnly supported if the \"IO-Compress-Lzma\" module is installed.\n\nCan I Read/Write Zip files larger the 4 Gig?\nYes, both the \"IO-Compress-Zip\" and \"IO-Uncompress-Unzip\" modules support the zip feature called\n*Zip64*. That allows them to read/write files/buffers larger than 4Gig.\n\nIf you are creating a Zip file using the one-shot interface, and any of the input files is\ngreater than 4Gig, a zip64 complaint zip file will be created.\n\nzip \"really-large-file\" => \"my.zip\";\n\nSimilarly with the one-shot interface, if the input is a buffer larger than 4 Gig, a zip64\ncomplaint zip file will be created.\n\nzip \\$reallylargebuffer => \"my.zip\";\n\nThe one-shot interface allows you to force the creation of a zip64 zip file by including the\n\"Zip64\" option.\n\nzip $filehandle => \"my.zip\", Zip64 => 1;\n\nIf you want to create a zip64 zip file with the OO interface you must specify the \"Zip64\"\noption.\n\nmy $zip = IO::Compress::Zip->new( \"whatever\", Zip64 => 1 );\n\nWhen uncompressing with \"IO-Uncompress-Unzip\", it will automatically detect if the zip file is\nzip64.\n\nIf you intend to manipulate the Zip64 zip files created with \"IO-Compress-Zip\" using an external\nzip/unzip, make sure that it supports Zip64.\n\nIn particular, if you are using Info-Zip you need to have zip version 3.x or better to update a\nZip64 archive and unzip version 6.x to read a zip64 archive.\n\nCan I write more that 64K entries is a Zip files?\nYes. Zip64 allows this. See previous question.\n", "subsections": [ { "name": "Zip Resources", "content": "The primary reference for zip files is the \"appnote\" document available at\n\n\nAn alternatively is the Info-Zip appnote. This is available from\n\n" } ] }, "GZIP": { "content": "", "subsections": [ { "name": "Gzip Resources", "content": "The primary reference for gzip files is RFC 1952 \n\nThe primary site for gzip is .\n" }, { "name": "Dealing with concatenated gzip files", "content": "If the gunzip program encounters a file containing multiple gzip files concatenated together it\nwill automatically uncompress them all. The example below illustrates this behaviour\n\n$ echo abc | gzip -c >x.gz\n$ echo def | gzip -c >>x.gz\n$ gunzip -c x.gz\nabc\ndef\n\nBy default \"IO::Uncompress::Gunzip\" will *not* behave like the gunzip program. It will only\nuncompress the first gzip data stream in the file, as shown below\n\n$ perl -MIO::Uncompress::Gunzip=:all -e 'gunzip \"x.gz\" => \\*STDOUT'\nabc\n\nTo force \"IO::Uncompress::Gunzip\" to uncompress all the gzip data streams, include the\n\"MultiStream\" option, as shown below\n\n$ perl -MIO::Uncompress::Gunzip=:all -e 'gunzip \"x.gz\" => \\*STDOUT, MultiStream => 1'\nabc\ndef\n" }, { "name": "Reading bgzip files with IO::Uncompress::Gunzip", "content": "A \"bgzip\" file consists of a series of valid gzip-compliant data streams concatenated together.\nTo read a file created by \"bgzip\" with \"IO::Uncompress::Gunzip\" use the \"MultiStream\" option as\nshown in the previous section.\n\nSee the section titled \"The BGZF compression format\" in\n for a definition of \"bgzip\".\n" } ] }, "ZLIB": { "content": "", "subsections": [ { "name": "Zlib Resources", "content": "The primary site for the *zlib* compression library is .\n" } ] }, "Bzip2": { "content": "", "subsections": [ { "name": "Bzip2 Resources", "content": "The primary site for bzip2 is .\n" }, { "name": "Dealing with Concatenated bzip2 files", "content": "If the bunzip2 program encounters a file containing multiple bzip2 files concatenated together\nit will automatically uncompress them all. The example below illustrates this behaviour\n\n$ echo abc | bzip2 -c >x.bz2\n$ echo def | bzip2 -c >>x.bz2\n$ bunzip2 -c x.bz2\nabc\ndef\n\nBy default \"IO::Uncompress::Bunzip2\" will *not* behave like the bunzip2 program. It will only\nuncompress the first bunzip2 data stream in the file, as shown below\n\n$ perl -MIO::Uncompress::Bunzip2=:all -e 'bunzip2 \"x.bz2\" => \\*STDOUT'\nabc\n\nTo force \"IO::Uncompress::Bunzip2\" to uncompress all the bzip2 data streams, include the\n\"MultiStream\" option, as shown below\n\n$ perl -MIO::Uncompress::Bunzip2=:all -e 'bunzip2 \"x.bz2\" => \\*STDOUT, MultiStream => 1'\nabc\ndef\n" }, { "name": "Interoperating with Pbzip2", "content": "Pbzip2 () is a parallel implementation of bzip2. The output from\npbzip2 consists of a series of concatenated bzip2 data streams.\n\nBy default \"IO::Uncompress::Bzip2\" will only uncompress the first bzip2 data stream in a pbzip2\nfile. To uncompress the complete pbzip2 file you must include the \"MultiStream\" option, like\nthis.\n\nbunzip2 $input => \\$output, MultiStream => 1\nor die \"bunzip2 failed: $Bunzip2Error\\n\";\n\nHTTP & NETWORK" }, { "name": "Apache::GZip Revisited", "content": "Below is a modperl Apache compression module, called \"Apache::GZip\", taken from\n\n\npackage Apache::GZip;\n#File: Apache::GZip.pm\n\nuse strict vars;\nuse Apache::Constants ':common';\nuse Compress::Zlib;\nuse IO::File;\nuse constant GZIPMAGIC => 0x1f8b;\nuse constant OSMAGIC => 0x03;\n\nsub handler {\nmy $r = shift;\nmy ($fh,$gz);\nmy $file = $r->filename;\nreturn DECLINED unless $fh=IO::File->new($file);\n$r->headerout('Content-Encoding'=>'gzip');\n$r->sendhttpheader;\nreturn OK if $r->headeronly;\n\ntie *STDOUT,'Apache::GZip',$r;\nprint($) while <$fh>;\nuntie *STDOUT;\nreturn OK;\n}\n\nsub TIEHANDLE {\nmy($class,$r) = @;\n# initialize a deflation stream\nmy $d = deflateInit(-WindowBits=>-MAXWBITS()) || return undef;\n\n# gzip header -- don't ask how I found out\n$r->print(pack(\"nccVcc\",GZIPMAGIC,ZDEFLATED,0,time(),0,OSMAGIC));\n\nreturn bless { r => $r,\ncrc => crc32(undef),\nd => $d,\nl => 0\n},$class;\n}\n\nsub PRINT {\nmy $self = shift;\nforeach (@) {\n# deflate the data\nmy $data = $self->{d}->deflate($);\n$self->{r}->print($data);\n# keep track of its length and crc\n$self->{l} += length($);\n$self->{crc} = crc32($,$self->{crc});\n}\n}\n\nsub DESTROY {\nmy $self = shift;\n\n# flush the output buffers\nmy $data = $self->{d}->flush;\n$self->{r}->print($data);\n\n# print the CRC and the total length (uncompressed)\n$self->{r}->print(pack(\"LL\",@{$self}{qw/crc l/}));\n}\n\n1;\n\nHere's the Apache configuration entry you'll need to make use of it. Once set it will result in\neverything in the /compressed directory will be compressed automagically.\n\n\nSetHandler perl-script\nPerlHandler Apache::GZip\n\n\nAlthough at first sight there seems to be quite a lot going on in \"Apache::GZip\", you could sum\nup what the code was doing as follows -- read the contents of the file in \"$r->filename\",\ncompress it and write the compressed data to standard output. That's all.\n\nThis code has to jump through a few hoops to achieve this because\n\n1. The gzip support in \"Compress::Zlib\" version 1.x can only work with a real filesystem\nfilehandle. The filehandles used by Apache modules are not associated with the filesystem.\n\n2. That means all the gzip support has to be done by hand - in this case by creating a tied\nfilehandle to deal with creating the gzip header and trailer.\n\n\"IO::Compress::Gzip\" doesn't have that filehandle limitation (this was one of the reasons for\nwriting it in the first place). So if \"IO::Compress::Gzip\" is used instead of \"Compress::Zlib\"\nthe whole tied filehandle code can be removed. Here is the rewritten code.\n\npackage Apache::GZip;\n\nuse strict vars;\nuse Apache::Constants ':common';\nuse IO::Compress::Gzip;\nuse IO::File;\n\nsub handler {\nmy $r = shift;\nmy ($fh,$gz);\nmy $file = $r->filename;\nreturn DECLINED unless $fh=IO::File->new($file);\n$r->headerout('Content-Encoding'=>'gzip');\n$r->sendhttpheader;\nreturn OK if $r->headeronly;\n\nmy $gz = IO::Compress::Gzip->new( '-', Minimal => 1 )\nor return DECLINED ;\n\nprint $gz $ while <$fh>;\n\nreturn OK;\n}\n\nor even more succinctly, like this, using a one-shot gzip\n\npackage Apache::GZip;\n\nuse strict vars;\nuse Apache::Constants ':common';\nuse IO::Compress::Gzip qw(gzip);\n\nsub handler {\nmy $r = shift;\n$r->headerout('Content-Encoding'=>'gzip');\n$r->sendhttpheader;\nreturn OK if $r->headeronly;\n\ngzip $r->filename => '-', Minimal => 1\nor return DECLINED ;\n\nreturn OK;\n}\n\n1;\n\nThe use of one-shot \"gzip\" above just reads from \"$r->filename\" and writes the compressed data\nto standard output.\n\nNote the use of the \"Minimal\" option in the code above. When using gzip for Content-Encoding you\nshould *always* use this option. In the example above it will prevent the filename being\nincluded in the gzip header and make the size of the gzip data stream a slight bit smaller.\n" }, { "name": "Compressed files and Net::FTP", "content": "The \"Net::FTP\" module provides two low-level methods called \"stor\" and \"retr\" that both return\nfilehandles. These filehandles can used with the \"IO::Compress/Uncompress\" modules to compress\nor uncompress files read from or written to an FTP Server on the fly, without having to create a\ntemporary file.\n\nFirstly, here is code that uses \"retr\" to uncompressed a file as it is read from the FTP Server.\n\nuse Net::FTP;\nuse IO::Uncompress::Gunzip qw(:all);\n\nmy $ftp = Net::FTP->new( ... )\n\nmy $retrfh = $ftp->retr($compressedfilename);\ngunzip $retrfh => $outFilename, AutoClose => 1\nor die \"Cannot uncompress '$compressedfile': $GunzipError\\n\";\n\nand this to compress a file as it is written to the FTP Server\n\nuse Net::FTP;\nuse IO::Compress::Gzip qw(:all);\n\nmy $storfh = $ftp->stor($filename);\ngzip \"filename\" => $storfh, AutoClose => 1\nor die \"Cannot compress '$filename': $GzipError\\n\";\n" } ] }, "MISC": { "content": "Using \"InputLength\" to uncompress data embedded in a larger file/buffer.\nA fairly common use-case is where compressed data is embedded in a larger file/buffer and you\nwant to read both.\n\nAs an example consider the structure of a zip file. This is a well-defined file format that\nmixes both compressed and uncompressed sections of data in a single file.\n\nFor the purposes of this discussion you can think of a zip file as sequence of compressed data\nstreams, each of which is prefixed by an uncompressed local header. The local header contains\ninformation about the compressed data stream, including the name of the compressed file and, in\nparticular, the length of the compressed data stream.\n\nTo illustrate how to use \"InputLength\" here is a script that walks a zip file and prints out how\nmany lines are in each compressed file (if you intend write code to walking through a zip file\nfor real see \"Walking through a zip file\" in IO::Uncompress::Unzip ). Also, although this\nexample uses the zlib-based compression, the technique can be used by the other\n\"IO::Uncompress::*\" modules.\n\nuse strict;\nuse warnings;\n\nuse IO::File;\nuse IO::Uncompress::RawInflate qw(:all);\n\nuse constant ZIPLOCALHDRSIG => 0x04034b50;\nuse constant ZIPLOCALHDRLENGTH => 30;\n\nmy $file = $ARGV[0] ;\n\nmy $fh = IO::File->new( \"<$file\" )\nor die \"Cannot open '$file': $!\\n\";\n\nwhile (1)\n{\nmy $sig;\nmy $buffer;\n\nmy $x ;\n($x = $fh->read($buffer, ZIPLOCALHDRLENGTH)) == ZIPLOCALHDRLENGTH\nor die \"Truncated file: $!\\n\";\n\nmy $signature = unpack (\"V\", substr($buffer, 0, 4));\n\nlast unless $signature == ZIPLOCALHDRSIG;\n\n# Read Local Header\nmy $gpFlag = unpack (\"v\", substr($buffer, 6, 2));\nmy $compressedMethod = unpack (\"v\", substr($buffer, 8, 2));\nmy $compressedLength = unpack (\"V\", substr($buffer, 18, 4));\nmy $uncompressedLength = unpack (\"V\", substr($buffer, 22, 4));\nmy $filenamelength = unpack (\"v\", substr($buffer, 26, 2));\nmy $extralength = unpack (\"v\", substr($buffer, 28, 2));\n\nmy $filename ;\n$fh->read($filename, $filenamelength) == $filenamelength\nor die \"Truncated file\\n\";\n\n$fh->read($buffer, $extralength) == $extralength\nor die \"Truncated file\\n\";\n\nif ($compressedMethod != 8 && $compressedMethod != 0)\n{\nwarn \"Skipping file '$filename' - not deflated $compressedMethod\\n\";\n$fh->read($buffer, $compressedLength) == $compressedLength\nor die \"Truncated file\\n\";\nnext;\n}\n\nif ($compressedMethod == 0 && $gpFlag & 8 == 8)\n{\ndie \"Streamed Stored not supported for '$filename'\\n\";\n}\n\nnext if $compressedLength == 0;\n\n# Done reading the Local Header\n\nmy $inf = IO::Uncompress::RawInflate->new( $fh,\nTransparent => 1,\nInputLength => $compressedLength )\nor die \"Cannot uncompress $file [$filename]: $RawInflateError\\n\" ;\n\nmy $linecount = 0;\n\nwhile (<$inf>)\n{\n++ $linecount;\n}\n\nprint \"$filename: $linecount\\n\";\n}\n\nThe majority of the code above is concerned with reading the zip local header data. The code\nthat I want to focus on is at the bottom.\n\nwhile (1) {\n\n# read local zip header data\n# get $filename\n# get $compressedLength\n\nmy $inf = IO::Uncompress::RawInflate->new( $fh,\nTransparent => 1,\nInputLength => $compressedLength )\nor die \"Cannot uncompress $file [$filename]: $RawInflateError\\n\" ;\n\nmy $linecount = 0;\n\nwhile (<$inf>)\n{\n++ $linecount;\n}\n\nprint \"$filename: $linecount\\n\";\n}\n\nThe call to \"IO::Uncompress::RawInflate\" creates a new filehandle $inf that can be used to read\nfrom the parent filehandle $fh, uncompressing it as it goes. The use of the \"InputLength\" option\nwill guarantee that *at most* $compressedLength bytes of compressed data will be read from the\n$fh filehandle (The only exception is for an error case like a truncated file or a corrupt data\nstream).\n\nThis means that once RawInflate is finished $fh will be left at the byte directly after the\ncompressed data stream.\n\nNow consider what the code looks like without \"InputLength\"\n\nwhile (1) {\n\n# read local zip header data\n# get $filename\n# get $compressedLength\n\n# read all the compressed data into $data\nread($fh, $data, $compressedLength);\n\nmy $inf = IO::Uncompress::RawInflate->new( \\$data,\nTransparent => 1 )\nor die \"Cannot uncompress $file [$filename]: $RawInflateError\\n\" ;\n\nmy $linecount = 0;\n\nwhile (<$inf>)\n{\n++ $linecount;\n}\n\nprint \"$filename: $linecount\\n\";\n}\n\nThe difference here is the addition of the temporary variable $data. This is used to store a\ncopy of the compressed data while it is being uncompressed.\n\nIf you know that $compressedLength isn't that big then using temporary storage won't be a\nproblem. But if $compressedLength is very large or you are writing an application that other\npeople will use, and so have no idea how big $compressedLength will be, it could be an issue.\n\nUsing \"InputLength\" avoids the use of temporary storage and means the application can cope with\nlarge compressed data streams.\n\nOne final point -- obviously \"InputLength\" can only be used whenever you know the length of the\ncompressed data beforehand, like here with a zip file.\n", "subsections": [] }, "SUPPORT": { "content": "General feedback/questions/bug reports should be sent to \n(preferred) or .\n", "subsections": [] }, "SEE ALSO": { "content": "Compress::Zlib, IO::Compress::Gzip, IO::Uncompress::Gunzip, IO::Compress::Deflate,\nIO::Uncompress::Inflate, IO::Compress::RawDeflate, IO::Uncompress::RawInflate,\nIO::Compress::Bzip2, IO::Uncompress::Bunzip2, IO::Compress::Lzma, IO::Uncompress::UnLzma,\nIO::Compress::Xz, IO::Uncompress::UnXz, IO::Compress::Lzip, IO::Uncompress::UnLzip,\nIO::Compress::Lzop, IO::Uncompress::UnLzop, IO::Compress::Lzf, IO::Uncompress::UnLzf,\nIO::Compress::Zstd, IO::Uncompress::UnZstd, IO::Uncompress::AnyInflate,\nIO::Uncompress::AnyUncompress\n\nIO::Compress::FAQ\n\nFile::GlobMapper, Archive::Zip, Archive::Tar, IO::Zlib\n", "subsections": [] }, "AUTHOR": { "content": "This module was written by Paul Marquess, \"pmqs@cpan.org\".\n", "subsections": [] }, "MODIFICATION HISTORY": { "content": "See the Changes file.\n", "subsections": [] }, "COPYRIGHT AND LICENSE": { "content": "Copyright (c) 2005-2021 Paul Marquess. All rights reserved.\n\nThis program is free software; you can redistribute it and/or modify it under the same terms as\nPerl itself.\n", "subsections": [] } }, "summary": "IO::Compress::FAQ -- Frequently Asked Questions about IO::Compress", "flags": [], "examples": [], "see_also": [] }