mozpack package

Submodules

mozpack.archive module

mozpack.archive.create_tar_bz2_from_files(fp, files, compresslevel=9)

Create a tar.bz2 file deterministically from files.

This is a glorified wrapper around create_tar_from_files that adds bzip2 compression.

This function is similar to create_tar_gzip_from_files().

mozpack.archive.create_tar_from_files(fp, files)

Create a tar file deterministically.

Receives a dict mapping names of files in the archive to local filesystem paths or mozpack.files.BaseFile instances.

The files will be archived and written to the passed file handle opened for writing.

Only regular files can be written.

FUTURE accept a filename argument (or create APIs to write files)

mozpack.archive.create_tar_gz_from_files(fp, files, filename=None, compresslevel=9)

Create a tar.gz file deterministically from files.

This is a glorified wrapper around create_tar_from_files that adds gzip compression.

The passed file handle should be opened for writing in binary mode. When the function returns, all data has been written to the handle.

mozpack.copier module

class mozpack.copier.FileCopier

Bases: mozpack.copier.FileRegistry

FileRegistry with the ability to copy the registered files to a separate directory.

copy(destination, skip_if_older=True, remove_unaccounted=True, remove_all_directory_symlinks=True, remove_empty_directories=True)

Copy all registered files to the given destination path. The given destination can be an existing directory, or not exist at all. It can’t be e.g. a file. The copy process acts a bit like rsync: files are not copied when they don’t need to (see mozpack.files for details on file.copy).

By default, files in the destination directory that aren’t registered are removed and empty directories are deleted. In addition, all directory symlinks in the destination directory are deleted: this is a conservative approach to ensure that we never accidently write files into a directory that is not the destination directory. In the worst case, we might have a directory symlink in the object directory to the source directory.

To disable removing of unregistered files, pass remove_unaccounted=False. To disable removing empty directories, pass remove_empty_directories=False. In rare cases, you might want to maintain directory symlinks in the destination directory (at least those that are not required to be regular directories): pass remove_all_directory_symlinks=False. Exercise caution with this flag: you almost certainly do not want to preserve directory symlinks.

Returns a FileCopyResult that details what changed.

class mozpack.copier.FileCopyResult

Bases: object

Represents results of a FileCopier.copy operation.

property existing_files_count
property removed_directories_count
property removed_files_count
property updated_files_count
class mozpack.copier.FileRegistry

Bases: object

Generic container to keep track of a set of BaseFile instances. It preserves the order under which the files are added, but doesn’t keep track of empty directories (directories are not stored at all). The paths associated with the BaseFile instances are relative to an unspecified (virtual) root directory.

registry = FileRegistry() registry.add(‘foo/bar’, file_instance)

add(path, content)

Add a BaseFile instance to the container, under the given path.

contains(pattern)

Return whether the container contains paths matching the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.

input_to_outputs_tree()

Return a dictionary mapping each input path to the set of impacted output paths.

All paths are normalized.

match(pattern)

Return the list of paths, stored in the container, matching the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.

output_to_inputs_tree()

Return a dictionary mapping each output path to the set of its required input paths.

All paths are normalized.

paths()

Return all paths stored in the container, in the order they were added.

remove(pattern)

Remove paths matching the given pattern from the container. See the mozpack.path.match documentation for a description of the handled patterns.

required_directories()

Return the set of directories required by the paths in the container, in no particular order. The returned directories are relative to an unspecified (virtual) root directory (and do not include said root directory).

class mozpack.copier.FileRegistrySubtree(base, registry)

Bases: object

A proxy class to give access to a subtree of an existing FileRegistry.

Note this doesn’t implement the whole FileRegistry interface.

add(path, content)
contains(pattern)
match(pattern)
paths()
remove(pattern)
class mozpack.copier.Jarrer(compress=True)

Bases: mozpack.copier.FileRegistry, mozpack.files.BaseFile

FileRegistry with the ability to copy and pack the registered files as a jar file. Also acts as a BaseFile instance, to be copied with a FileCopier.

add(path, content, compress=None)

Add a BaseFile instance to the container, under the given path.

copy(dest, skip_if_older=True)

Pack all registered files in the given destination jar. The given destination jar may be a path to jar file, or a Dest instance for a jar file. If the destination jar file exists, its (compressed) contents are used instead of the registered BaseFile instances when appropriate.

open()

Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.

preload(paths)

Add the given set of paths to the list of preloaded files. See mozpack.mozjar.JarWriter documentation for details on jar preloading.

mozpack.dmg module

mozpack.dmg.check_tools(*tools)

Check that each tool named in tools exists in SUBSTS and is executable.

mozpack.dmg.chmod(dir)

Set permissions of DMG contents correctly

Make a symlink to /Applications. The symlink name is a space so we don’t have to localize it. The Applications folder icon will be shown in Finder, which should be clear enough for users.

mozpack.dmg.create_dmg(source_directory, output_dmg, volume_name, extra_files)

Create a DMG disk image at the path output_dmg from source_directory.

Use volume_name as the disk image volume name, and use extra_files as a list of tuples of (filename, relative path) to copy into the disk image.

mozpack.dmg.create_dmg_from_staged(stagedir, output_dmg, tmpdir, volume_name)

Given a prepared directory stagedir, produce a DMG at output_dmg.

mozpack.dmg.extract_dmg(dmgfile, output, dsstore=None, icon=None, background=None)
mozpack.dmg.extract_dmg_contents(dmgfile, destdir)
mozpack.dmg.generate_hfs_file(stagedir, tmpdir, volume_name)

When cross compiling, we zero fill an hfs file, that we will turn into a DMG. To do so we test the size of the staged dir, and add some slight padding to that.

mozpack.dmg.mkdir(dir)
mozpack.dmg.rsync(source, dest)

rsync the contents of directory source into directory dest

mozpack.dmg.set_folder_icon(dir, tmpdir)

Set HFS attributes of dir to use a custom icon

mozpack.errors module

exception mozpack.errors.AccumulatedErrors

Bases: Exception

Exception type raised from errors.accumulate()

class mozpack.errors.ErrorCollector

Bases: object

Error handling/logging class. A global instance, errors, is provided for convenience.

Warnings, errors and fatal errors may be logged by calls to the following functions:

errors.warn(message) errors.error(message) errors.fatal(message)

Warnings only send the message on the logging output, while errors and fatal errors send the message and throw an ErrorMessage exception. The exception, however, may be deferred. See further below.

Errors may be ignored by calling:

errors.ignore_errors()

After calling that function, only fatal errors throw an exception.

The warnings, errors or fatal errors messages may be augmented with context information when a context is provided. Context is defined by a pair (filename, linenumber), and may be set with errors.context() used as a context manager:

with errors.context(filename, linenumber):

errors.warn(message)

Arbitrary nesting is supported, both for errors.context calls:
with errors.context(filename1, linenumber1):

errors.warn(message) with errors.context(filename2, linenumber2):

errors.warn(message)

as well as for function calls:
def func():

errors.warn(message)

with errors.context(filename, linenumber):

func()

Errors and fatal errors can have their exception thrown at a later time, allowing for several different errors to be reported at once before throwing. This is achieved with errors.accumulate() as a context manager:

with errors.accumulate():
if test1:

errors.error(message1)

if test2:

errors.error(message2)

In such cases, a single AccumulatedErrors exception is thrown, but doesn’t contain information about the exceptions. The logged messages do.

ERROR = 2
FATAL = 3
WARN = 1
accumulate()
context(file, line)
property count
error(msg)
fatal(msg)
get_context()
ignore_errors(ignore=True)
out = <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
warn(msg)
exception mozpack.errors.ErrorMessage

Bases: Exception

Exception type raised from errors.error() and errors.fatal()

mozpack.executables module

mozpack.executables.elfhack(path)

Execute the elfhack command on the given path.

mozpack.executables.get_type(path_or_fileobj)

Check the signature of the give file and returns what kind of executable matches.

mozpack.executables.is_executable(path)

Return whether a given file path points to an executable or a library, where an executable or library is identified by:

  • the file extension on OS/2 and WINNT

  • the file signature on OS/X and ELF systems (GNU/Linux, Android, BSD, Solaris)

As this function is intended for use to choose between the ExecutableFile and File classes in FileFinder, and choosing ExecutableFile only matters on OS/2, OS/X, ELF and WINNT (in GCC build) systems, we don’t bother detecting other kind of executables.

mozpack.executables.may_elfhack(path)

Return whether elfhack() should be called

mozpack.executables.may_strip(path)

Return whether strip() should be called

mozpack.executables.strip(path)

Execute the STRIP command with STRIP_FLAGS on the given path.

mozpack.files module

class mozpack.files.AbsoluteSymlinkFile(path)

Bases: mozpack.files.File

File class that is copied by symlinking (if available).

This class only works if the target path is absolute.

copy(dest, skip_if_older=True)

Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).

static excluded(dest)
class mozpack.files.BaseFile

Bases: object

Base interface and helper for file copying. Derived class may implement their own copy function, or rely on BaseFile.copy using the open() member function and/or the path property.

static any_newer(dest, inputs)

Compares the modification time of dest to multiple input files, and returns whether any of the inputs is newer (has a later mtime) than dest.

copy(dest, skip_if_older=True)

Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).

inputs()

Return an iterable of the input file paths that impact this output file.

static is_older(first, second)

Compares the modification time of two files, and returns whether the first file is older than the second file.

property mode

Return the file’s unix mode, or None if it has no meaning.

static normalize_mode(mode)
open()

Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.

read()
size()

Returns size of the entry.

Derived classes are highly encouraged to override this with a more optimal implementation.

class mozpack.files.BaseFinder(base, minify=False, minify_js=False, minify_js_verify_command=None)

Bases: object

contains(pattern)

Return whether some files under the base directory match the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.

find(pattern)

Yield path, BaseFile_instance pairs for all files under the base directory and its subdirectories that match the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.

get(path)

Obtain a single file.

Where find is tailored towards matching multiple files, this method is used for retrieving a single file. Use this method when performance is critical.

Returns a BaseFile if at most one file exists or None otherwise.

class mozpack.files.ComposedFinder(finders)

Bases: mozpack.files.BaseFinder

Composes multiple File Finders in some sort of virtual file system.

A ComposedFinder is initialized from a dictionary associating paths to *Finder instances.

Note this could be optimized to be smarter than getting all the files in advance.

find(pattern)

Yield path, BaseFile_instance pairs for all files under the base directory and its subdirectories that match the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.

class mozpack.files.DeflatedFile(file)

Bases: mozpack.files.BaseFile

File class for members of a jar archive. DeflatedFile.copy() effectively extracts the file from the jar archive.

open()

Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.

class mozpack.files.Dest(path)

Bases: object

Helper interface for BaseFile.copy. The interface works as follows: - read() and write() can be used to sequentially read/write from the

underlying file.

  • a call to read() after a write() will re-open the underlying file and read from it.

  • a call to write() after a read() will re-open the underlying file, emptying it, and write to it.

close()
exists()
property name
read(length=- 1)
write(data)
class mozpack.files.ExecutableFile(path)

Bases: mozpack.files.File

File class for executable and library files on OS/2, OS/X and ELF systems. (see mozpack.executables.is_executable documentation).

copy(dest, skip_if_older=True)

Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).

class mozpack.files.ExistingFile(required)

Bases: mozpack.files.BaseFile

File class that represents a file that may exist but whose content comes from elsewhere.

This purpose of this class is to account for files that are installed via external means. It is typically only used in manifests or in registries to account for files.

When asked to copy, this class does nothing because nothing is known about the source file/data.

Instances of this class come in two flavors: required and optional. If an existing file is required, it must exist during copy() or an error is raised.

copy(dest, skip_if_older=True)

Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).

inputs()

Return an iterable of the input file paths that impact this output file.

class mozpack.files.ExtractedTarFile(tar, info)

Bases: mozpack.files.GeneratedFile

File class for members of a tar archive. Contents of the underlying file are extracted immediately and stored in memory.

property mode

Return the file’s unix mode, or None if it has no meaning.

read()
class mozpack.files.File(path)

Bases: mozpack.files.BaseFile

File class for plain files.

inputs()

Return an iterable of the input file paths that impact this output file.

property mode

Return the file’s unix mode, as returned by os.stat().st_mode.

read()

Return the contents of the file.

size()

Returns size of the entry.

Derived classes are highly encouraged to override this with a more optimal implementation.

class mozpack.files.FileFinder(base, find_executables=False, ignore=(), ignore_broken_symlinks=False, find_dotfiles=False, **kargs)

Bases: mozpack.files.BaseFinder

Helper to get appropriate BaseFile instances from the file system.

get(path)

Obtain a single file.

Where find is tailored towards matching multiple files, this method is used for retrieving a single file. Use this method when performance is critical.

Returns a BaseFile if at most one file exists or None otherwise.

class mozpack.files.FileListFinder(files)

Bases: mozpack.files.BaseFinder

Finder for a literal list of file names.

find(pattern)

Yield path, BaseFile_instance pairs for all files under the base directory and its subdirectories that match the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.

class mozpack.files.GeneratedFile(content)

Bases: mozpack.files.BaseFile

File class for content with no previous existence on the filesystem.

property content
inputs()

Return an iterable of the input file paths that impact this output file.

open()

Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.

read()
size()

Returns size of the entry.

Derived classes are highly encouraged to override this with a more optimal implementation.

class mozpack.files.HardlinkFile(path)

Bases: mozpack.files.File

File class that is copied by hard linking (if available)

This is similar to the AbsoluteSymlinkFile, but with hard links. The symlink implementation requires paths to be absolute, because they are resolved at read time, which makes relative paths messy. Hard links resolve paths at link-creation time, so relative paths are fine.

copy(dest, skip_if_older=True)

Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).

class mozpack.files.JarFinder(base, reader, **kargs)

Bases: mozpack.files.BaseFinder

Helper to get appropriate DeflatedFile instances from a JarReader.

class mozpack.files.ManifestFile(base, entries=None)

Bases: mozpack.files.BaseFile

File class for a manifest file. It takes individual manifest entries (using the add() and remove() member functions), and adjusts them to be relative to the base path for the manifest, given at creation. .. rubric:: Example

There is a manifest entry “content foobar foobar/content/” relative to “foobar/chrome”. When packaging, the entry will be stored in jar:foobar/omni.ja!/chrome/chrome.manifest, which means the entry will have to be relative to “chrome” instead of “foobar/chrome”. This doesn’t really matter when serializing the entry, since this base path is not written out, but it matters when moving the entry at the same time, e.g. to jar:foobar/omni.ja!/chrome.manifest, which we don’t do currently but could in the future.

add(entry)

Add the given entry to the manifest. Entries are rebased at open() time instead of add() time so that they can be more easily remove()d.

isempty()

Return whether there are manifest entries to write

open()

Return a file-like object allowing to read() the serialized content of the manifest.

remove(entry)

Remove the given entry from the manifest.

class mozpack.files.MercurialFile(client, rev, path)

Bases: mozpack.files.BaseFile

File class for holding data from Mercurial.

open()

Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.

read()
class mozpack.files.MercurialRevisionFinder(repo, rev='.', recognize_repo_paths=False, **kwargs)

Bases: mozpack.files.BaseFinder

A finder that operates on a specific Mercurial revision.

get(path)

Obtain a single file.

Where find is tailored towards matching multiple files, this method is used for retrieving a single file. Use this method when performance is critical.

Returns a BaseFile if at most one file exists or None otherwise.

class mozpack.files.MinifiedJavaScript(file, verify_command=None)

Bases: mozpack.files.BaseFile

File class for minifying JavaScript files.

open()

Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.

class mozpack.files.MinifiedProperties(file)

Bases: mozpack.files.BaseFile

File class for minified properties. This wraps around a BaseFile instance, and removes lines starting with a # from its content.

open()

Return a file-like object allowing to read() the minified content of the properties file.

class mozpack.files.PreprocessedFile(path, depfile_path, marker, defines, extra_depends=None, silence_missing_directive_warnings=False)

Bases: mozpack.files.BaseFile

File class for a file that is preprocessed. PreprocessedFile.copy() runs the preprocessor on the file to create the output.

copy(dest, skip_if_older=True)

Invokes the preprocessor to create the destination file.

inputs()

Return an iterable of the input file paths that impact this output file.

class mozpack.files.TarFinder(base, tar, **kargs)

Bases: mozpack.files.BaseFinder

Helper to get files from a TarFile.

mozpack.manifests module

class mozpack.manifests.InstallManifest(path=None, fileobj=None)

Bases: object

Describes actions to be used with a copier.FileCopier instance.

This class facilitates serialization and deserialization of data used to construct a copier.FileCopier and to perform copy operations.

The manifest defines source paths, destination paths, and a mechanism by which the destination file should come into existence.

Entries in the manifest correspond to the following types:

copy – The file specified as the source path will be copied to the

destination path.

link – The destination path will be a symlink or hardlink to the source

path. If symlinks are not supported, a copy will be performed.

exists – The destination path is accounted for and won’t be deleted by

the FileCopier. If the destination path doesn’t exist, an error is raised.

optional – The destination path is accounted for and won’t be deleted by

the FileCopier. No error is raised if the destination path does not exist.

patternlink – Paths matched by the expression in the source path

will be symlinked or hardlinked to the destination directory.

patterncopy – Similar to patternlink except files are copied, not

symlinked/hardlinked.

preprocess – The file specified at the source path will be run through

the preprocessor, and the output will be written to the destination path.

content – The destination file will be created with the given content.

Version 1 of the manifest was the initial version. Version 2 added optional path support Version 3 added support for pattern entries. Version 4 added preprocessed file support. Version 5 added content support.

CONTENT = 8
COPY = 2
CURRENT_VERSION = 5
FIELD_SEPARATOR = '\x1f'
OPTIONAL_EXISTS = 4
PATTERN_COPY = 6
PREPROCESS = 7
REQUIRED_EXISTS = 3
add_content(content, dest)

Add a file with the given content.

add_copy(source, dest)

Add a copy to this manifest.

source will be copied to dest.

add_entries_from(other, base='')

Copy data from another mozpack.copier.InstallManifest instance, adding an optional base prefix to the destination.

This allows to merge two manifests into a single manifest, or two take the tagged union of two manifests.

Add a link to this manifest.

dest will be either a symlink or hardlink to source.

add_optional_exists(dest)

Record that a destination file may exist.

This effectively prevents the listed file from being deleted. Unlike a “required exists” file, files of this type do not raise errors if the destination file does not exist.

add_pattern_copy(base, pattern, dest)

Add a pattern match that results in copies.

See add_pattern_link() for usage.

Add a pattern match that results in links being created.

A FileFinder will be created with its base set to base and FileFinder.find() will be called with pattern to discover source files. Each source file will be either symlinked or hardlinked under dest.

Filenames under dest are constructed by taking the path fragment after base and concatenating it with dest. e.g.

<base>/foo/bar.h -> <dest>/foo/bar.h

add_preprocess(source, dest, deps, marker='#', defines={}, silence_missing_directive_warnings=False)

Add a preprocessed file to this manifest.

source will be passed through preprocessor.py, and the output will be written to dest.

add_required_exists(dest)

Record that a destination file must exist.

This effectively prevents the listed file from being deleted.

populate_registry(registry, defines_override={}, link_policy='symlink')

Populate a mozpack.copier.FileRegistry instance with data from us.

The caller supplied a FileRegistry instance (or at least something that conforms to its interface) and that instance is populated with data from this manifest.

Defines can be given to override the ones in the manifest for preprocessing.

The caller can set a link policy. This determines whether symlinks, hardlinks, or copies are used for LINK and PATTERN_LINK.

write(path=None, fileobj=None, expand_pattern=False)

Serialize this manifest to a file or file object.

If path is specified, that file will be written to. If fileobj is specified, the serialized content will be written to that file object.

It is an error if both are specified.

exception mozpack.manifests.UnreadableInstallManifest

Bases: Exception

Raised when an invalid install manifest is parsed.

mozpack.mozjar module

class mozpack.mozjar.Deflater(compress=True, compress_level=9)

Bases: object

File-like interface to zlib compression. The data is actually not compressed unless the compressed form is smaller than the uncompressed data.

close()

Close the Deflater.

property compressed

Return whether the data should be compressed.

property compressed_data

Return the compressed data, if the data should be compressed (real compressed size smaller than the uncompressed size), or the uncompressed data otherwise.

property compressed_size

Return the compressed size of the data written to the Deflater. If the Deflater is set not to compress, the uncompressed size is returned. Otherwise, if the data should not be compressed (the real compressed size is bigger than the uncompressed size), return the uncompressed size.

property uncompressed_size

Return the size of the data written to the Deflater.

write(data)

Append a buffer to the Deflater.

class mozpack.mozjar.JarCdirEnd(data=None)

Bases: mozpack.mozjar.JarStruct

End of central directory record.

MAGIC = 101010256
STRUCT = {'cdir_disk': 'uint16', 'cdir_entries': 'uint16', 'cdir_offset': 'uint32', 'cdir_size': 'uint32', 'comment': 'comment_size', 'comment_size': 'uint16', 'disk_entries': 'uint16', 'disk_num': 'uint16'}
class mozpack.mozjar.JarCdirEntry(data=None)

Bases: mozpack.mozjar.JarStruct

Central directory file header

MAGIC = 33639248
STRUCT = {'compressed_size': 'uint32', 'compression': 'uint16', 'crc32': 'uint32', 'creator_version': 'uint16', 'disknum': 'uint16', 'external_attr': 'uint32', 'extrafield': 'extrafield_size', 'extrafield_size': 'uint16', 'filecomment': 'filecomment_size', 'filecomment_size': 'uint16', 'filename': 'filename_size', 'filename_size': 'uint16', 'general_flag': 'uint16', 'internal_attr': 'uint16', 'lastmod_date': 'uint16', 'lastmod_time': 'uint16', 'min_version': 'uint16', 'offset': 'uint32', 'uncompressed_size': 'uint32'}
class mozpack.mozjar.JarFileReader(header, data)

Bases: object

File-like class for use by JarReader to give access to individual files within a Jar archive.

close()

Free the uncompressed data buffer.

property compressed_data

Return the raw compressed data.

read(length=- 1)

Read some amount of uncompressed data.

readlines()

Return a list containing all the lines of data in the uncompressed data.

seek(pos, whence=0)

Change the current position in the uncompressed data. Subsequent reads will start from there.

property uncompressed_data

Return the uncompressed data.

class mozpack.mozjar.JarLocalFileHeader(data=None)

Bases: mozpack.mozjar.JarStruct

Local file header

MAGIC = 67324752
STRUCT = {'compressed_size': 'uint32', 'compression': 'uint16', 'crc32': 'uint32', 'extra_field': 'extra_field_size', 'extra_field_size': 'uint16', 'filename': 'filename_size', 'filename_size': 'uint16', 'general_flag': 'uint16', 'lastmod_date': 'uint16', 'lastmod_time': 'uint16', 'min_version': 'uint16', 'uncompressed_size': 'uint32'}
class mozpack.mozjar.JarLog(file=None, fileobj=None)

Bases: dict

Helper to read the file Gecko generates when setting MOZ_JAR_LOG_FILE. The jar log is then available as a dict with the jar path as key, and the corresponding access log as a list value. Only the first access to a given member of a jar is stored.

class mozpack.mozjar.JarReader(file=None, fileobj=None, data=None)

Bases: object

Class with methods to read Jar files. Can open standard jar files as well as Mozilla jar files (see further details in the JarWriter documentation).

close()

Free some resources associated with the Jar.

property compression
property entries

Return an ordered dict of central directory entries, indexed by filename, in the order they appear in the Jar archive central directory. Directory entries are skipped.

property is_optimized

Return whether the jar archive is optimized.

property last_preloaded

Return the name of the last file that is set to be preloaded. See JarWriter documentation for more details on preloading.

exception mozpack.mozjar.JarReaderError

Bases: Exception

Error type for Jar reader errors.

class mozpack.mozjar.JarStruct(data=None)

Bases: object

Helper used to define ZIP archive raw data structures. Data structures handled by this helper all start with a magic number, defined in subclasses MAGIC field as a 32-bits unsigned integer, followed by data structured as described in subclasses STRUCT field.

The STRUCT field contains a list of (name, type) pairs where name is a field name, and the type can be one of ‘uint32’, ‘uint16’ or one of the field names. In the latter case, the field is considered to be a string buffer with a length given in that field. For example,

STRUCT = [

(‘version’, ‘uint32’), (‘filename_size’, ‘uint16’), (‘filename’, ‘filename_size’)

]

describes a structure with a ‘version’ 32-bits unsigned integer field, followed by a ‘filename_size’ 16-bits unsigned integer field, followed by a filename_size-long string buffer ‘filename’.

Fields that are used as other fields size are not stored in objects. In the above example, an instance of such subclass would only have two attributes:

obj[‘version’] obj[‘filename’]

filename_size would be obtained with len(obj[‘filename’]).

JarStruct subclasses instances can be either initialized from existing data (deserialized), or with empty fields.

TYPE_MAPPING = {'uint16': (b'H', 2), 'uint32': (b'I', 4)}
static get_data(type, data)

Deserialize a single field of given type (must be one of JarStruct.TYPE_MAPPING) at the given offset in the given data.

serialize()

Serialize the data structure according to the data structure definition from self.STRUCT.

property size

Return the size of the data structure, given the current values of all variable length fields.

class mozpack.mozjar.JarWriter(file=None, fileobj=None, compress=True, compress_level=9)

Bases: object

Class with methods to write Jar files. Can write more-or-less standard jar archives as well as jar archives optimized for Gecko. See the documentation for the close() member function for a description of both layouts.

add(name, data, compress=None, mode=None, skip_duplicates=False)

Add a new member to the jar archive, with the given name and the given data. The compress option indicates how the given data should be compressed (one of JAR_STORED or JAR_DEFLATE), or compressed according to the default defined when creating the JarWriter (None). True and False are allowed values for backwards compatibility, mapping, respectively, to JAR_DEFLATE and JAR_STORED. When the data should be compressed, it is only really compressed if the compressed size is smaller than the uncompressed size. The mode option gives the unix permissions that should be stored for the jar entry. If a duplicated member is found skip_duplicates will prevent raising an exception if set to True. The given data may be a buffer, a file-like instance, a Deflater or a JarFileReader instance. The latter two allow to avoid uncompressing data to recompress it.

finish()

Flush and close the Jar archive.

Standard jar archives are laid out like the following:

  • Local file header 1

  • File data 1

  • Local file header 2

  • File data 2

  • (…)

  • Central directory entry pointing at Local file header 1

  • Central directory entry pointing at Local file header 2

  • (…)

  • End of central directory, pointing at first central directory entry.

Jar archives optimized for Gecko are laid out like the following:

  • 32-bits unsigned integer giving the amount of data to preload.

  • Central directory entry pointing at Local file header 1

  • Central directory entry pointing at Local file header 2

  • (…)

  • End of central directory, pointing at first central directory entry.

  • Local file header 1

  • File data 1

  • Local file header 2

  • File data 2

  • (…)

  • End of central directory, pointing at first central directory entry.

The duplication of the End of central directory is to accomodate some Zip reading tools that want an end of central directory structure to follow the central directory entries.

preload(files)

Set which members of the jar archive should be preloaded when opening the archive in Gecko. This reorders the members according to the order of given list.

exception mozpack.mozjar.JarWriterError

Bases: Exception

Error type for Jar writer errors.

mozpack.path module

Like os.path, with a reduced set of functions, and with normalized path separators (always use forward slashes). Also contains a few additional utilities not found in os.path.

mozpack.path.abspath(path)
mozpack.path.basedir(path, bases)

Given a list of directories (bases), return which one contains the given path. If several matches are found, the deepest base directory is returned.

basedir('foo/bar/baz', ['foo', 'baz', 'foo/bar']) returns 'foo/bar' (‘foo’ and ‘foo/bar’ both match, but ‘foo/bar’ is the deepest match)

mozpack.path.basename(path)
mozpack.path.cargo_workaround(path)
mozpack.path.commonprefix(paths)
mozpack.path.dirname(path)
mozpack.path.join(*paths)
mozpack.path.match(path, pattern)

Return whether the given path matches the given pattern. An asterisk can be used to match any string, including the null string, in one part of the path:

foo matches *, f* or fo*o

However, an asterisk matching a subdirectory may not match the null string:

foo/bar does not match foo/*/bar

If the pattern matches one of the ancestor directories of the path, the patch is considered matching:

foo/bar matches foo

Two adjacent asterisks can be used to match files and zero or more directories and subdirectories.

foo/bar matches foo/**/bar, or **/bar

mozpack.path.normpath(path)
mozpack.path.normsep(path)

Normalize path separators, by using forward slashes instead of whatever os.sep is.

mozpack.path.realpath(path)
mozpack.path.rebase(oldbase, base, relativepath)

Return relativepath relative to base instead of oldbase.

mozpack.path.relpath(path, start)
mozpack.path.split(path)

Return the normalized path as a list of its components.

split('foo/bar/baz') returns ['foo', 'bar', 'baz']

mozpack.path.splitext(path)

mozpack.unify module

class mozpack.unify.UnifiedBuildFinder(finder1, finder2, **kargs)

Bases: mozpack.unify.UnifiedFinder

Specialized UnifiedFinder for Mozilla applications packaging. It allows “*.manifest” files to differ in their order, and unifies “buildconfig.html” files by merging their content.

unify_file(path, file1, file2)

Unify files taking Mozilla application special cases into account. Otherwise defer to UnifiedFinder.unify_file.

class mozpack.unify.UnifiedExecutableFile(executable1, executable2)

Bases: mozpack.files.BaseFile

File class for executable and library files that to be unified with ‘lipo’.

copy(dest, skip_if_older=True)

Create a fat executable from the two Mach-O executable given when creating the instance. skip_if_older is ignored.

class mozpack.unify.UnifiedFinder(finder1, finder2, sorted=[], **kargs)

Bases: mozpack.files.BaseFinder

Helper to get unified BaseFile instances from two distinct trees on the file system.

unify_file(path, file1, file2)

Given two BaseFiles and the path they were found at, return a unified version of the files. If the files match, the first BaseFile may be returned. If the files don’t match or one of them is None, the method returns None. Subclasses may decide to unify by using one of the files in that case.

mozpack.unify.may_unify_binary(file)

Return whether the given BaseFile instance is an ExecutableFile that may be unified. Only non-fat Mach-O binaries are to be unified.

Module contents