{
    "mode": "man",
    "parameter": "GITFORMAT-CHUNK",
    "section": "5",
    "url": "https://www.chedong.com/phpMan.php/man/GITFORMAT-CHUNK/5/json",
    "generated": "2026-10-22T13:16:51Z",
    "synopsis": "Used by gitformat-commit-graph(5) and the \"MIDX\" format (see the pack format documentation in\ngitformat-pack(5)).",
    "sections": {
        "NAME": {
            "content": "gitformat-chunk - Chunk-based file formats\n",
            "subsections": []
        },
        "SYNOPSIS": {
            "content": "Used by gitformat-commit-graph(5) and the \"MIDX\" format (see the pack format documentation in\ngitformat-pack(5)).\n",
            "subsections": []
        },
        "DESCRIPTION": {
            "content": "Some file formats in Git use a common concept of \"chunks\" to describe sections of the file.\nThis allows structured access to a large file by scanning a small \"table of contents\" for the\nremaining data. This common format is used by the commit-graph and multi-pack-index files.\nSee the multi-pack-index format in gitformat-pack(5) and the commit-graph format in\ngitformat-commit-graph(5) for how they use the chunks to describe structured data.\n\nA chunk-based file format begins with some header information custom to that format. That\nheader should include enough information to identify the file type, format version, and\nnumber of chunks in the file. From this information, that file can determine the start of the\nchunk-based region.\n\nThe chunk-based region starts with a table of contents describing where each chunk starts and\nends. This consists of (C+1) rows of 12 bytes each, where C is the number of chunks. Consider\nthe following table:\n\n| Chunk ID (4 bytes) | Chunk Offset (8 bytes) |\n|--------------------|------------------------|\n| ID[0]              | OFFSET[0]              |\n| ...                | ...                    |\n| ID[C]              | OFFSET[C]              |\n| 0x0000             | OFFSET[C+1]            |\n\nEach row consists of a 4-byte chunk identifier (ID) and an 8-byte offset. Each integer is\nstored in network-byte order.\n\nThe chunk identifier ID[i] is a label for the data stored within this file from OFFSET[i]\n(inclusive) to OFFSET[i+1] (exclusive). Thus, the size of the i`th chunk is equal to the\ndifference between `OFFSET[i+1] and OFFSET[i]. This requires that the chunk data appears\ncontiguously in the same order as the table of contents.\n\nThe final entry in the table of contents must be four zero bytes. This confirms that the\ntable of contents is ending and provides the offset for the end of the chunk-based data.\n\nNote: The chunk-based format expects that the file contains at least a trailing hash after\nOFFSET[C+1].\n\nFunctions for working with chunk-based file formats are declared in chunk-format.h. Using\nthese methods provide extra checks that assist developers when creating new file formats.\n",
            "subsections": []
        },
        "WRITING CHUNK-BASED FILE FORMATS": {
            "content": "To write a chunk-based file format, create a struct chunkfile by calling initchunkfile() and\npass a struct hashfile pointer. The caller is responsible for opening the hashfile and\nwriting header information so the file format is identifiable before the chunk-based format\nbegins.\n\nThen, call addchunk() for each chunk that is intended for writing. This populates the\nchunkfile with information about the order and size of each chunk to write. Provide a\nchunkwritefn function pointer to perform the write of the chunk data upon request.\n\nCall writechunkfile() to write the table of contents to the hashfile followed by each of the\nchunks. This will verify that each chunk wrote the expected amount of data so the table of\ncontents is correct.\n\nFinally, call freechunkfile() to clear the struct chunkfile data. The caller is responsible\nfor finalizing the hashfile by writing the trailing hash and closing the file.\n",
            "subsections": []
        },
        "READING CHUNK-BASED FILE FORMATS": {
            "content": "To read a chunk-based file format, the file must be opened as a memory-mapped region. The\nchunk-format API expects that the entire file is mapped as a contiguous memory region.\n\nInitialize a struct chunkfile pointer with initchunkfile(NULL).\n\nAfter reading the header information from the beginning of the file, including the chunk\ncount, call readtableofcontents() to populate the struct chunkfile with the list of\nchunks, their offsets, and their sizes.\n\nExtract the data information for each chunk using pairchunk() or readchunk():\n\n•   pairchunk() assigns a given pointer with the location inside the memory-mapped file\ncorresponding to that chunk’s offset. If the chunk does not exist, then the pointer is\nnot modified.\n\n•   readchunk() takes a chunkreadfn function pointer and calls it with the appropriate\ninitial pointer and size information. The function is not called if the chunk does not\nexist. Use this method to read chunks if you need to perform immediate parsing or if you\nneed to execute logic based on the size of the chunk.\n\nAfter calling these methods, call freechunkfile() to clear the struct chunkfile data. This\nwill not close the memory-mapped region. Callers are expected to own that data for the\ntimeframe the pointers into the region are needed.\n",
            "subsections": []
        },
        "EXAMPLES": {
            "content": "These file formats use the chunk-format API, and can be used as examples for future formats:\n\n•   commit-graph: see writecommitgraphfile() and parsecommitgraph() in commit-graph.c\nfor how the chunk-format API is used to write and parse the commit-graph file format\ndocumented in the commit-graph file format in gitformat-commit-graph(5).\n\n•   multi-pack-index: see writemidxinternal() and loadmultipackindex() in midx.c for how\nthe chunk-format API is used to write and parse the multi-pack-index file format\ndocumented in the multi-pack-index file format section of gitformat-pack(5).\n",
            "subsections": []
        },
        "GIT": {
            "content": "Part of the git(1) suite\n\nGit 2.43.0                                   07/02/2025                           GITFORMAT-CHUNK(5)",
            "subsections": []
        }
    },
    "summary": "gitformat-chunk - Chunk-based file formats",
    "flags": [],
    "examples": [
        "These file formats use the chunk-format API, and can be used as examples for future formats:",
        "•   commit-graph: see writecommitgraphfile() and parsecommitgraph() in commit-graph.c",
        "for how the chunk-format API is used to write and parse the commit-graph file format",
        "documented in the commit-graph file format in gitformat-commit-graph(5).",
        "•   multi-pack-index: see writemidxinternal() and loadmultipackindex() in midx.c for how",
        "the chunk-format API is used to write and parse the multi-pack-index file format",
        "documented in the multi-pack-index file format section of gitformat-pack(5)."
    ],
    "see_also": []
}