{
    "mode": "perldoc",
    "parameter": "Hash::Ordered::Benchmarks",
    "section": "",
    "url": "https://www.chedong.com/phpMan.php/perldoc/Hash%3A%3AOrdered%3A%3ABenchmarks/json",
    "generated": "2026-10-07T18:28:01Z",
    "sections": {
        "NAME": {
            "content": "Hash::Ordered::Benchmarks - Ordered hash benchmarking\n",
            "subsections": []
        },
        "VERSION": {
            "content": "version 0.014\n",
            "subsections": []
        },
        "INTRODUCTION": {
            "content": "The Hash::Ordered internals are simple: a hash of data and an array of ordered keys. I thought\nthis would perform well for common tasks and likely outperform more complicated ordered hash\nimplementations, so I decided to do some benchmarking to test it.\n\nNote: since the initial benchmarking, \"Hash::Ordered\" gained just-in-time indexing of the keys\narray to support faster tombstone deletion, which adds some conditional data structures to the\ninternals. It also now supports \"tie\". The revised benchmarks include the \"tie\" mode for\ncomparison with other tied hash implementations.\n",
            "subsections": []
        },
        "MODULES TESTED": {
            "content": "In my review of alternatives to \"Hash::Ordered\", six seemed sufficiently general-purpose to be\nworth benchmarking. The modules tested are listed in the benchmark output in shorthand:\n\n*   Array::AsHash — denoted \"a:ah\"\n\n*   Array::OrdHash — denoted \"a:oh\"\n\n*   Data::XHash — denoted \"d:xh\"\n\n*   Hash::Ordered — denoted \"h:o\" and marked with \"*\"\n\n*   Tie::Hash::Indexed — denoted \"t:h:i\"\n\n*   Tie::IxHash — denoted \"t:ix\"\n\n*   Tie::LLHash — denoted \"t:llh\"\n\nNote that Tie::Hash::Indexed is written in XS and also may require forced installation as its\ntests often fail for Perl 5.18+ due to the hash randomization change.\n\nIf there are different methods of doing something with a module, the variations are described in\neach section below.\n",
            "subsections": []
        },
        "BENCHMARKS": {
            "content": "I conducted benchmarking with the Benchmark module. The test script is in the \"devel\" directory\nof the distribution. Tests were run on Perl 5.20.2 on a Mac Book Pro\n(darwin-thread-multi-2level). Each benchmark ran for 5 CPU seconds.\n\nBenchmarks were run at several different scales to reveal differences in efficiency as hash size\ngrows. The details are described in each section below.\n\nA seed list of keys and values was generated from random integers using Math::Random::MT::Auto.\nThe same seed list was used for all benchmarks unless otherwise noted.\n\nI did not test advanced features of these modules, as apples-to-apples comparison is difficult.\nStill, the performance on common, simple measures could suggest how features that combine these\noperations might perform.\n",
            "subsections": [
                {
                    "name": "Ordered hash creation",
                    "content": "I tested hash creation for 10, 100 and 1000 elements. For some modules there were different\noptions for creating a hash:\n\n*   \"Array::AsHash\" takes an array-reference with an option to use it directly or to clone it.\nIn one case I provided the seed list array reference with the clone option to true\n(\"a:ahcp\"). In another case I created a new array reference from the seed list and provided\nit directly (\"a:ahrf\").\n\n*   \"Hash::Ordered\" can be initialized either with \"new\" (\"h:ooo\") or via \"tie\" (\"h:oth\").\n\n*   \"Tie::IxHash\" can be initialized either with \"new\" (\"t:ixoo\") or via \"tie\" (\"t:ixth\").\n\n*   \"Data::XHash\" can be created with a list (\"t:xhls\") or an array reference (\"t:xhrf\").\n\nAs expected, when \"Array::AsHash\" gets an array reference, it's very fast. \"Tie::Hash::Indexed\"\ndoes well here, also. Of the non-XS, more hash-like choices, \"Hash::Ordered\" does well.\n\nResults for ordered hash creation for 10 elements\nt:h:i   136030/s\na:ahrf   111411/s\nh:ooo   101293/s  *\nh:oth    98646/s  *\nt:ixoo    61853/s\nt:ixth    61715/s\na:ahcp    56375/s\na:oh    54337/s\nt:llh    33553/s\nd:xhls    14068/s\nd:xhrf    13926/s\n\nResults for ordered hash creation for 100 elements\nt:h:i    16503/s\na:ahrf    15398/s\nh:ooo    11226/s  *\nh:oth    10793/s  *\na:oh     7783/s\nt:ixth     7570/s\nt:ixoo     7405/s\na:ahcp     7035/s\nt:llh     3533/s\nd:xhls     1561/s\nd:xhrf     1550/s\n\nResults for ordered hash creation for 1000 elements\nt:h:i     1552/s\na:ahrf     1509/s\nh:ooo     1160/s  *\nh:oth     1158/s  *\na:oh      815/s\nt:ixth      772/s\nt:ixoo      757/s\na:ahcp      684/s\nt:llh      340/s\nd:xhls      154/s\nd:xhrf      152/s\n"
                },
                {
                    "name": "Getting hash elements",
                    "content": "I tested retrieving values for 10% of the keys, randomly selected, from hashes of 10, 100 and\n1000 elements. The hash was created beforehand so the benchmarks reflect only element access.\n\nSome modules had choices for how to retrieve an value, usually between a method (denoted with\n\"oo\"), tied hash access (\"th\") or with a dereference (\"rf\").\n\nGenerally, method calls turned out faster than other approaches for a given module,\ndemonstrating the inefficiency of tied objects.\n\nResults for fetching ~10% of 10 elements\nh:ooo  1844781/s  *\nd:xhoo  1292883/s\nt:ixoo  1187104/s\nt:h:i   932793/s\nh:oth   817346/s  *\nd:xhrf   703441/s\nt:ixth   649291/s\na:oh   560060/s\nt:llh   514911/s\na:ah   260639/s\n\nResults for fetching ~10% of 100 elements\nh:ooo   285983/s  *\nd:xhoo   183292/s\nt:ixoo   165100/s\nt:h:i   128713/s\nh:oth   107213/s  *\nd:xhrf    87049/s\nt:ixth    79642/s\na:oh    66109/s\nt:llh    58741/s\na:ah    27533/s\n\nResults for fetching ~10% of 1000 elements\nh:ooo    30342/s  *\nd:xhoo    19004/s\nt:ixoo    17132/s\nt:h:i    13269/s\nh:oth    11100/s  *\nd:xhrf     8919/s\nt:ixth     7844/s\na:oh     6763/s\nt:llh     5666/s\na:ah     2772/s\n"
                },
                {
                    "name": "Setting hash elements",
                    "content": "I tested changing values for 10% of the keys, randomly selected, from hashes of 10, 100 and 1000\nelements. The hash was created beforehand so the benchmarks reflect only element mutation. No\nnew keys were added.\n\nSome modules had choices for how to modify a value, usually between a method (denoted with\n\"oo\"), tied hash access (\"th\") or with a dereference (\"rf\").\n\nAgain, methods outperformed.\n\nResults for replacing ~10% of 10 elements\nh:ooo  1378880/s  *\nt:h:i   945403/s\nd:xhoo   941643/s\nt:ixoo   887283/s\nh:oth   652269/s  *\nt:llh   590160/s\nd:xhrf   537694/s\na:oh   530787/s\nt:ixth   508001/s\na:ah   159258/s\n\nResults for replacing ~10% of 100 elements\nh:ooo   192769/s  *\nt:h:i   126284/s\nd:xhoo   119845/s\nt:ixoo   113992/s\nh:oth    81159/s  *\nt:llh    72403/s\nd:xhrf    64791/s\na:oh    62666/s\nt:ixth    59809/s\na:ah    16405/s\n\nResults for replacing ~10% of 1000 elements\nh:ooo    19909/s  *\nt:h:i    13445/s\nd:xhoo    12487/s\nt:ixoo    11601/s\nh:oth     8357/s  *\nt:llh     7503/s\nd:xhrf     6599/s\na:oh     6410/s\nt:ixth     6118/s\na:ah     1651/s\n"
                },
                {
                    "name": "Adding hash elements",
                    "content": "I tested adding 10, 100 and 1000 elements to an empty hash.\n\nSome modules had choices for how to append a value, usually between a method (denoted with\n\"oo\"), tied hash access (\"th\") or with a dereference (\"rf\").\n\nFor \"Tie::LLHash\", I did not use the \"lazy\" option, but did the equivalent using \"tied\" and a\nmethod call:\n\ntied(%tllh)->last( irand(), 42 ) for 1 .. $n;\n\nGenerally, it seemed like the differences were smaller than for other benchmarks. Methods still\noutperformed.\n\nResults for adding 10 elements to empty hash\nh:ooo   341022/s  *\nt:h:i   295079/s\nt:ixoo   258981/s\nh:oth   245996/s  *\nt:ixth   211341/s\nt:llh   191298/s\na:oh   137447/s\na:ah   112651/s\nd:xhoo    87215/s\nd:xhrf    80379/s\n\nResults for adding 100 elements to empty hash\nh:ooo    58519/s  *\nt:h:i    55166/s\nt:ixoo    48658/s\nh:oth    42066/s  *\nt:ixth    38632/s\na:oh    34842/s\nt:llh    28384/s\nd:xhoo    24841/s\nd:xhrf    21517/s\na:ah    13726/s\n\nResults for adding 1000 elements to empty hash\nh:ooo     6497/s  *\nt:h:i     6108/s\nt:ixoo     5528/s\nh:oth     4650/s  *\nt:ixth     4329/s\na:oh     4233/s\nd:xhoo     3121/s\nt:llh     3011/s\nd:xhrf     2696/s\na:ah     1423/s\n"
                },
                {
                    "name": "Deleting hash elements",
                    "content": "I tested creating hashes of 10, 100 and 1000 elements and then deleting 10% of the keys, chosen\nrandomly. I would have liked to have isolated creation from deletion, but I couldn't figure out\na way to do that given how \"Benchmark\" runs the same tests over and over.\n\nSome modules had choices for how to delete a value, usually between a method (denoted with\n\"oo\"), tied hash access (\"th\") or with a dereference (\"rf\").\n\nThe performance changes (or lack thereof) at the three different sizes reveals implementation\ndifferences. (Though recall that some of this is the creation performance difference as well as\ndeletion difference.)\n\nFor example, \"Tie::Hash::Indexed\" XS does very well, which could be its good creation\nperformance, but could also be good deletion.\n\n\"Hash::Ordered\" does linear search deleting a key for the 10 element hash, but automatically\nswitches to indexed, tombstone deletion for the larger hashes. When deleting only 10% of keys,\ngarbage collection of tombstoned keys never occurs, so that amortized cost is not included.\n\n\"Tie::LLHash\" improves at larger sizes as deleting from a linked list is faster than splicing\nout an element of an array. Conversely, \"Array::AsHash\" just gets worse.\n\nResults for creating 10 element hash then deleting ~10%\nt:h:i   131578/s\nh:ooo    94598/s  *\nh:oth    84018/s  *\na:ah    67109/s\nt:ixoo    55477/s\nt:ixth    52792/s\na:oh    46938/s\nt:llh    30399/s\nd:xhoo    13756/s\nd:xhrf    13499/s\n\nResults for creating 100 element hash then deleting ~10%\nt:h:i    17420/s\nh:ooo     9242/s  *\nh:oth     8438/s  *\na:oh     5738/s\nt:ixoo     3922/s\nt:ixth     3862/s\na:ah     3286/s\nt:llh     3250/s\nd:xhoo     1508/s\nd:xhrf     1499/s\n\nResults for creating 1000 element hash then deleting ~10%\nt:h:i     1635/s\nh:ooo      934/s  *\nh:oth      799/s  *\nt:llh      319/s\na:oh      204/s\nd:xhoo      152/s\nd:xhrf      151/s\nt:ixoo       78/s\nt:ixth       78/s\na:ah       40/s\n"
                },
                {
                    "name": "Extracting the hash as a list",
                    "content": "I tested getting an ordered list of pairs from hashes of 10, 100 and 1000 elements. The hash was\ncreated beforehand so the benchmarks reflect only conversion to a list.\n\nOddly, modules that usually have more than one way to do things don't for this. Even\n\"Tie::IxHash\" doesn't really have an OO way to do it, so I did it longhand:\n\n@list = map { $ => $tixoo->FETCH($) } $tixoo->Keys;\n\nBecause \"Array::AsHash\" keeps its internal representation as an ordered list of pairs, it\noutperforms the rest handily as it merely needs to dereference that data structure.\n\nResults for listing pairs of 10 element hash\na:ah   321044/s\nh:ooo   178288/s  *\nt:ixoo    89263/s\nt:h:i    79184/s\nh:oth    56112/s  *\nt:ixth    48009/s\na:oh    47433/s\nt:llh    37996/s\nd:xh    37439/s\n\nResults for listing pairs of 100 element hash\na:ah    36399/s\nh:ooo    19537/s  *\nt:ixoo     9049/s\nt:h:i     7768/s\nh:oth     6254/s  *\na:oh     5060/s\nt:ixth     4907/s\nd:xh     4122/s\nt:llh     3813/s\n\nResults for listing pairs of 1000 element hash\na:ah     3784/s\nh:ooo     1959/s  *\nt:ixoo      905/s\nt:h:i      773/s\nh:oth      625/s  *\na:oh      523/s\nt:ixth      492/s\nd:xh      427/s\nt:llh      377/s\n"
                }
            ]
        },
        "CONCLUSION": {
            "content": "With the exception of hash creation and element deletion, \"Hash::Ordered\" generally outperformed\nthe other ordered hash implementations. Even for creation, it was the fastest of the pure-Perl,\nhash-based implementations, often by a large margin.\n\nIn the original release of \"Hash::Ordered\", deletion got worse as the hash size grew. The new\nJIT indexing with tombstones now makes deletion far faster than any pure-Perl implementation.\n\n\"Array::AsHash\", with the opposite internal implementation compared to \"Hash::Ordered\", performs\nbest at creation and listing pairs, but is dead last at element access and modification. I\nbelieve the poor performance is mostly due to extra indirection (e.g. an extra function call)\nand logic in the element access methods. For uses that don't require much element access and\nhave lots of creation/serialization, it could still be a useful choice.\n\nGenerally, every module that depends on \"tie\" for some portion of its implementation pays a\nsubstantial performance penalty. Comparing \"Hash::Ordered\" benchmarks with and without \"tie\" for\nindividual element operations shows how severe this penalty can be. \"Tie::Hash::Indexed\" —\nlikely because of its XS implementation — performs decently, but not well enough in my opinion\nto justify its use.\n\nAs the author of \"Hash::Ordered\", I'm clearly biased, but I think these benchmarks make a very\ngood case for it being the \"go to\" module for pure-Perl, general-purpose ordered hashes.\n",
            "subsections": []
        },
        "AUTHOR": {
            "content": "David Golden <dagolden@cpan.org>\n",
            "subsections": []
        },
        "COPYRIGHT AND LICENSE": {
            "content": "This software is Copyright (c) 2014 by David Golden.\n\nThis is free software, licensed under:\n\nThe Apache License, Version 2.0, January 2004\n",
            "subsections": []
        }
    },
    "summary": "Hash::Ordered::Benchmarks - Ordered hash benchmarking",
    "flags": [],
    "examples": [],
    "see_also": []
}