perldoc > DB_File

man perldoc info pydoc3 ri search

🏷️ NAME

DB_File - Perl5 access to Berkeley DB version 1.x

🚀 Quick Reference

Use Case	Command	Description
Open DB_HASH database	tie %hash, 'DB_File', $filename, $flags, $mode, $DB_HASH	🔑 Create or open a hash-type database
Open DB_BTREE database	tie %hash, 'DB_File', $filename, $flags, $mode, $DB_BTREE	🌳 Create or open a btree-type database
Open DB_RECNO database	tie @array, 'DB_File', $filename, $flags, $mode, $DB_RECNO	📄 Create or open a record-number database
Set custom sort order	$DB_BTREE->{'compare'} = \&Compare;	🔤 Override default BTREE key comparison
Enable duplicate keys (BTREE)	$DB_BTREE->{'flags'} = R_DUP;	🔁 Allow multiple values per key
Read duplicates with API	$x->seq($key, $value, R_FIRST/R_NEXT)	📖 Sequential access to all key/value pairs
Install DBM Filter	$db->filter_store_key( sub { ... } );	🔄 Transform keys/values on read/write
In-memory database	tie %h, 'DB_File', undef, ...	💾 Create a temporary database in memory

📋 SYNOPSIS

 use DB_File;

 [$X =] tie %hash,  'DB_File', [$filename, $flags, $mode, $DB_HASH] ;
 [$X =] tie %hash,  'DB_File', $filename, $flags, $mode, $DB_BTREE ;
 [$X =] tie @array, 'DB_File', $filename, $flags, $mode, $DB_RECNO ;

 $status = $X->del($key [, $flags]) ;
 $status = $X->put($key, $value [, $flags]) ;
 $status = $X->get($key, $value [, $flags]) ;
 $status = $X->seq($key, $value, $flags) ;
 $status = $X->sync([$flags]) ;
 $status = $X->fd ;

 # BTREE only
 $count = $X->get_dup($key) ;
 @list  = $X->get_dup($key) ;
 %list  = $X->get_dup($key, 1) ;
 $status = $X->find_dup($key, $value) ;
 $status = $X->del_dup($key, $value) ;

 # RECNO only
 $a = $X->length;
 $a = $X->pop ;
 $X->push(list);
 $a = $X->shift;
 $X->unshift(list);
 @r = $X->splice(offset, length, elements);

 # DBM Filters
 $old_filter = $db->filter_store_key  ( sub { ... } ) ;
 $old_filter = $db->filter_store_value( sub { ... } ) ;
 $old_filter = $db->filter_fetch_key  ( sub { ... } ) ;
 $old_filter = $db->filter_fetch_value( sub { ... } ) ;

 untie %hash ;
 untie @array ;

📖 DESCRIPTION

DB_File is a module which allows Perl programs to make use of the facilities provided by Berkeley DB version 1.x (if you have a newer version of DB, see Using DB_File with Berkeley DB version 2 or greater). It is assumed that you have a copy of the Berkeley DB manual pages at hand when reading this documentation. The interface defined here mirrors the Berkeley DB interface closely.

Berkeley DB is a C library which provides a consistent interface to a number of database formats. DB_File provides an interface to all three of the database types currently supported by Berkeley DB.

🗂️ File Types

• DB_HASH – This database type allows arbitrary key/value pairs to be stored in data files. Equivalent to DBM, NDBM, ODBM, GDBM, SDBM. Files are not compatible with other packages. A default hashing algorithm is built in; you can also provide your own Perl hashing routine.
• DB_BTREE – Stores key/value pairs in a sorted, balanced binary tree. Keys are sorted lexically by default; you can provide a custom comparison routine.
• DB_RECNO – Allows fixed-length and variable-length flat text files to be manipulated via key/value pairs (key = record number).

🔗 Using DB_File with Berkeley DB version 2 or greater

Although DB_File is intended for version 1, it can also be used with versions 2, 3, or 4. The interface is limited to version 1 functionality; differences are handled transparently. For new features, use the BerkeleyDB module. Note: database file format changed multiple times; dump existing databases with db_dump or db_dump185 before recreating with db_load. See "COPYRIGHT" before using version 2.x or greater.

🔌 Interface to Berkeley DB

DB_File allows access via Perl's tie() mechanism: associative array for DB_HASH & DB_BTREE, ordinary array for DB_RECNO. Additionally, most Berkeley DB API functions are available directly (see THE API INTERFACE).

🔓 Opening a Berkeley DB Database File

Berkeley DB uses dbopen() to open/create a database. In DB_File:

tie %array, 'DB_File', $filename, $flags, $mode, $DB_HASH;

The parameters filename, flags, mode correspond to dbopen(). The final parameter (e.g., $DB_HASH) serves as both the type and openinfo structure. Three predefined references exist: $DB_HASH, $DB_BTREE, $DB_RECNO. Their keys mirror the C structure fields. For example:

$DB_HASH->{'cachesize'} = 10000;

Constructors are available for custom instances:

$a = DB_File::HASHINFO->new();
$a->{'bsize'};
$a->{'cachesize'};
$a->{'ffactor'};
$a->{'hash'};
$a->{'lorder'};
$a->{'nelem'};

$b = DB_File::BTREEINFO->new();
$b->{'flags'};
$b->{'cachesize'};
$b->{'maxkeypage'};
$b->{'minkeypage'};
$b->{'psize'};
$b->{'compare'};
$b->{'prefix'};
$b->{'lorder'};

$c = DB_File::RECNOINFO->new();
$c->{'bval'};
$c->{'cachesize'};
$c->{'psize'};
$c->{'flags'};
$c->{'lorder'};
$c->{'reclen'};
$c->{'bfname'};

The keys hash, compare, prefix store references to Perl subs. Template subs:

sub hash {
    my ($data) = @_;
    return $hash;
}

sub compare {
    my ($key, $key2) = @_;
    return (-1, 0, 1);
}

sub prefix {
    my ($key, $key2) = @_;
    return $bytes;
}

⚙️ Default Parameters

It is possible to omit the final 4 parameters in tie. tie %A, "DB_File", "filename" is equivalent to tie %A, "DB_File", "filename", O_CREAT|O_RDWR, 0666, $DB_HASH. Omitting the filename as well creates an in-memory database.

💾 In Memory Databases

Use undef in place of the filename to create a database in memory.

📁 DB_HASH

The DB_HASH format is the most commonly used. Simple example:

use warnings ;
use strict ;
use DB_File ;
our (%h, $k, $v) ;

unlink "fruit" ;
tie %h, "DB_File", "fruit", O_RDWR|O_CREAT, 0666, $DB_HASH
    or die "Cannot open file 'fruit': $!\n";

$h{"apple"} = "red" ;
$h{"orange"} = "orange" ;
$h{"banana"} = "yellow" ;
$h{"tomato"} = "red" ;

print "Banana Exists\n\n" if $h{"banana"} ;
delete $h{"apple"} ;

while (($k, $v) = each %h)
  { print "$k -> $v\n" }

untie %h ;

Output:

Banana Exists

orange -> orange
tomato -> red
banana -> yellow

Keys appear in random order (like standard Perl hashes).

🌳 DB_BTREE

Useful for ordered storage. Default lexical order; custom sort possible.

🔤 Changing the BTREE sort order

Override the default comparison:

use warnings ;
use strict ;
use DB_File ;

my %h ;

sub Compare {
    my ($key1, $key2) = @_ ;
    "\L$key1" cmp "\L$key2" ;
}

$DB_BTREE->{'compare'} = \&Compare ;

unlink "tree" ;
tie %h, "DB_File", "tree", O_RDWR|O_CREAT, 0666, $DB_BTREE
    or die "Cannot open file 'tree': $!\n" ;

$h{'Wall'} = 'Larry' ;
$h{'Smith'} = 'John' ;
$h{'mouse'} = 'mickey' ;
$h{'duck'}  = 'donald' ;

delete $h{"duck"} ;

foreach (keys %h)
  { print "$_\n" }

untie %h ;

Output:

mouse
Smith
Wall

Points:

• 📌 The compare function must be specified at database creation.
• 📌 Cannot change ordering after creation; must use the same function every time.
• 📌 Duplicate keys are defined by the comparison function. Case-insensitive compare may treat 'KEY' and 'key' as duplicates. To avoid information loss, use a fully qualified compare if needed.

🔁 Handling Duplicate Keys

Enable with $DB_BTREE->{'flags'} = R_DUP. The tied hash interface can only read the first value for a key. Use the seq API method for full access:

use warnings ;
use strict ;
use DB_File ;

my ($filename, $x, %h, $status, $key, $value) ;

$filename = "tree" ;
unlink $filename ;

$DB_BTREE->{'flags'} = R_DUP ;

$x = tie %h, "DB_File", $filename, O_RDWR|O_CREAT, 0666, $DB_BTREE
    or die "Cannot open $filename: $!\n";

$h{'Wall'} = 'Larry' ;
$h{'Wall'} = 'Brick' ;
$h{'Wall'} = 'Brick' ;
$h{'Smith'} = 'John' ;
$h{'mouse'} = 'mickey' ;

$key = $value = 0 ;
for ($status = $x->seq($key, $value, R_FIRST) ;
     $status == 0 ;
     $status = $x->seq($key, $value, R_NEXT) )
  {  print "$key -> $value\n" }

undef $x ;
untie %h ;

Output:

Smith   -> John
Wall    -> Brick
Wall    -> Brick
Wall    -> Larry
mouse   -> mickey

📊 The get_dup() Method

$count = $x->get_dup($key) ;      # scalar: number of values
@list  = $x->get_dup($key) ;      # list: all values
%list  = $x->get_dup($key, 1) ;   # hash: value => count

Example:

my $cnt  = $x->get_dup("Wall") ;
print "Wall occurred $cnt times\n" ;

my %hash = $x->get_dup("Wall", 1) ;
print "Larry is there\n" if $hash{'Larry'} ;
print "There are $hash{'Brick'} Brick Walls\n" ;

my @list = sort $x->get_dup("Wall") ;
print "Wall =>      [@list]\n" ;

@list = $x->get_dup("Smith") ;
print "Smith =>     [@list]\n" ;

@list = $x->get_dup("Dog") ;
print "Dog =>       [@list]\n" ;

Output:

Wall occurred 3 times
Larry is there
There are 2 Brick Walls
Wall =>     [Brick Brick Larry]
Smith =>    [John]
Dog =>      []

🔍 The find_dup() Method

$status = $X->find_dup($key, $value) ;

Returns 0 if the key/value pair exists, non-zero otherwise. Cursor is left at the pair.

🗑️ The del_dup() Method

$status = $X->del_dup($key, $value) ;

Deletes a specific key/value pair; returns 0 on success.

🔍 Matching Partial Keys

Use seq with R_CURSOR to find the smallest key greater than or equal to the specified key (partial match).

use warnings ;
use strict ;
use DB_File ;
use Fcntl ;

my ($filename, $x, %h, $st, $key, $value) ;

sub match {
    my $key = shift ;
    my $value = 0;
    my $orig_key = $key ;
    $x->seq($key, $value, R_CURSOR) ;
    print "$orig_key\t-> $key\t-> $value\n" ;
}

$filename = "tree" ;
unlink $filename ;

$x = tie %h, "DB_File", $filename, O_RDWR|O_CREAT, 0666, $DB_BTREE
    or die "Cannot open $filename: $!\n";

$h{'mouse'} = 'mickey' ;
$h{'Wall'} = 'Larry' ;
$h{'Walls'} = 'Brick' ;
$h{'Smith'} = 'John' ;

$key = $value = 0 ;
print "IN ORDER\n" ;
for ($st = $x->seq($key, $value, R_FIRST) ;
     $st == 0 ;
     $st = $x->seq($key, $value, R_NEXT) )
  {  print "$key    -> $value\n" }

print "\nPARTIAL MATCH\n" ;
match "Wa" ;
match "A" ;
match "a" ;

undef $x ;
untie %h ;

Output:

IN ORDER
Smith -> John
Wall  -> Larry
Walls -> Brick
mouse -> mickey

PARTIAL MATCH
Wa -> Wall  -> Larry
A  -> Smith -> John
a  -> mouse -> mickey

📄 DB_RECNO

Interface to flat text files. Array offset starts at 0. Negative indexes allowed. The bval option marks the end of variable-length records (default "\n") or pad character for fixed-length (default space).

💡 The 'bval' Option

Discussed in Berkeley DB documentation. In DB_File, if you specify any openinfo options, you must explicitly set bval; otherwise it defaults to "\n" for variable-length, space for fixed-length. Only a single byte is allowed.

🔢 A Simple Example

use warnings ;
use strict ;
use DB_File ;

my $filename = "text" ;
unlink $filename ;

my @h ;
tie @h, "DB_File", $filename, O_RDWR|O_CREAT, 0666, $DB_RECNO
    or die "Cannot open file 'text': $!\n" ;

$h[0] = "orange" ;
$h[1] = "blue" ;
$h[2] = "yellow" ;

push @h, "green", "black" ;

my $elements = scalar @h ;
print "The array contains $elements entries\n" ;

my $last = pop @h ;
print "popped $last\n" ;

unshift @h, "white" ;
my $first = shift @h ;
print "shifted $first\n" ;

print "Element 1 Exists with value $h[1]\n" if $h[1] ;
print "The last element is $h[-1]\n" ;
print "The 2nd last element is $h[-2]\n" ;

untie @h ;

Output:

The array contains 5 entries
popped black
shifted white
Element 1 Exists with value blue
The last element is green
The 2nd last element is yellow

🔧 Extra RECNO Methods

For older Perl versions that lack tied array methods, DB_File provides:

• $X->push(list) – push elements to end
• $value = $X->pop – pop last element
• $X->shift – shift first element
• $X->unshift(list) – unshift elements to start
• $X->length – number of elements
• $X->splice(offset, length, elements) – splice array

📝 Another Example

use warnings ;
use strict ;
my (@h, $H, $file, $i) ;
use DB_File ;
use Fcntl ;

$file = "text" ;
unlink $file ;

$H = tie @h, "DB_File", $file, O_RDWR|O_CREAT, 0666, $DB_RECNO
    or die "Cannot open file $file: $!\n" ;

$h[0] = "zero" ;
$h[1] = "one" ;
$h[2] = "two" ;
$h[3] = "three" ;
$h[4] = "four" ;

print "\nORIGINAL\n" ;
foreach $i (0 .. $H->length - 1) {
    print "$i: $h[$i]\n" ;
}

$a = $H->pop ;
$H->push("last") ;
print "\nThe last record was [$a]\n" ;

$a = $H->shift ;
$H->unshift("first") ;
print "The first record was [$a]\n" ;

$i = 2 ;
$H->put($i, "Newbie", R_IAFTER) ;

$i = 1 ;
$H->put($i, "New One", R_IBEFORE) ;

$H->del(3) ;

print "\nREVERSE\n" ;
for ($i = $H->length - 1 ; $i >= 0 ; -- $i)
  { print "$i: $h[$i]\n" }

print "\nREVERSE again\n" ;
my ($s, $k, $v)  = (0, 0, 0) ;
for ($s = $H->seq($k, $v, R_LAST) ;
         $s == 0 ;
         $s = $H->seq($k, $v, R_PREV))
  { print "$k: $v\n" }

undef $H ;
untie @h ;

Output:

ORIGINAL
0: zero
1: one
2: two
3: three
4: four

The last record was [four]
The first record was [zero]

REVERSE
5: last
4: three
3: Newbie
2: one
1: New One
0: first

REVERSE again
5: last
4: three
3: Newbie
2: one
1: New One
0: first

Notes:

• 📌 Iterate with 0 .. $H->length - 1 or seq API.
• 📌 put returns the record number via the parameter.

🔌 THE API INTERFACE

Besides tied access, you can call Berkeley DB API functions directly on the object returned by tie:

$db = tie %hash, "DB_File", "filename" ;
$db->put($key, $value, R_NOOVERWRITE) ;

Important: The database file stays open until both the tied variable is untied and the saved object is destroyed (see The untie() Gotcha).

All dbopen functions are available except close() and dbopen() itself. Methods return 0 on success, -1 on error (with $! set), 1 if key not found. Flags constants are defined.

📚 Available Methods

• $status = $X->get($key, $value [, $flags]) – Read value for key. No flags currently defined.
• $status = $X->put($key, $value [, $flags]) – Store key/value. Valid flags: R_CURSOR, R_IAFTER, R_IBEFORE, R_NOOVERWRITE, R_SETCURSOR. With R_IAFTER or R_IBEFORE, $key is set to the record number.
• $status = $X->del($key [, $flags]) – Remove all pairs with key $key. Returns 1 if key not found. Valid flag: R_CURSOR.
• $status = $X->fd – Returns file descriptor. See Locking: The Trouble with fd.
• $status = $X->seq($key, $value, $flags) – Sequential retrieval. $flags mandatory: R_CURSOR, R_FIRST, R_LAST, R_NEXT, R_PREV.
• $status = $X->sync([$flags]) – Flush cached buffers. Valid flag: R_RECNOSYNC.

🔄 DBM FILTERS

Transform keys/values consistently. Two ways: low-level API or the DBM_Filter module (recommended).

🔧 DBM Filter Low-level API

Four methods: filter_store_key, filter_store_value, filter_fetch_key, filter_fetch_value. Each takes a sub reference. The filter modifies $_. Returns the existing filter or undef. Pass undef to delete.

📝 Example: NULL termination problem

use warnings ;
use strict ;
use DB_File ;

my %hash ;
my $filename = "filt" ;
unlink $filename ;

my $db = tie %hash, 'DB_File', $filename, O_CREAT|O_RDWR, 0666, $DB_HASH
  or die "Cannot open $filename: $!\n" ;

$db->filter_fetch_key  ( sub { s/\0$//    } ) ;
$db->filter_store_key  ( sub { $_ .= "\0" } ) ;
$db->filter_fetch_value( sub { s/\0$//    } ) ;
$db->filter_store_value( sub { $_ .= "\0" } ) ;

$hash{"abc"} = "def" ;
my $a = $hash{"ABC"} ;
# ...
undef $db ;
untie %hash ;

🔢 Example: Key is a C int

use warnings ;
use strict ;
use DB_File ;
my %hash ;
my $filename = "filt" ;
unlink $filename ;

my $db = tie %hash, 'DB_File', $filename, O_CREAT|O_RDWR, 0666, $DB_HASH
  or die "Cannot open $filename: $!\n" ;

$db->filter_fetch_key  ( sub { $_ = unpack("i", $_) } ) ;
$db->filter_store_key  ( sub { $_ = pack ("i", $_) } ) ;
$hash{123} = "def" ;
# ...
undef $db ;
untie %hash ;

💡 HINTS AND TIPS

🔒 Locking: The Trouble with fd

Using flock on the file descriptor from fd is flawed. Two processes can cache inconsistent initial blocks. Avoid this technique. Instead, use BerkeleyDB module for internal locking, or one of these CPAN wrappers:

• Tie::DB_Lock – Multiversioning concurrent reads, serial updates.
• Tie::DB_LockFile – Re-ties on lock acquire/release.
• DB_File::Lock – Lightweight flock on a lockfile.

🤝 Sharing Databases With C Applications

C strings are NULL-terminated; Perl strings are not. Use DBM Filters to add/remove NULL terminators. Example: Netscape history.db (key = URL with NULL, value = 4-byte binary time).

use warnings ;
use strict ;
use DB_File ;
use Fcntl ;

my ($dotdir, $HISTORY, %hist_db, $href, $binary_time, $date) ;
$dotdir = $ENV{HOME} || $ENV{LOGNAME};
$HISTORY = "$dotdir/.netscape/history.db";

tie %hist_db, 'DB_File', $HISTORY
    or die "Cannot open $HISTORY: $!\n" ;;

while ( ($href, $binary_time) = each %hist_db ) {
    $href =~ s/\x00$// ;
    $date = localtime unpack("V", $binary_time);
    print "$date $href\n" ;
}

if ( $binary_time = $hist_db{"<a href="http://mox.perl.com/">http://mox.perl.com/</a>\x00"} ) {
    $date = localtime unpack("V", $binary_time) ;
    print "Last visited mox.perl.com on $date\n" ;
}
else {
    print "Never visited mox.perl.com\n"
}

untie %hist_db ;

⚠️ The untie() Gotcha

If you save the object from tie, the database file remains open until both the tied variable and the saved object are destroyed. Always undef $X before untie.

use DB_File ;
use Fcntl ;

my %x ;
my $X ;

$X = tie %x, 'DB_File', 'tst.fil' , O_RDWR|O_TRUNC
    or die "Cannot tie first time: $!" ;

$x{123} = 456 ;

undef $X ;   # must destroy before untie
untie %x ;

$X = tie %x, 'DB_File', 'tst.fil' , O_RDWR|O_CREAT
    or die "Cannot tie second time: $!" ;
...

❓ COMMON QUESTIONS

🤔 Why is there Perl source in my database?

Berkeley DB uses dynamic memory for buffers; uninitialized memory may contain random junk (including parts of Perl scripts). Nothing to worry about.

🏗️ How do I store complex data structures with DB_File?

Use the MLDBM module, which layers over DB_File.

🌐 What does "wide character in subroutine entry" mean?

Occurs with UTF-8 data. Use the utf8 DBM_Filter from DBM_Filter:

use DB_File;
use DBM_Filter;

my $db = tie %h, 'DB_File', '/tmp/try.db', O_CREAT|O_RDWR, 0666, $DB_BTREE;
$db->Filter_Key_Push('utf8');
$db->Filter_Value_Push('utf8');

my $key = "\N{LATIN SMALL LETTER A WITH ACUTE}";
my $value = "\N{LATIN SMALL LETTER E WITH ACUTE}";
$h{ $key } = $value;

❓ What does "Invalid Argument" mean?

Usually a wrong parameter in tie. Common causes: reopening a database without closing it, or using O_WRONLY.

🚫 What does "Bareword 'DB_File' not allowed" mean?

When use strict is active, you must quote the module name: tie %x, "DB_File", "filename".

📚 REFERENCES

• "Full-Text Searching in Perl", Tim Kientzle, Dr. Dobb's Journal, Issue 295, January 1999, pp 34-41

📜 HISTORY

Moved to the Changes file.

🐛 BUGS

Some older Berkeley DB versions had problems with fixed-length records in RECNO. That has been fixed since version 1.85. Report bugs to GitHub issues or RT.

🤝 SUPPORT

Send feedback, questions, bug reports to GitHub (preferred) or CPAN RT.

📦 AVAILABILITY

DB_File comes with standard Perl distribution (in ext/DB_File). Latest version on CPAN. Designed for Berkeley DB v1; for v2+ use BerkeleyDB. Official web site: Oracle. Version 1 available at CPAN src/misc/db.1.85.tar.gz.

©️ COPYRIGHT

Copyright (c) 1995-2020 Paul Marquess. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. Berkeley DB has its own license; see the FAQ at Oracle for details. In short, the license requires that software using Berkeley DB be freely redistributable, but your Perl scripts are your property.

👀 SEE ALSO

perl, dbopen(3), hash(3), recno(3), btree(3), perldbmfilter, DBM_Filter

✍️ AUTHOR

The DB_File interface was written by Paul Marquess <pmqs@cpan.org>.

🏷️ NAME 🚀 Quick Reference 📋 SYNOPSIS 📖 DESCRIPTION 📁 DB_HASH 🌳 DB_BTREE 📄 DB_RECNO 🔌 THE API INTERFACE 🔄 DBM FILTERS 💡 HINTS AND TIPS ❓ COMMON QUESTIONS 📚 REFERENCES 📜 HISTORY 🐛 BUGS 🤝 SUPPORT 📦 AVAILABILITY ©️ COPYRIGHT 👀 SEE ALSO ✍️ AUTHOR