File Coverage

blib/lib/WARC/Index.pm

Criterion	Covered	Total	%
statement	54	54	100.0
branch	21	22	95.4
condition	3	3	100.0
subroutine	10	10	100.0
pod	6	6	100.0
total	94	95	98.9

package WARC::Index;						# -*- CPerl -*-

use strict;
use warnings;

use Carp;

our @ISA = qw();

require WARC; *WARC::Index::VERSION = \$WARC::VERSION;

=head1 NAME

WARC::Index - base class for WARC index classes

=head1 SYNOPSIS

  use WARC::Index::File::CDX;	# or ...
  use WARC::Index::File::SDBM;
  # or some other WARC::Index::File::* implementation

  $index = attach WARC::Index::File::CDX (...);	# or ...
  $index = attach WARC::Index::File::SDBM (...);

  $record = $index->search(url => $url, time => $when);
  @records = $index->search(url => $url, time => $when);

  build WARC::Index::File::CDX (...);	# or ...
  build WARC::Index::File::SDBM (...);

=head1 DESCRIPTION

C is an abstract base class for indexes on WARC files and
WARC-alike files.  This class establishes the expected interface and
provides a simple interface for building indexes.

=head2 Methods

=over

=item $index = attach WARC::Index::File::* (...)

Construct an index object using the indicated technology and whatever
parameters the index implementation needs.

Typically, indexes are file-based and a single parameter is the name of an
index file which in turn contains the names of the indexed WARC files.

=cut

sub attach {
  die __PACKAGE__." is an abstract base class and "
    .(shift)." must override the 'attach' method"
}

=item $yes_or_no = $index-Esearchable( $key )

Return true or false to reflect if the index can search for the requested
key.  Indexes may be able to search for keys that are not present in
entries returned from those indexes.

See the L<"Search Keys" section|WARC::Collection/"Search Keys"> of the
C page for details on the implemented search keys.

=cut

sub searchable {
  die __PACKAGE__." is an abstract base class and "
    .(ref shift)." must override the 'searchable' method"
}

=item $record = $index-Esearch( ... )

=item @records = $index-Esearch( ... )

Search an index for records matching parameters.  The C
class uses this method to search each index in a collection.

If the none of the requested search keys are searchable, returns an
undefined value in scalar context and the empty list in list context.

The details of the parameters for this method are documented in the
L<"Search Keys" section|WARC::Collection/"Search Keys"> of the
C page.

=cut

sub search {
  die __PACKAGE__." is an abstract base class and "
    .(ref shift)." must override the 'search' method"
}

=item build WARC::Index::File::* (into =E $dest, from =E ...)

=item build WARC::Index::File::* (from =E [...], into =E $dest)

The C base class B provide this method, however.  The
C method works by loading the corresponding index builder class and
driving the process or simply returning the newly-constructed object.

The C method itself handles the C key for specifying the files
to index.  The C key can be given an array reference, after which
more key =E value pairs may follow, or can simply use the rest of the
argument list as its value.

If the C key is given, the C method will read the indicated
files, construct an index, and return nothing.  If the C key is not
given, the C method will construct and return an index builder.

All index builders accept at least the C key for specifying where to
store the index.  See the documentation for WARC::Index::File::*::Builder
for more information.

=cut

sub build {
  my $class = shift;

  croak "'build' is a class method"
    if ref $class;
  croak "no arguments given to 'build' class method"
    unless scalar @_;

  my @args = (); my $from = undef;
  while (@_) {
    my $key = shift;
    if ($key eq 'from') {
      if (UNIVERSAL::isa($_[0], 'ARRAY'))	{ $from = shift }
      else					{ $from = [splice @_] }
    } else { push @args, $key, shift }
  }

  croak "empty list of index sources given"
    if defined $from && scalar @$from == 0;

  my $bclass = $class . q{::Builder};
  {
    no strict 'refs';
    unless (exists ${$class.'::'}{'Builder::'})
      { eval q{require }.$bclass; die $@ if $@ }
  }

  my $ob = _new $bclass (@args);

  return $ob unless defined $from;

  $ob->add(@$from);
  return ();
}

=back

=head2 Optional Methods

Some index systems may also provide these methods:

=over

=item $entry = $index-Efirst_entry

An index that has a sequential ordering may provide this method to obtain
the first entry in the index.  Indexes that do not have a meaningful
sequence amongst their entries do not provide this method.

=item $entry = $index-Eentry_at( $position )

An index that has a sequential ordering may provide this method to obtain
an entry at a specified position in the index.  The exact format of the
position parameter is not specified in general, but should be a value
previously obtained from the C method on an entry from the
same index.  Valid positions may be sparse.

=back

=head2 Index system registration

The C package also provides a registry of loaded index
support.  The C function adds the calling package to the list.

=cut

# Array of arrays listing index implementations and filename patterns.
#  Each element:  [ Package => qr/pattern1/, qr/pattern2/, ... ]
our @Index_Handlers = ();

=over

=item WARC::Index::register( filename =E $filename_re )

Add the calling package to an internal list of available index handlers.
The calling package must be a subclass of C or this function
will croak().

The C key indicates that the calling package expects to handle
index files with names matching the provided regex.

=cut

sub register {
  my %opt = @_;
  my $caller = scalar caller;

  croak "WARC::Index implementations must subclass WARC::Index"
    unless $caller->isa('WARC::Index');

  croak "WARC::Index implementations must handle a filename pattern"
    unless $opt{filename};

  foreach my $row (grep {$_->[0] eq $caller} @Index_Handlers) {
    push @$row, $opt{filename};	# add pattern to existing row
    return # ensure that there will be at most one row per package
  }
  push @Index_Handlers, [$caller => $opt{filename}];

  return # nothing
}

=item WARC::Index::find_handler( $filename )

Return the registered handler for $filename or undef if none match.  If
multiple handlers match, which one is returned is unspecified.

=cut

sub find_handler {
  my $filename = shift;
  my @match = grep {grep {$filename =~ $_} @$_[1..$#$_]} @Index_Handlers;
  return undef unless @match;
  return $match[0][0];
}

=back

=cut

1;
__END__