File Coverage

blib/lib/Net/Async/Graphite/API.pm

Criterion	Covered	Total	%
statement	50	64	78.1
branch	5	8	62.5
condition	0	6	0.0
subroutine	15	19	78.9
pod	5	5	100.0
total	75	102	73.5

package Net::Async::Graphite::API;

our $VERSION = '0.1_1';

=head1 NAME

Net::Async::Graphite::API - Interface between perl and graphite.

=head1 SYNOPSIS

    with 'Net::Async::Graphite::API';

=head1 DESCRIPTION

Don't use this module directly, use L and create
objects using its C method in the normal way. Those objects will
include the functionality documented here.

This role brings the capacity to transmit HTTP requests to graphite
and asynchronously wait for and return the response.

It also defines a structure describing complex target paths in a perl
data structure.

=head1 BUGS

Assumes graphite-api and doesn't handle transport errors at all.

Using Memoize may be pointles.

=cut

use v5.14;
use strictures 2;
use Moo::Role;
use Carp;

use Memoize      qw(memoize);
use PerlX::Maybe qw(maybe);
use URI;
use namespace::clean;

=head1 ROLE

Asynchronity is achieved by using L inside
L.

JSON data is decoded using a L object handled by
L.

=cut

with 'Net::Async::Graphite::HTTPClient';

with 'Net::Async::Graphite::JSONCodec';

=head1 ATTRIBUTES

=over

=item endpoint (required, read-only)

The base of the graphite URL to which C and C
requests will be directed. Everything up to and B the
trailing C. eg. C.

If the trailing C is not included it's assumed you know what you're
doing so only a warning will be printed.

=cut

has endpoint => (
  is       => 'ro',
  isa      => sub {
    carp 'endpoint should include a trailing "/", attempting to continue'
      unless $_[0] =~ /\/$/;
    croak 'endpoint must include a protocol' unless $_[0] =~ /^[a-z-]+:/i;
  },
  required => 1,
);

=back

=head1 PUBLIC METHODS

All of these methods return a L which will complete with the
indicated result (or which will die).

=over

=item metrics ($method, $query, [%extra])

Perform C<$method> on/with the metrics which match C<$query>, which
should be a simple scalar describing the query to perform, or an
arrayref of such scalars if the method can handle multiple query
parameters. ie. this will request a URI which looks like:
C<< /metrics/<$method>?query=<$query>[&query=...] >>. If
arguments are supplied in C<%extra>, they are added to the generated
URI as encoded query parameters.

The three methods which graphite-api exposes at this time are supported:

=over

=item find

=item expand

=item index[.json]

C can be requested with C<$method> of C or
C because requiring the C<.json> suffix is silly. Also I
can consider creating real methods C, C and C.

=back

The L will complete with the http content. If you want the
HTTP request object see the C<_download> private method, but note
I.

=cut

my %allowed_metrics_params = (
  expand => [qw(groupByExpr leavesOnly)],
  find   => [qw(wildcards from until position)],
  index  => [],
);
$allowed_metrics_params{'index.json'} = $allowed_metrics_params{index}; # FFS

sub metrics {
  my $self = shift;
  my ($method, $query, %extra) = @_;
  $method = 'index.json' if $method eq 'index';
  return Future->fail("Invalid metrics method: $method")
    unless exists $allowed_metrics_params{$method};
  my $uri = URI->new($self->endpoint . "metrics/$method");
  my %query = (
    maybe query => $query, # No query in index
    map { maybe $_ => $extra{$_} } @{ $allowed_metrics_params{$method} },
  );
  # Only if .../find. Is default_* stupid?
  $query{from}  ||= $self->default_from  if $self->has_default_from;
  $query{until} ||= $self->default_until if $self->has_default_until;
  $uri->query_form(%query);
  $self->_download($uri)->then(sub { Future->done($_[0]->content) });
}

=item metrics_asperl ($method, $query, [%extra])

Calls C with the same arguments and decodes the result,
which is assumed to be JSON text, into a perl data structure.

=cut

sub metrics_asperl {
  my $self = shift;
  $self->metrics(@_)->then(sub { Future->done($self->_json->decode(shift)); });
}

# L. ie. this

=item render ($format, $target, [%extra])

Fetch the metric data for the given C<$target>, which should be a
simple scalar specifying a path identifying metrics, or an arrayref of
such scalars, as described in the
L<< graphite C documentation | http://graphite.readthedocs.io/en/latest/render_api.html >>
C will then request a URI which looks like:
C<< /render?format=<$format>&target=<$query>[&target=...] >>.

The name of the format may also be called as a method directly,
without the first (C<$format>) argument:

=over

=item csv ($target, [%extra])

=item dygraph ($target, [%extra])

=item json ($target, [%extra])

=item pdf ($target, [%extra])

=item png ($target, [%extra])

=item raw ($target, [%extra])

=item rickshaw ($target, [%extra])

=item svg ($target, [%extra])

=back

=cut

sub render {
  my $self = shift;
  my ($format, $target, %extra) = @_;
  my $uri = URI->new($self->endpoint . 'render');
  my %query = (
    format => $format,
    target => $target,
    map { maybe $_ => $extra{$_} } qw(graphType jsonp maxDataPoints noNullPoints), # Many more
  );
  $uri->query_form(%query);
  $self->_download($uri)->then(sub { Future->done($_[0]->content) });
}

for my $format (qw(
  csv
  dygraph
  json
  pdf
  png
  raw
  rickshaw
  svg
)) {
  no strict 'refs';
  *{$format} = sub { $_[0]->render($format => @_[1..$#_]) };
}

=item render_asperl ($target, [%extra])

Calls C with the same arguments but prepended by the format
C, and decodes the result.

Graphite's raw format is documented in
L<< graphite C documentation | http://graphite.readthedocs.io/en/latest/render_api.html >>
but in short it's an ASCII (or UTF-8 if you're so inclined) encoded
file consisting of one line per metric found. Each line consists of a
header and the data separated by a C<|>. Within each of these each
datum is separated by a C<,>.

I don't know or care if the line endings include C<\r>.

The data is returned as a list of hashrefs containing the data broken
up into B of its components. ie. Each datum of each metric will
be turned into its own perl scalar.

Each hashref will contain five values:

=over

=item target

The path of metric.

=item start

Unix timestamp pinpointing the beginning of the data.

=item end

Unix timestamp pinpointing the end of the data.

=item step

The time interval in seconds (probably) between each datum.

=item data

The data. An arrayref of however many scalars it takes.

The data are B converted from their original text form, and
graphite include the string C in the list of returned values, so
not every datum will necessarily look like a number.

=back

=cut

sub render_asperl {
  my $self = shift;
  $self->render(raw => @_)->then(sub {
    my @sets = map {
      chomp;
      my ($head, $data) = split /\|/;
      my %set;
      (@set{qw(target start end step)}) = split /,/, $head;
      $set{data} = [ split /,/, $data ];
      \%set;
    } split /\n/, $_[0];
    Future->done(@sets);
  });
}

=item find_target_from_spec ($spec, ...)

Construct strings for use by the C parameter of C
queries from the specifications in each C<$spec>, which must be a
plain text scalar of the metric's path or an arrayref containing 2
items (second optional):

=over

=item $function

The name of a function as defined by the graphite API.

=item [ @arguments ] (optional)

An arrayref containing 0 or more arguments. Each argument may be a
scalar, which left alone, or an arrayref which must itself be another
C<$spec>-like scalar/pair.

=back

It sounds a lot more confusing than it is. These are valid:

    # Scalar

    'simple.metric.name'

    # Becomes "simple.metric.name"


    # Function

    [ 'summarize' => [ 'simple.metric.name' ] ]

    # Becomes "summarize(simple.metric.name)"


    # Function with more arguments

    [ 'summarize' => [ 'me.tr.ic', '"arg1"', 42 ] ]

    # Becomes "summarize(me.tr.ic,%52arg1%52,42)"


    # Recursive function

    [
        'summarize' => [
            'randomize' => [
                'pointless.metric', 'argue', 'bicker'
            ],
            'and-fight',
        ]
    ]

    # Becomes "summarize(randomize(pointless.metric,argue,bicker),and-fight)"


Any other form is not.

Simply put, where it's not just a plain string the specification is a
recursive nest of function/argument-list pairs, where each argument
can itself be another function pair.

The implementation of the target string construction uses L
to avoid repeatedly calling functions which return the same data. It
probably destroys any advantage gained by doing this by normalising
the function arguments with JSON first. I'll measure it some day.

=cut

sub find_target_from_spec {
  my $self = shift;
  return map { __construct_target_argument($_) } @_;
}

=back

=head1 PRIVATE METHODS

=over

=item __construct_target_argument ($argument)

This is not an object or class method.

Compile C<$argument> into its part of the string which will build up
the (or a) target component of a Graphite request URI. C<$argument>
must be a scalar or an arrayref with exactly one or two items in
it. If included, the second must be an arrayref of the function's
arguments.

Returns the scalar as-is or calls C<__construct_target_function> to
decode the arrayref.

This function uses L for which it normalises its arguments
using L.

=cut

memoize('__construct_target_argument',
        NORMALIZER => sub { ref($_[0]) ? encode_json($_[0]) : $_[0] });

sub __construct_target_argument {
  my ($argument) = @_;
  my $ref = ref $argument;
  if (not $ref) {
    $argument;
  } elsif ($ref eq 'ARRAY' and scalar @$argument <= 2) {
    croak 'Function arguments must be an arrayref'
      unless not defined $argument->[1] or ref $argument->[1] eq 'ARRAY';
    __construct_target_function($argument->[0], @{ $argument->[1] || [] });
  } else {
    croak "Unknown argument reference type: $ref";
  }
}

=item __construct_target_function ($name, [@arguments])

This is not an object or class method.

Compile C<$name> and C<@arguments> into their part of the string which
will build up the target component of a Graphite request URI. Recurses
back into C<__construct_target_argument> to build each argument
component.

This function uses L for which it normalises its arguments
using L.

=cut

memoize('__construct_target_function',
        NORMALIZER => sub { encode_json({@_}) });

sub __construct_target_function {
  my $name = shift;
  croak 'Function name must be a scalar' if ref $name;
  my @arguments = map { __construct_target_argument($_) } @_;
  $name . '(' . join(',', @arguments) . ')';
}

1;

=back

=head1 SEE ALSO

Graphite's C documentation L

L

L

L

L

L

L

=head1 AUTHOR

Matthew King

=cut