LNATION

Modules · Concurrency · 0.16

Shared::Arena

Memory two processes can both read, without a syscall.

Two processes that want to share a structure have, in Perl, a pipe, a socket, a file or a database, each a syscall and a copy per message. Shared::Arena is a region of memory mapped once, carved into named sub-regions, and read by every process that maps it without copying, locking or calling into the kernel.

On MetaCPAN 

use Shared::Arena;

# in the parent, before the fork
my $arena  = Shared::Arena->create(size => 1024 * 1024);
my $ring   = $arena->ring('events', slots => 64, slot_size => 256);
my $cursor = $ring->cursor;

my $pid = fork() // die "fork: $!";
if ($pid == 0) {
    $ring->publish('hello', 'from the child');   # no pipe, no socket
    exit 0;
}
waitpid $pid, 0;

for my $rec ($cursor->drain) {
    my ($topic, $payload, $seq) = @$rec;
    print "$topic: $payload\n";                  # hello: from the child
}

What you get

  • One region, no syscall

    Mapped once and carved into named sub-regions. Every process that maps it reads it directly, by name with attach or as a child after a fork.

  • Twelve structures

    Ring, Map, Cache, Bloom, Cuckoo, CountMin, HyperLogLog, Histogram, Rate, Frozen, Lease and Scoreboard, each living inside the arena.

  • Counters in one instruction

    A map's counters increment in a single atomic instruction, so a rate limit across a pool of workers is the limit you wrote, not that limit times the workers.

  • Structures, through Struct::Codec

    A map or a cache made with serialise => 1 flattens nested data going in and rebuilds it coming out, through Struct::Codec.

  • Leader election and a scoreboard

    A Lease has one holder, and a successor when it dies. A Scoreboard gives every worker a row it writes with no lock and a supervisor reads in one pass.

In use

use Shared::Arena;

my $arena = Shared::Arena->create(size => 1024 * 1024);
my $map   = $arena->map('limits', slots => 256, slot_size => 64);

# a counter is one atomic instruction, so every worker shares the one count
my @kids = map {
    my $pid = fork() // die "fork: $!";
    if ($pid == 0) { $map->incr('rate:10.0.0.1') for 1 .. 100; exit 0 }
    $pid;
} 1 .. 4;
waitpid $_, 0 for @kids;

print $map->incr('rate:10.0.0.1', 0), "\n";      # 400

Get it

cpanm Shared::Arena

Shared::Arena on MetaCPAN Every module