LNATION

File::Raw 0.19 · files for Perl, in C

Files through system calls, not PerlIO.

Reading and writing files with read(2) and write(2), not PerlIO: buffers sized from the file before the first byte, lines walked without the file in memory, memory-mapped access, and one stat for the size, times and type. And a plugin pipeline the other seven dists plug into.

The reference The plugins On CPAN 

use File::Raw qw(slurp spew lines);

my $config = file_slurp('app.conf');
my $lines  = file_lines('access.log');

file_spew('first-ten.log', join '', map "$_\n", @$lines[0 .. 9]);

What the core does

Reading and writing, a line at a time or all at once, mapping and stat-ing: about fifty functions, each one a system call or two in C.

  • Slurp and spew

    slurp reads a file into one buffer sized from its length before the first byte, and spew and append write it out. atomic_spew writes a temporary file and renames it, so a reader never sees half a file.

  • Lines without the whole file

    each_line calls your sub once a line with the line in $_, through MULTICALL. lines_iter is the pull shape, for an early exit, and head, tail and range_lines return part of a file.

  • Chunks for your own parser

    chunk_iter hands out the file 64 KiB at a time, as the caller asks: the loop belongs to your parser or state machine, not to File::Raw.

  • Memory-mapped

    mmap_open maps a file and data gives zero-copy access to it.

  • One stat call

    stat returns the size, the times, the mode and the type from one system call, and results are cached until clear_stat_cache. size, mtime and the is_* tests are there one at a time too.

  • Copy, move, make, remove

    copy uses copyfile() on macOS and sendfile() on Linux. move, touch, mkpath, rm_rf, readdir, and the path helpers basename, dirname, extname and join.

  • Search the lines

    grep_lines, count_lines, find_line and map_lines, with named predicates (is_blank, or your own through register_predicate).

  • Custom ops when imported

    use File::Raw qw(slurp spew) installs file_slurp and file_spew as custom ops, without the cost of a sub call.

use File::Raw ();

# A line at a time; the file is never in memory whole.
File::Raw::each_line('access.log', sub {
    print "$_\n" if / 500 /;
});

# Or pull lines, and stop when there are enough.
my $iter = File::Raw::lines_iter('access.log');
until ($iter->eof) {
    my $line = $iter->next;
    last if $line =~ /^#/;
}
$iter->close;

# Size, mtime and type from one stat, not three.
my $st = File::Raw::stat('access.log');
printf "%d bytes, modified %d\n", @{$st}{qw(size mtime)};

One call, a named plugin

A plugin is what File::Raw does to the bytes on their way in or out. Load the dist and name its plugin on the call: slurp with plugin => 'json' returns a structure, spew with it writes one, and each_line hands over a record at a time.

Plugins chain. plugin => ['gzip', 'csv'] inflates and then parses, so a .csv.gz is rows in one call, and writes the other way round with the same spelling. Each plugin in a chain takes its own options.

Each plugin is up to four callbacks in C, one for each of the read, write, record and stream phases, registered when its module loads. Nothing is converted to Perl in between.

use File::Raw qw(slurp);
use File::Raw::Gzip;
use File::Raw::Separated;

# Gunzip, then parse: one read, two plugins, rows out.
my $rows = file_slurp('metrics.csv.gz',
    plugin => ['gzip', 'csv'],
    csv    => { header => 1 },     # each plugin its own options
);

print "$_->{host}: $_->{p99}\n" for @$rows;

The family

Seven dists on CPAN, each a plugin or a container for them.

The numbers

Each against the fastest alternative that was installed, measured, and shown whichever way it went.

  • 1.3x read a 4 KB file into a scalar, against PerlIO File::Raw 0.19 at 129865/s, PerlIO 1.12 at 101789/s
  • 1.2x read a 1 MB file into a scalar, against Path::Tiny File::Raw 0.19 at 33600/s, Path::Tiny 0.150 at 28174/s
  • 1.0x read a 64 MB file into a scalar, against Path::Tiny File::Raw 0.19 at 250/s, Path::Tiny 0.150 at 246/s
  • 1.9x read a 1 MB file as a list of lines, against Path::Tiny File::Raw 0.19 at 1879/s, Path::Tiny 0.150 at 966/s
  • 2.6x call a sub for every line of a 1 MB file, against PerlIO File::Raw 0.19 at 3940/s, PerlIO 1.12 at 1541/s
  • 2.4x write a 1 MB scalar to a file, against Path::Tiny File::Raw 0.19 at 11671/s, Path::Tiny 0.150 at 4947/s
  • 1.6x copy a 64 MB file, against File::Copy File::Raw 0.19 at 178/s, File::Copy 2.41 at 112/s
  • 1.0x File::Raw::JSON: decode a 0.8 MB document, against JSON::XS File::Raw::JSON 0.08 at 315/s, JSON::XS 4.04 at 300/s
  • 2.4x File::Raw::XML: parse a 1.0 MB document to a tree, against XML::LibXML File::Raw::XML 0.04 at 225/s, XML::LibXML 2.0213 at 93.0/s
  • 3.4x File::Raw::Separated: read a 6.8 MB CSV file into rows, against Text::CSV_XS File::Raw::Separated 0.04 at 23.2/s, Text::CSV_XS 1.62 at 6.8/s
  • 6.8x File::Raw::Gzip: read a gzipped 7.5 MB log, against IO::Uncompress::Gunzip File::Raw::Gzip 0.03 at 313/s, IO::Uncompress::Gunzip 2.213 at 45.9/s
  • 2.0x File::Raw::Base64: read a base64 file holding 1 MB of bytes, against MIME::Base64 File::Raw::Base64 0.05 at 2410/s, MIME::Base64 3.16_01 at 1188/s
  • 3.7x File::Raw::Hash: sha256 of a 64 MB file, against Digest::SHA File::Raw::Hash 0.07 at 25.6/s, Digest::SHA 6.04 at 6.9/s
  • 16.4x File::Raw::Archive: list a 100-file tar, against Archive::Tar File::Raw::Archive 0.04 at 6100/s, Archive::Tar 3.04 at 371/s

Measured on Apple M5, darwin with perl 5.42.0, 2026-10-19: each entry timed for 3 CPU seconds by the Benchmark module, 3 times, the median kept, after checking that both sides produced the same result. Rates are per CPU second. The scripts are in the site's bench/file-raw/, and yours will differ.

For C, too

File::Raw publishes file_plugin.h: a plugin in another XS dist is a struct of callbacks and one file_register_plugin call at BOOT. That is how the seven plugins here are built.

JSON, XML and Hash publish tables of their own (frj_abi.h, frx_abi.h, frh_abi.h), so another XS module calls the parser or the digest directly, with no Perl in between. This is JSON's own example: a Hyperman handler decoding a request body.

#include "frj_abi.h"   /* found via ExtUtils::Depends, not copied */

static const frj_abi *FRJ = NULL;

MODULE = My::Consumer   PACKAGE = My::Consumer

BOOT:
{
    IV p = 0; dSP;
    eval_pv("require File::Raw::JSON;", FALSE);
    PUSHMARK(SP); PUTBACK;
    if (call_pv("File::Raw::JSON::_abi_ptr", G_SCALAR|G_EVAL) > 0) {
        SPAGAIN; p = POPi; PUTBACK;
    }
    if (p) {
        const frj_abi *a = INT2PTR(const frj_abi *, p);
        if (a && a->abi_version == FRJ_ABI_VERSION) FRJ = a;
    }
    /* FRJ == NULL => the installed File::Raw::JSON is too old or absent;
     * fall back to the Perl codec. */
}

# per request, on the JSON body bytes:
#   SV *val = FRJ->decode(aTHX_ body, body_len, NULL);
#   ... use val ...  (sv_2mortal it or SvREFCNT_dec when done)

Install

cpanm File::Raw
cpanm File::Raw::JSON File::Raw::Separated File::Raw::Gzip

Each plugin is its own dist: install the ones you use. Every one is C under a Perl interface. Gzip and Archive link against the system zlib, so its development headers come first (zlib1g-dev on Debian and Ubuntu).

The reference File::Raw on CPAN