FeatherFiles

This package provides load and save support for Feather V1 files (the original FEA1 format from wesm/feather) under the FileIO.jl package.

Feather V1 is not the same format as Feather V2. V2 is exactly the Arrow IPC file format on disk, and is what every current tool writes — pyarrow, the R arrow package, pandas and Polars — regardless of whether the file is named .feather or .arrow. Apache has deprecated reading and writing V1 as of Arrow 25.0.0 and plans to remove it.

This package handles V1 only. It cannot read a V2 file, so in practice it is useful for archives rather than for files produced today; use Arrow.jl for those. Both formats carry magic bytes (FEA1 and ARROW1), so FileIO identifies which one a given file actually is regardless of its extension.

Feather stores dates and times using the Arrow wire types Datestamp, Timestamp and TimeOfDay. FeatherFiles unwraps these, so date columns arrive as Date, DateTime and Time rather than as FeatherLib internals. (FeatherLib.featherread deliberately hands out the raw types: it is a faithful low-level mirror of the file.)

Use Pkg.add("FeatherFiles") in Julia to install FeatherFiles and its dependencies.

To read a feather file into a DataFrame, use the following julia code:

using FeatherFiles, DataFrames

df = DataFrame(load("data.feather"))

The call to load returns a struct that is an IterableTable.jl, so it can be passed to any function that can handle iterable tables, i.e. all the sinks in IterableTable.jl. Here are some examples of materializing a feather file into data structures that are not a DataFrame:

using FeatherFiles, DataTables, IndexedTables, TimeSeries, Temporal, Gadfly

# Load into a DataTable
dt = DataTable(load("data.feather"))

# Load into an IndexedTable
it = IndexedTable(load("data.feather"))

# Load into a TimeArray
ta = TimeArray(load("data.feather"))

# Load into a TS
ts = TS(load("data.feather"))

# Plot directly with Gadfly
plot(load("data.feather"), x=:a, y=:b, Geom.line)

The following code saves any iterable table as a feather file:

using FeatherFiles

save("output.feather", it)

This will work as long as it is any of the types supported as sources in IterableTables.jl.

Both load and save also support the pipe syntax. For example, to load a feather file into a DataFrame, one can use the following code:

using FeatherFiles, DataFrame

df = load("data.feather") |> DataFrame

To save an iterable table, one can use the following form:

using FeatherFiles, DataFrame

df = # Aquire a DataFrame somehow

df |> save("output.feather")

The pipe syntax is especially useful when combining it with Query.jl queries, for example one can easily load a feather file, pipe it into a query, then pipe it to the save function to store the results in a new file.