Common table operations on Tables.jl compatible sources
Installation: at the Julia REPL, import Pkg; Pkg.add("TableOperations")
Maintenance: TableOperations is maintained collectively by the JuliaData collaborators. Responsiveness to pull requests and issues can vary, depending on the availability of key collaborators.
The TableOperations.select function allows specifying a custom subset and order of columns from a Tables.jl source, like:
ctable = (A=[1, missing, 3], B=[1.0, 2.0, 3.0], C=["hey", "there", "sailor"])
table_subset = ctable |> TableOperations.select(:C, :A) |> Tables.columntableThis "selects" the C and A columns from the original table, and re-orders them with C first. The column names can be provided as Strings, Symbols, or Integers.
The TableOperations.transform function allows specifying a "transform" function per column that will be applied per element. This is handy
when a simple transformation is needed for a specific column (or columns). Note that this doesn't allow the creation of new columns,
but only applies the transform function to the specified column, and thus, replacing the original column. Usage is like:
ctable = (A=[1, missing, 3], B=[1.0, 2.0, 3.0], C=["hey", "there", "sailor"])
table = ctable |> TableOperations.transform(C=x->Symbol(x)) |> Tables.columntableHere, we're providing the transform function x->Symbol(x), which turns an argument into a Symbol, and saying we should apply it to the C column.
Multiple tranfrom functions can be provided for multiple columns and the column to transform function can also be provided in Dicts that
map column names as Strings, Symbols, or even Ints (referring to the column index).
The TableOperations.filter function allows applying a "filter" function to each row in the input table source, keeping rows for which f(row) is true.
Usage is like:
ctable = (A=[1, missing, 3], B=[1.0, 2.0, 3.0], C=["hey", "there", "sailor"])
table = ctable |> TableOperations.filter(x->Tables.getcolumn(x, :B) > 2.0) |> Tables.columntableThe TableOperations.map function allows applying a "mapping" function to each row in the input table source; the function f should take and
return a Tables.jl Row compatible object. Usage is like:
ctable = (A=[1, missing, 3], B=[1.0, 2.0, 3.0], C=["hey", "there", "sailor"])
table = ctable |> TableOperations.map(x->(A=Tables.getcolumn(x, :A), C=Tables.getcolumn(x, :C), B=Tables.getcolumn(x, :B) * 2)) |> Tables.columntableThe TableOperations.narrowtypes function allows infering column element types to better fit the stored data. Usage is like:
ctable_type_any = (A=Any[1, missing, 3], B=Any[1.0, 2.0, 3.0], C=Any["hey", "there", "sailor"])
table = TableOperations.narrowtypes(ctable_type_any) |> Tables.columntableThe TableOperations.dropmissing function allows to lazily remove every row where missing values are present. Usage is like:
ctable = (A=[1, missing, 3], B=[1.0, 2.0, 3.0], C=["hey", "there", "sailor"])
table = ctable |> TableOperations.dropmissing |> Tables.columntableThe TableOperations.joinpartitions function allows you to lazily chain (or "join") multiple tables into a single long table. Usage is like:
ctables = Tables.partitioner(i -> (A=fill(i, 10), B=rand(10) * i), 1:3)
table = ctables |> TableOperations.joinpartitions |> Tables.columntableContributions are very welcome, as are feature requests and suggestions. Please open an issue if you encounter any problems or would just like to ask a question.