materializer for matrix tables

[ablaom]

This is not currently overloaded, ie, falls back to columntable.

In MLJ (and elsewhere) core models operate on matrices. An MLJ interface for a model accepts a table X, converts it to a matrix (using Tables.matrix) and reports any output as a table using materializer(X)(Xout). This is currently leading to unnecessary copying and computation in the last step.

Is there an overloading of materializer for matrix tables that Tables.jl would be willing to implement to address this use case?

[nalimilan]

You mean something like Tables.materializer(X::AbstractMatrix) = Tables.table?

I must be missing something, as Tables.columntable doesn't support matrices:

julia> X = [1 2; 3 4]
2×2 Matrix{Int64}:
 1  2
 3  4

julia> Tables.materializer(X)
columntable (generic function with 5 methods)

julia> Tables.materializer(X)(X)
ERROR: ArgumentError: a 'Matrix{Int64}' is not a table; see `?Tables.table` for ways to treat an AbstractVecOrMat as a table
Stacktrace:
 [1] columns(m::Matrix{Int64})
   @ Tables ~/.julia/packages/Tables/M26tI/src/matrix.jl:5
 [2] columntable(itr::Matrix{Int64})
   @ Tables ~/.julia/packages/Tables/M26tI/src/namedtuples.jl:171
 [3] top-level scope
   @ REPL[10]:1

[ablaom]

You mean something like Tables.materializer(X::AbstractMatrix) = Tables.table?

No. Currently, we have:

using Tables
matrix_table = Tables.table(rand(2,3))
some_table = [(x=1, y=2), (x=3, y=4)]
julia> Tables.materializer(matrix_table)(some_table)
(x = [1, 3], y = [2, 4])

I'm getting a column table for the last call because that's the fallback behaviour for materializer. But I want Tables.materializer(matrix_table)(some_table) to be a matrix table.

[nalimilan]

OK, makes sense. So that would mean Tables.materializer(::Type{<:Tables.MatrixTable}) = Tables.table ∘ Tables.matrix?

[ablaom]

Syntactically, that would work for my use-case. However, I'm not sure that would be ideal as an implementation. I was hoping that Tables.materializer(matrix_table), when applied to Tables.table(some_matrix), would compile to a no-op. But that doesn't appear to be the case, although it does seem that some_matrix is not copied.

[nalimilan]

Yeah we could define a separate function with a fallback method calling Tables.table ∘ Tables.matrix and add a no-op method for MatrixTable inputs.

[ablaom]

Perfect.