Retire this package?

[oxinabox]

Back when thuis was made it was intended that anyone could subtype AbstractDataFrame.
My impression is it is now intended hat noone outside the DataFrames.jl package creates a subtype.

@bkamins is that right?

I believe Invenia only uses this for one thing internally.

[bkamins]

Well - it is OK to subtype AbstractDataFrame outside DataFrames.jl.

The problem is that it becomes increasingly complex to do this properly unfortunately, e.g. you would have to correctly handle view of your custom type or broadcasting which is not easy.
Simply the more functionality people ask for the harder it gets to extend AbstractDataFrame correctly (unless you are careful to only use a subset of functionality provided by DataFrames.jl).

Given the functionality of this package provides I think it would be better to revive JuliaData/DataFrames.jl#1458 (or start anew) to allow adding custom metadata to a DataFrame. It will be less work and we want to add an option to annotate a DataFrame with metadata anyway. Then you could just switch to DataFrames.jl

[bkamins]

• a proper implementation of indexing for a custom type would probably also be a challenge.

[tk3369]

Is there a formal definition about the interface of AbstractDataFrame?

I'm experimenting something similar to this - wrapping a DataFrame in my own struct and I would like my type to be accepted anywhere that AbstractDataFrame is accepted. I think I can delegate most functions but need to know which ones...

[bkamins]

Is there a formal definition about the interface of AbstractDataFrame?

I assume you are looking for something like https://docs.julialang.org/en/latest/manual/interfaces/.

Unfortunately we do not have one now. Also even if you analyze https://docs.julialang.org/en/latest/manual/interfaces/ you will note that there are corner cases required for interfaces to work not covered and some of the requirements e.g. for broadcasting changed in a breaking way along 1.x releases.

What I want to say by this is not that https://docs.julialang.org/en/latest/manual/interfaces/ is bad - it is great to have it, but rather that it is extremely hard to write and maintain a full list of required interfaces.

I think I can delegate most functions but need to know which ones...

If you need just a thin wrapper I think it is the way to go. A side benefit is that you can implement things as you need them (i.e. if you do not use something just do not define it and add definitions only as you progress).

Now - to find functions that are essential to be defined look for functions that define methods for DataFrame as a starting point. There are currently 110 such methods, but as I have said - you probably can skip most to start with. The core are:

• constructors
• getindex
• setindex!
• getproperty
• setproperty
• propertynames
• insertcols!
• ncol, nrow
• transform! and select! (hard)
• append!, push!
• copy
• delete!
• empty!
• hcat (tricky)
• vcat
• sort!
• parent
• parentindices
• index

[tk3369]

Thanks for the details above. It looks much more complicated than I thought 😅

Now I am having a second thought.... Perhaps I will just define a function to expose the underlying data frame instead of implementing the AbstractDataFrame interface.

[bkamins]

This is what I would do 😄.