Implement `group_by` and `with_groups` for `tidySingleCellExperiment` and `tidyseurat`

[stemangiola]

the issue for tidyseurat is here

stemangiola/tidyseurat#65

[stemangiola]

group_split is also a new useful function

[B0ydT]

I'm interested in having a go at this. I've looked at the dplyr code and think I have a handle on it. Are there any implementations, especially part of tidyomics that you'd prefer I use as a starting point?

[stemangiola]

Hello @B0ydT, great initiative.

This is probably the transcriptomics hardest challenge (the one for tidySE is even harder).

Currently, group_by leads to a table in return, as I never approached the hard problem of making a proper API for this method.

This challenge needs a design document first, as we need to iron out all possible pitfalls of grouping an arbitrary set of columns and doing some operations on them. Some combinations may lead to valid single cell experiment objects, while others may lead to invalid objects, and therefore A tibble must be returned.

• have a look to this workshop to learn more about tidySingleCellExperiment https://tidytranscriptomics-workshops.github.io/bioc2022_tidytranscriptomics/articles/tidytranscriptomics_case_study.html

Let's start from group_by, imagine how many ways a user could group_by across cell metadata and reduced dimensions, and how many ways it could operate on this object, e.g. mutate, summarize, something else.

Then pretty much you have creative freedom in proposing a design document, describing the logics of the group_by API for SingleCellExperiment

[stemangiola]

I'm interested in having a go at this. I've looked at the dplyr code and think I have a handle on it. Are there any implementations, especially part of tidyomics that you'd prefer I use as a starting point?

Any news? Let me know if you need help.

[B0ydT]

Hi @stemangiola. Apologies for the slow response, I've been in the middle of changing jobs. I certainly wasn't looking to jump on the thorniest issue of the lot; It took your comment for me to see how this could be quite a lot trickier than the dplyr implementation.

I have tried to think through the issues you've raised. From what I remember of the dplyr implementation, you define the groups when you group_by, but most error detection occurs when you attempt to perform an operation on the grouped object. If we're starting with with_groups as opposed to methods for each relevant function, I could catch those errors and provide informative feedback to the user. Otherwise, I'm very happy to help with implementation, but I'm not sure I'm the right person to work it through from first principles.

[stemangiola]

Hello @B0ydT, congrats for your new job!

You can start with group_split, it is useful and much easier.

A quick function for backend I found is

splitColData <- function(x, f) {
  # This is by @jma1991
  # at https://github.com/drisso/SingleCellExperiment/issues/55
  
  i <- split(seq_along(f), f)
  
  v <- vector(mode = "list", length = length(i))
  
  names(v) <- names(i)
  
  for (n in names(i)) { v[[n]] <- x[, i[[n]]] }
  
  return(v)
  
}

Of course our interface will allow any of the cell-wise info, including reduced dimensions, etc..

[B0ydT]

Thank you very much! That looks great. I'll get it sorted shortly.