Handling NA values in `stat_summary(geom="line")` is inconsistent with `geom_line()`

[bnprks]

geom_line() nicely handles NA values in the middle of the data by making the line discontinuous.

I would have expected stat_summary(geom="line") to do the same, but it connects across the gap which can result in a misleading output plot.

Granted, the docs for stat_summary() don't promise consistency with geom_line() so I wouldn't classify this as a bug, but it would be a nice-to-have so I don't need to resort to setting a separate group across every data discontinuity when using stat_summary(geom="line")

library(tidyverse)

# Here, I would expect some way to make stat_summary not connect points
# across an NA value
tibble(x=c(1,2,3,4,5), y=c(1,3,NA,4,2)) |>
  ggplot(aes(x, y)) + stat_summary(fun="mean", geom="line")
#> Warning: Removed 1 row containing non-finite outside the scale range
#> (`stat_summary()`).

# geom_line() properly makes the line discontinuous 
tibble(x=c(1,2,3,4,5), y=c(1,3,NA,4,2)) |>
  ggplot(aes(x, y)) + geom_line() 

^{Created on 2024-12-10 with reprex v2.1.0}

[teunbrand]

I think most stats (except stat_identity()) automatically drop missing values before computation. There isn't really an opt-out for this behaviou. If you want to keep the discontinuous line, I'd recommend precomputing the summary before feeding the data to ggplot2.

[bnprks]

Yeah, I understand and your suggestion is probably the cleanest way to work around this behavior. So it's fine if this gets closed as a "won't fix". I'm just bringing up the inconsistency in case it seems easy/worthwhile to remove the inconsistency.

I think this line might be the offending culprit for automatically dropping missing values like you mention. I'm not sure if something else would break if missing values are retained for "line" geoms, but it would make ggplot's behavior more intuitive for stat_summary(geom="line") to behave the same as geom_line() (at least assuming the user hasn't opted in to removing missing values with stat_summary(na.rm = TRUE) )

[teunbrand]

You're right about that line of code dropping the missing observations. I do sympathise with the frustration and I do think this is a sensible request, but it is an impractical one to execute. Here are the reasons why I think that:

• Keeping all NAs has the danger that not all summary functions can deal with NAs.
• Stats do not really know about geoms: stat_summary() doesn't know what geom it is paired with. I'm not saying it is impossible, but encoding special behaviour for combining the stat with line geoms is not straightforward.
• stat_summary() is a very general stat that does not have strong ties to any particular geom, so also not to a line geom. I'd say it would be fine to search for a solution if they were close-knit, like stat_ydensity() and geom_violin(), but they are not.

I happily change my mind if one points out that it is straightforward to implement or my concerns are unfounded.

[bnprks]

I think the question is more about desired behavior and defaults rather than feasibility of implementation, as I assume that it would be possible to check for a param/flag in the "Stat" ggproto base to know when to skip removing NA values.

On the topic of user interface and expected behavior, one core critique here is that seemingly all the stat_* functions (maybe all of ggplot?) currently remove NAs both when na.rm = FALSE and when na.rm = TRUE. Personally, I might say the current behavior of na.rm = FALSE is better described as na.rm = "warn", as it still removes NAs. I would expect as a user that if na.rm = FALSE, it's my responsibility to make sure that the summary functions I'm using handle NA appropriately, and in exchange, ggplot won't remove NAs. (much as base R handles na.rm for e.g. mean, min, quantile, etc.)

I can't think of another way to easily let the user specify that they'd like NAs to be retained besides setting na.rm = FALSE. So I'll close this for now as it seems out of scope to try making a possibly breaking behavior change across a lot of ggplot, even if it does have some rather unfortunate side-effects in this case.