Issue with add_at_risk_counts, lifelines 0.25.9

[claudiocc1]

I went back to an old lifelines project to "prettify" some
plots for a publication.
I am now having problems with add_at_risk_counts.
Unfortunately I do not remember what version of lifelines I
used to run at the earlier time.
I am now running 0.25.9 which I believe is the latest.

There are two problems
(1) The at-risk counts at the end of the plot are set to zero (*)
(2) The at-risk counts are outputted in jupyter notebook but
do not show up when I try to save the plot + the counts
in a figure.pdf file

(*) In my project the "duration" is capped at less than or equal to 400
days. The last "tick" on the x-axis when the curve is plotted is at
400. Since it appears that the counts are given in correspondence
to the tickmarks (right?), the last displayed counts are at 400 and are set
to zero. This seems like a "greater than" vs "greater than or equal to"
bug (or feature?) that was introduced at some point (?).

You can see the outputs here:
https://www.dropbox.com/s/ctowotae1jjcpbs/at_risk.pdf?dl=0

The old code looked like this

ax = plt.subplot(111)
kmf1.fit(T1, E1, label = 'L')
kmf1.plot_survival_function(ax=ax)
kmf2.fit(T2, E2, label = 'M')
kmf2.plot_survival_function(ax=ax)
kmf3.fit(T3, E3, label = 'H')
kmf3.plot_survival_function(ax=ax)
ax.set_ylim([0., 1.1])
ax.set_xlim([0., Ndays])
ax.set_xlabel("Days")
ax.set_ylabel("Survival probability")
add_at_risk_counts(kmf1, kmf2, kmf3)
thisFig = plt.gcf()
thisFig.savefig("figure.pdf")

The new code is identical to the old code except for these three lines

ax.set_xlabel("Days", fontsize=15)
ax.set_ylabel("Survival probability", fontsize=15)
add_at_risk_counts(kmf1, kmf2, kmf3, ax=ax, rows_to_show=['At risk'])

Any help would be appreciated.

Thanks, and thank you for making this great code available!

C.

PS another different "feature" is that the "at risk" information
is much less compact that it used to be (blank lines, etc).
I wonder if there is a way to go back to make it compact.
(Yes, I understand that before it was only showing the "at risk"
counts and now it can optionally also show "censored" and "events"
counts).

[andradekc]

I second @claudiocc1 suggestion. it'd be great to have the "at risk" information more compact.

Thanks!

[CamDavidsonPilon]

Hi @claudiocc1 and @andradekc - these are good points, and will try to address them shortly! Thanks for the comments

[andradekc]

Thanks @CamDavidsonPilon. I'm afraid it still did not work. I've installed it using pip as you described above and rerun the code, exactly as it was before. Am I doing something wrong? My Lifelines version is 0.25.9. Thanks again.

[CamDavidsonPilon]

Hm, my only guess is that it's not installing correctly. Can you provide first dozen lines from the output of:

import lifelines
help(lifelines.plotting.add_at_risk_counts)

[andradekc]

Here you go:

add_at_risk_counts(*fitters, labels: Union[Iterable, bool, NoneType] = None, rows_to_show=None, ypos=-0.6, ax=None, **kwargs)
Add counts showing how many individuals were at risk, censored, and observed, at each time point in
survival/hazard plots.

Tip: you may want to call ``plt.tight_layout()`` afterwards.

Parameters
----------
fitters:
  One or several fitters, for example KaplanMeierFitter, WeibullFitter,
  NelsonAalenFitter, etc...
ax:
    a matplotlib axes
labels:
    provide labels for the fitters, default is to use the provided fitter label. Set to
    False for no labels.
rows_to_show: list
    list of a subset of {'At risk', 'Censored', 'Events'}. Default shows all columns.
ypos:
    increase to move the table down further.

Returns
--------
  ax:
    The axes which was used.

[CamDavidsonPilon]

right, so it looks like the branch wasn't installed (I've update the docs on that function, which aren't present in your output). Try:

 python -m pip install git+https://github.com/camdavidsonpilon/lifelines.git@b251fe6ad8329785b635c2850a89c4f9796abd3f

and if that doesn't work:

 python -m pip install git+https://github.com/camdavidsonpilon/lifelines.git@b251fe6ad8329785b635c2850a89c4f9796abd3f -U

[andradekc]

Great, it worked after trying the second option! Thanks a lot once again

[CamDavidsonPilon]

@claudiocc1 for 2), try adding a plt.tight_layout() after the call to add_at_risk_counts, and before you save. Previously, we called that internally, but had to remove it due to it causing other problems, but it should work for you.

Still looking into 1)

[CamDavidsonPilon]

@claudiocc1 - are you able to provide me with your data (over email is fine too)? I'm not able to repro the problem you are seeing.

[sebastianskejoe]

Alternative potential solution for anyone else, who are here to figure out how to save figures with risk counts: I managed to solve it by calling savefig() with bbox_inches='tight'.

[claudiocc1]

Yes. Let me clean up the code to make it easy for you. I assume that you are ok with a jupyter notebook?

On Thu, Feb 18, 2021 at 19:36 Cameron Davidson-Pilon < ***@***.***> wrote: @claudiocc1 <https://github.com/claudiocc1> - are you able to provide me with your data (over email is fine too)? I'm not able to repro the problem you are seeing. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1229 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABD7FG67X2M5PKVDBTK4XXTS7XMFNANCNFSM4XVYPLPA> .

-- Sent from phone

[CamDavidsonPilon]

Jupyter nb is fine, yup!

[claudiocc1]

how do I send it to you by email ?

[CamDavidsonPilon]

cam.davidson.pilon@gmail.com

[CamDavidsonPilon]

The problem was "at-risk" counts are calculated at end-of-time-period, and in this dataset, everyone was censored at day 400. Thus, at the end-of-day-400, no one was at risk (which is correct). However, the xticks happened to land on 400 as well, so there was a column of zeros. An easy fix was to set the xtickscommand in add_at_risk_counts (undocumented currently, changing that now) to something like xticks=[0, 100, 200, 300, 399]: