What to do if your movie is split into multiple recordings?

[steinkki]

Hello!
I really like Mesmerize's implementation of caiman! However I have a question:

I generate 2 photon calcium movies from behaving animals, but my file sizes get really large. Therefore, I was breaking up the files into smaller chunks to run each through caiman. This can easily be accomplished through the load_movie_chain() command... is there a similar function that can be used with mesmerize?

For clarification, my goal is to generate df/f traces and cell contours for the ENTIRE behavioral session so I can correlate calcium traces with stimuli.

I ran a smaller movie (9.8GB) and it crashed my jupyter kernel at the cnmf.run() command.

I am fairly novice with python, and I would appreciate any and all feedback!

Thank you very much!
Kira

[kushalkolar]

I don't recommend splitting up your movie. Mesmerize-core and mesmerize-viz use lazy loading for all visualization so as long as your file type is lazy-loadable you should be fine. If your kernel is crashing reduce n_processes, see #205 (comment) , so that caiman runs fewer chunks in parallel

[kushalkolar]

There's no reason a 10GB file should fail at the creating memmap step if you have 256GB of RAM, are you sure that's your RAM and not hard disk space? What does htop give you?

[steinkki]

Hello-
Looking at htop, I see the memory climb to about ~90G out of 251G, and then the Jupyter notebook and terminal close. I tried to look at hard disk space (I apologize, I am a novice Linux user), and the data drive caiman should be writing to has roughly 1000Gb available.

I tried setting os.environ[‘CAIMAN_TEMP’]=‘path/to/temp’ after your hard disk space suggestion, and this did not stop the crash.

Thank you so much for your help, I appreciate it so much!

[steinkki]

Hello!
I've been trying a variety of things, but I am still getting the python kernel crash. However, I do see I can generate a c order .mmap file by changing the temp directory path! This file is about 19.7GB in size. I am wondering if I can use this file for further processing, or if perhaps this gives you insight to what is happening?

[kushalkolar]

Try to create a memmap manually, unfortunately the caiman API is very messy but you can do it like this:

import caiman as cm

# use just 2 processes and see if it works
c, dview, n_processes = cm.cluster.setup_cluster(
    backend="local", n_processes=2, single_thread=False
)

# replace input_movie_path with the path to the movie
fname_new = cm.save_memmap(
    [input_movie_path], base_name=f"cnmf-memmap_", order="C", dview=dview
)

Yr, dims, T = cm.load_memmap(fname_new)
images = np.reshape(Yr.T, [T] + list(dims), order="F")

[kushalkolar]

I gave this a try: (set os.environ['MESMERIZE_N_PROCESSES']='50') and set it to a number of processes now, (50, 25, 20, 10, 5, 2) and I still crash at CNMF.

Where are you doing this? Can you provide the full context.