v0.6.0 #598

i-jey · 2025-01-30T16:43:06Z

Bumping up to v0.6.0 to denote a marked increase in SSAF frequency (and hopefully, a concomitant improvement in image focus over the course of a run)

Autobrightness adjustment frequency bugfix
Memory leak bugfix (extra instantiation of FlowController in scope_routines.py
Switch NCS to THROUGHPUT mode - allows us to run SSAF more frequently (went from 2 imgs/sec --> 10 imgs/sec)
QoL: faster fast flow during setup (increase the pneumatic module step size temporarily when doing the initial fast flow step
Add a defensive check to CellFinder to ensure motor is not currently in motion before starting CellFinder routine

Tested on scope while monitoring htop to ensure memory usage stayed flat, and tested in the oven at 40C (n=10 runs).

Reset counter after adjusting autobrightness

…d during a run.

remove old pyngrok dependency (no longer used)

Remove unused ngrok utils

…polling of SSAF, and thus improve our ability to track focal drift. May want to tweak EWMA parameters to be less conservative since we can now receive more samples. Needs to be tested extensively on scope however, in particular we need to monitor for any memory increases in case the NCS cannot keep up and we start building up a queue.

formatting

Manually reset predictions handler

…oller instance, use mscope's existing one

Fix potential memory leaks

…hich is excessive, up from 10 which is what we had previously)

… to speed faster. Set it back to normal once fast flow is done.

NCS performance hint --> throughput + increase SSAF polling

…on prior to starting

Faster fast flow

This reverts commit 1802246, reversing changes made to 60da548.

NicholasBratvold · 2025-03-11T19:58:55Z

ulc_mm_package/hardware/scope_routines.py

If we turn off fast flow in the future the pneumatic module min step size will stay doubled

I am similarly concerned if this reset adequately resets step size. Have you done multiple runs in a row where at least one run fails on fastflow? From the scope routine it looks like a failure in fastflow means it'll never reset the step size

NicholasBratvold

QOL change in scope_routines.py will keep pneumatic step size doubled if fast flow is disabled. EDIT: I was mistaken, it will only temporarily double if fast flow is enabled.

How does ngrok work now with the files deleted?

i-jey · 2025-03-11T20:03:29Z

QOL change in scope_routines.py will keep pneumatic step size doubled if fast flow is disabled.

How does ngrok work now with the files deleted?

A while back, ngrok updated to v3 and the library we were using to interface with it (pyngrok) stopped working. So we now use the ngrok CLI command instead - there is a bash alias that the technicians use to manually open the tunnel (they type ng when we ask to remote in/pull data).

NicholasBratvold

Thanks for the clarification.

i-jey · 2025-03-11T20:07:19Z

Sweet - I'll also wait on @mwlkhoo to review and then if all looks good, we can merge.

mwlkhoo

Some comments/questions

ulc_mm_package/QtGUI/oracle.py

mwlkhoo · 2025-03-12T21:18:20Z

ulc_mm_package/hardware/scope_routines.py

I am similarly concerned if this reset adequately resets step size. Have you done multiple runs in a row where at least one run fails on fastflow? From the scope routine it looks like a failure in fastflow means it'll never reset the step size

mwlkhoo · 2025-03-12T21:20:01Z

ulc_mm_package/hardware/scope_routines.py

            if counter >= processing_constants.PERIODIC_AB_PERIOD_NUM_FRAMES:
                autobrightness.autobrightness_pid_control(img)
                curr_img_brightness = autobrightness.prev_mean_img_brightness
+                counter = 0


This looks good, but how was this loop ever working without this line?

lol it embarrassingly wasn't, this was a long overlooked bug

mwlkhoo · 2025-03-12T21:21:28Z

ulc_mm_package/hardware/scope_routines.py

        cell_finder = CellFinder()
-        flow_controller = FlowController(mscope.pneumatic_module)
+        mscope.flow_controller.reset()
+        flow_controller = mscope.flow_controller


why do we have a local flow_controller function instead of directly referencing mscope.flow_controller?

Just for a bit of cleanliness/ease of typing, (instead of having to type mscope.flow_controller each time).

mwlkhoo · 2025-03-12T21:24:10Z

ulc_mm_package/neural_nets/predictions_handler.py

+        self.max_confs = {x: [] for x in self.class_ids}
+        self.curr_min_of_max_confs_by_class = {
+            x: HIGH_CONF_THRESH - 1e-6 for x in self.class_ids
+        }
+        self.min_confs = {x: [] for x in self.class_ids}
+        self.curr_max_of_min_confs_by_class = {
+            x: HIGH_CONF_THRESH for x in self.class_ids
+        }


I apologize in advance because I know very little about the predictions handler... but why do these variables need to reset? I assumed these would be consistent across each experiment?

no sweat! These variables hold the predictions for each run - but they are really big arrays - 15 rows * 3.5M columns (which is just an upper bound for how many cells we would reasonably expect YOGO to see). Instead of re-creating this giant array for each run, we just clear it, so that the next run's predictions can fill it up.

… normally when doing fast flow, so that the set up is quicker and more ergonomic for the user (i.e they don't need to wait for a long while for the syringe to hit its stop in the event that debris or stuck cells are messing up the flow estimation calculation. Fix: This properly resets the step size back to what it is normally supposed to be, in the event where fast flow does not reach the target flow rate. Previously it was only being reset when fast flow succeeded.

mwlkhoo · 2025-03-12T22:25:26Z

Looks good, thanks for addressing all my q's!

NicholasBratvold · 2025-03-12T22:34:31Z

Tested on the scope.

mwlkhoo

tested by Nick, looks good

…eset_if_fast_flow_fails Reset pneumatic module step size in the case where `fast_flow` fails

NicholasBratvold

Approved

i-jey added 30 commits January 21, 2025 14:57

Reset counter after adjusting autobrightness

0935d19

Oops, need to fmt with an older version

1159339

Merge pull request #597 from czbiohub-sf/periodic_ab_bugfix

c0cb7c2

Reset counter after adjusting autobrightness

Add monitoring and dialog box exception if a pressure leak is detecte…

d44aed5

…d during a run.

remove extraneous e

60da548

remove old pyngrok dependency (no longer used)

74eb338

Merge pull request #600 from czbiohub-sf/remove_pyngrok

8057ad0

remove old pyngrok dependency (no longer used)

Remove unused ngrok utils

748c2fd

Merge pull request #601 from czbiohub-sf/remove_old_ngrok_utils

4771f07

Remove unused ngrok utils

formatting

3f4fd5f

formatting

42bd123

Force the predictions handler to reset as opposed to relying on gc

5833fd8

Merge pull request #603 from czbiohub-sf/formatting

7404211

formatting

merge branch 'develop' into reset_predictions_handler

ea5d78d

Merge pull request #604 from czbiohub-sf/reset_predictions_handler

df0da79

Manually reset predictions handler

Address potential memory leaks

2b2588a

remove unnecessary __init__ call in PredictionsHandler reset

4e78eca

close per img metadata file

3b7166f

This was the main problem - we should not instantiate a new FlowContr…

819a5e7

…oller instance, use mscope's existing one

remove unused import

76bb1de

Merge branch 'fix_potential_memory_leaks' into ncs_high_throughput

5c576bb

Merge pull request #606 from czbiohub-sf/fix_potential_memory_leaks

604a197

Fix potential memory leaks

Merge branch 'develop' into ncs_high_throughput

0813fb7

Set Autofocus batch size for pre-flow/post-flow to 20 (down from 50 w…

d03f4cb

…hich is excessive, up from 10 which is what we had previously)

reset flow controller prior to starting cell finder

42914d4

Double the fast flow pneumatic module step size just so we can get up…

a939cfc

… to speed faster. Set it back to normal once fast flow is done.

fmt

b2fa3e3

fmt

796d854

Merge pull request #602 from czbiohub-sf/ncs_high_throughput

2f3a6fe

NCS performance hint --> throughput + increase SSAF polling

i-jey and others added 7 commits March 7, 2025 16:12

merge branch 'develop' into faster_fast_flow

7aa24a3

Add in a small check to CellFinder to ensure the motor is not in moti…

6ceacca

…on prior to starting

Merge pull request #607 from czbiohub-sf/faster_fast_flow

ac952d0

Faster fast flow

Merge branch 'develop' into pressure_leak_warning

1802246

Revert "Merge branch 'develop' into pressure_leak_warning"

97f47d8

This reverts commit 1802246, reversing changes made to 60da548.

rollback to PR607 merge

09eff4d

remove old unused files

7f19eb8

NicholasBratvold reviewed Mar 11, 2025

View reviewed changes

NicholasBratvold approved these changes Mar 11, 2025

View reviewed changes

mwlkhoo requested changes Mar 12, 2025

View reviewed changes

i-jey changed the title ~~v0.5.6~~ v0.6.0 Mar 12, 2025

NicholasBratvold previously approved these changes Mar 12, 2025

View reviewed changes

mwlkhoo previously approved these changes Mar 12, 2025

View reviewed changes

Merge pull request #608 from czbiohub-sf/pneumatic_module_step_size_r…

f13e84b

…eset_if_fast_flow_fails Reset pneumatic module step size in the case where `fast_flow` fails

i-jey dismissed stale reviews from mwlkhoo and NicholasBratvold via f13e84b March 12, 2025 23:14

NicholasBratvold approved these changes Mar 13, 2025

View reviewed changes

i-jey merged commit 5b20c53 into master Mar 13, 2025
4 checks passed

v0.6.0 #598

v0.6.0 #598

Uh oh!

Conversation

i-jey commented Jan 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NicholasBratvold left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

i-jey commented Mar 11, 2025

Uh oh!

NicholasBratvold left a comment

Choose a reason for hiding this comment

Uh oh!

i-jey commented Mar 11, 2025

Uh oh!

mwlkhoo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mwlkhoo commented Mar 12, 2025

Uh oh!

NicholasBratvold commented Mar 12, 2025

Uh oh!

mwlkhoo left a comment

Choose a reason for hiding this comment

Uh oh!

NicholasBratvold left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

i-jey commented Jan 30, 2025 •

edited

Loading

NicholasBratvold left a comment •

edited

Loading