Have camera auto calibrate with specific analoguegain

[quitmeyer]

Hi everyone thanks to all your work our cool open source scientific tool for studying insect biodiversity has been working really great in the field!

There's one challenge I'm working on right now which is that I want to try to shorten my exposure times because sometimes our insects fly across the screen quite quickly and we get a big rolling shutter smear.

Currently, as you can see in our code:
https://github.com/Digital-Naturalism-Laboratories/Mothbox/blob/4957273d84673e9d69ca90a163188950dddd453f/Software/TakePhoto.py

We run an auto calibration every 6 minutes that sets the exposure time and the focus. This has been working well (with one caveat I'll explain below)***

But I wanted to boost the gain before I run the auto calibration in order to try to calibrate it for a shorter exposure time.

However when I try to do this, It doesn't seem to have any effect on the auto calibration. And so it just calibrates it with a normal type of exposure, And then when I apply that exposure to a photo with the analog gain, I just get a blown out overexposed photo.

*** Side question I had a strange problem where when I run the calibration. For some reason if If I run an auto calibration and then I take a photo using the settings that were collected on that auto calibration this photo comes out slightly brighter then if I use those exact same settings but do not perform an auto calibration before that. Anyone have any ideas about that? It's not really that significantly brighter so it's not that big of a deal, But it's irking me! :)

[davidplowman]

Hi, thanks for the questions. There was a bit too much code there for me to understand exactly what you're trying to calibrate and how you're doing it. Is there any way you could provide a short example that shows the camera system doing something that you don't expect or don't understand? Thanks!

[quitmeyer]

Hi david! No problem!

So on my DSLR camera, under a fixed lighting situation, if i want to try to have a faster image with a shorter shutter speed, i could do something like increase the gain, and then when i run an autoexposure, it will use the increased gain to brighten my photo and automatically decrease the shutter speed.

I wanted to try to do something like that with the picamera2, but after i set the gain, and then run the autoexposure algo, it looks like it always just reverts back to gain of 0 and gives the same values as if i did not set the gain before the autoexposure. Chances are I am doing something wrong!

Here's what I do:

(earlier in the code i set the gain to something like 2.0 in a dict called camera_settings)

    preview_config = picam2.create_preview_configuration(main={'format': 'RGB888', 'size': (1920*2, 1080*2)})
    still_config = picam2.create_still_configuration(main={"size": (width, height), "format": "RGB888"}, buffer_count=1)
    picam2.configure(preview_config)
 
    #picam2.set_controls({"AfMode":0,"AfSpeed":0,"AfRange":0, "LensPosition":7.0})

    time.sleep(1)
    flashOn()
    afstart = time.time()
    print("Autofocusing ")
    picam2.pre_callback = print_af_state
    
    picam2.start(show_preview=False)
    
    time.sleep(2)
    picam2.set_controls({"LensPosition":8.0})
    gain=camera_settings["AnalogueGain"]
    picam2.set_controls({"AnalogueGain":gain})
    time.sleep(3)

    success = picam2.autofocus_cycle()

    #picam2.pre_callback = None
    print("Autofocus completed! "+str(time.time()-afstart))
    calib_lens_position = picam2.capture_metadata()['LensPosition']
    calib_exposure = picam2.capture_metadata()['ExposureTime']
    focusstate = picam2.capture_metadata()['AfState']
    flashOff()

    print("LensPosition: "+str(calib_lens_position))
    print("Exposure: "+str(calib_exposure))

my expectation would be that if I set the gain to like 4.0 or something vs the default 0.0 i would get an image of similar brightness but shorter exposure time. but it seems like during the calibration it doesn't let me calibrate with a different gain because i always just get the same exposure time?

[davidplowman]

Thanks for sending the code. It looks to me like it really should work. Once thing to bear in mind is that there are several frames of delay for the analogue gain to update (if you change it after the camera has started). Can you maybe try running something like this and post the output?

from picamera2 import Picamera2

picam2 = Picamera2()
config = picam2.create_preview_configuration()
picam2.configure(config)
picam2.start()

for i in range(30):
    if i == 5:
        picam2.set_controls({'AnalogueGain': 2.0})
    elif i == 15:
        picam2.set_controls({'AnalgoueGain': 4.0})
    md = picam2.capture_metadata()
    print(i, md['ExposureTime'], md['AnalogueGain'])

Note that if you set the analogue gain that you want before calling start(), then the camera should start with the requested value.

[quitmeyer]

Ok, so this looks more like what I am seeing!
See how the exposure doesn't change whatsoever from a starting gain around 5.0 to 2.0?
it does change on the 15 iteration between 2.0 to 8.0

0 66647 4.7265625
1 66647 5.1015625
2 66647 5.1015625
3 66647 5.359375
4 66647 5.203125
5 66647 2.0
6 66647 2.0
7 66647 2.0
8 66647 2.0
9 66647 2.0
10 66647 2.0
11 66647 2.0
12 66647 2.0
13 66647 2.0
14 66647 2.0
15 36553 8.0
16 36553 8.0
17 36553 8.0
18 36553 8.0
19 36553 8.0
20 36553 8.0
21 36553 8.0
22 36553 8.0
23 36553 8.0
24 36553 8.0
25 36553 8.0
26 36553 8.0
27 36553 8.0
28 36553 8.0
29 36553 8.0

I just modified your code by adding an autofocus cycle and putting a high resolution still configuration at the top, and then it starts acting funny.

*note if i have JUST the autofocus cycle in the code you showed then it works as expected, but when i specify the configurations at top, it starts acting funny
** Note i am using the arducam owlsight 64mp cam

Is this just a slow camera problem? do i need to add a delay of longer than a second or so you think?

from picamera2 import Picamera2

picam2 = Picamera2()
config = picam2.create_preview_configuration()
picam2.configure(config)
#default
width=9000
height=6000

preview_config = picam2.create_preview_configuration(main={'format': 'RGB888', 'size': (1920*2, 1080*2)})
still_config = picam2.create_still_configuration(main={"size": (width, height), "format": "RGB888"}, buffer_count=1)
picam2.configure(preview_config)
picam2.start()

for i in range(30):
    if i == 5:
        picam2.set_controls({'AnalogueGain': 2.0})
        success = picam2.autofocus_cycle()

    elif i == 15:
        picam2.set_controls({'AnalogueGain': 8.0})
        success = picam2.autofocus_cycle()

    md = picam2.capture_metadata()
    print(i, md['ExposureTime'], md['AnalogueGain'])

[davidplowman]

Be careful with picam2.autofocus_cycle() because it will block for many frames until the autofocus cycle is complete. Is that what you indend? Maybe try picam2.autofocus_cycle(wait=False) which will leave it running asychronously in the background. Though you probably don't want to start a second autofocus cycle if the first one hasn't finished.

[quitmeyer]

Interesting.

I don't want it asynchronous because i am trying to optimize my power use. Basically I have really bright lights that i can turn on and off, but because we want to save battery, i want to turn them on just for the autofocusing and autoexposing, and turn them off exactly when it is done autofocusing. Then i save the details (lensposition and exposure value) to take shorter photos in the future

[davidplowman]

I wouldn't try and change exposure and focus at the same time. It's possible that the changing brightness level of the image will upset the autofocus. So I would:

1. Turn on the bright lights.
2. Start the camera and let the auto exposure settle for ~1/2 second. Make a note of the ExposureTime and Analogue gain. Also record the ColourGains.
3. Now run the autofocus cycle. When it finishes, get the LensPosition.

Then when you want to capture an image:

1. Start the camera setting the ExposureTime, AnalogueGain, ColourGains and LensPosition before you call picam2.start().
2. It's now safe to capture the very first image that comes out of the camera.

[quitmeyer]

Oh this is SUPER fantastic! Shall try it out and report back!

[quitmeyer]

Ok, with David's help, i think i fixed it up. I think I had two problems:

1st: i think when it was running the autofocuscycle it was blocking the exposure values from changing, and i think that prevented me from being able to set a gain and have it expose based on that.

To fix that, I did what @davidplowman suggested which is start the camera, let it figure out its exposure values, record those values, THEN run a cycle to get the focus!

so that let my exposure values start functioning and responding to the changes in gain i made!

2nd: the camera seems to have a minimum exposure of 469 (microseconds?), and so after i fixed the above problem, i started losing my mind because it seemed like it stopped responding past certain gain values, but i think it just can't adjust the exposure below that. Can you confirm that @davidplowman ? if so i think we found all the bugs i needed to work around and this question should be all set!

[davidplowman]

I'm afraid I don't know the minimum exposure time for non-Raspberry Pi supplied cameras. You'd have to refer to the module vendor (Arducam).

Having said that, it you try to set the exposure time lower, but it won't go lower, then that's what's meant to happen once you get to the minimum value...

[quitmeyer]

Awesome! thanks again for all your amazing help @davidplowman !