Decide how to structure the analysis code

[lionfish0]

The bee_track module runs on the pi and is responsible for collecting the images etc.

Once this is done the images need analysing:

• finding the retroreflectors in the photos (currently done by the retrodetect module); an upgrade could be to use the data from the 3d path inference and alignment to find the tags more efficiently.
• the above stage can maybe be completely automatic, or more likely, support human labelling using the 'btviewer' web tool (extract from bee_track and put it into its own repo). The human labelling might also be necessary to identify landmarks in different photos (to help with alignment).
• alignment (this is a mess! -- I tried to write some code that found the striped black & white posts [in future I'd like to avoid having them in the landscape at all anyway] - this didn't work brilliantly, then the idea was that it would find the location and orientation of the cameras in space. Code in beelabel repo): I think this needs to be its own module (input: set of images, maybe a set of [approximate] locations of cameras and landmarks. output: their 3d locations and orientations).
• doing 3d path inference (uses the retroreflectors found by retrodetect and the alignment). At the moment this is in the 'beelabel' repo.

The beelabel repo doesn't really make sense - it just ended up being where I stuck code, but I think the best thing is to:

1. Remove the beelabel repo completely
2. Create a new repo for an alignment module (ideally we find a module/tool that already does this) [but could pick out some of the stuff from beelabel]
3. Create a new repo for the 3d path inference module [can reuse code from the beelabel repo]
4. Create a new repo for the 'btviewer' web-interface for human labelling [extract from the bee_track module].

So we would have these modules (repos):

• bee_track
• retrodetect
• btviewer
• alignment
• flight_path_inference

I think in terms of analysis pathway, sorting the alignment is priority (I had a couple of days last summer of data collection, but never got it analysed, as I got stuck on alignment -- also I'm aware while manually clicking on bees flying around, that having the alignment and flight_path_inference available would make it far easier (and potentially completely automated) to do the 3d path inference.

[Joe-Heffer-Shef]

This makes sense, thanks for clarifying. We'll start work organising these repos and making them into proper Python packages that are automatically tested and published to PyPI.

Alignment

Regarding the alignment code, I've been looking into this a little and it looks like there's existing code out there to perform camera "resectioning" or calibrating the extrinsic parameters/pose a camera. If I understand your situation, you have 3x cameras and some kind of calibration sticks/measuring posts with known positions and heights? So the challenge is to find the relative pose of each of the three cameras. (or the position of each relative to the world? I'm not sure exactly how you'd define this.)

From a quick search, it looks like MATLAB have a camera calibration tool and there's also some tools in OpenCV that are relevant e.g. measuring a "chessboard" for pose estimation.

What's the general approach with this code? I'm struggling to read and understand the existing code (alignment.py) and the Jupyter Notebook but I'll keep trying and let you know if we have any questions.