Skip to content

Check bounding boxes and trees before starting neuron segmentation #8678

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions frontend/javascripts/messages.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -503,4 +503,6 @@ instead. Only enable this option if you understand its effect. All layers will n
`This feature is not available in your organization's plan. Ask the owner of your organization ${organizationOwnerName} to upgrade to a ${requiredPlan} plan or higher.`,
"organization.plan.feature_not_available.owner": (requiredPlan: string) =>
`This feature is not available in your organization's plan. Consider upgrading to a ${requiredPlan} plan or higher.`,
"jobs.wrongNumberOfBoundingBoxes":
"To use the split/merger evaluation, make sure to have exactly one bounding box, either user-defined or from a task.",
};
11 changes: 8 additions & 3 deletions frontend/javascripts/viewer/controller/scene_controller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -334,13 +334,18 @@ class SceneController {
adding a new one. Since this function is executed very rarely,
this is not a performance problem.
*/
for (const [tracingId, boundingBox] of Object.entries(taskCubeByTracingId)) {

// Clean up old entries
for (const [tracingId, _boundingBox] of Object.entries(this.taskCubeByTracingId)) {
let taskCube = this.taskCubeByTracingId[tracingId];
// Remove the old box if it exists
if (taskCube != null) {
taskCube.getMeshes().forEach((mesh) => this.rootNode.remove(mesh));
}
this.taskCubeByTracingId[tracingId] = null;
}
// Add new entries
for (const [tracingId, boundingBox] of Object.entries(taskCubeByTracingId)) {
let taskCube = this.taskCubeByTracingId[tracingId];
if (boundingBox == null || Store.getState().task == null) {
continue;
}
Expand Down Expand Up @@ -733,7 +738,7 @@ class SceneController {
this.updateMeshesAccordingToLayerVisibility(),
),
listenToStoreProperty(
(storeState) => getTaskBoundingBoxes(storeState.annotation),
(storeState) => getTaskBoundingBoxes(storeState),
(boundingBoxesByTracingId) => this.updateTaskBoundingBoxes(boundingBoxesByTracingId),
true,
),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,10 @@ export function findTreeByNodeId(trees: TreeMap, nodeId: number): Tree | undefin
return trees.values().find((tree) => tree.nodes.has(nodeId));
}

export function hasEmptyTrees(trees: TreeMap): boolean {
return trees.values().some((tree: Tree) => tree.nodes.size() === 0);
}

export function findTreeByName(trees: TreeMap, treeName: string): Tree | undefined {
return trees.values().find((tree: Tree) => tree.name === treeName);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,10 @@ export function selectTracing(
return tracing;
}

function _getTaskBoundingBoxes(annotation: StoreAnnotation) {
const layers = _.compact([annotation.skeleton, ...annotation.volumes, annotation.readOnly]);

function _getTaskBoundingBoxes(state: WebknossosState) {
const { annotation, task } = state;
if (task == null) return {};
const layers = _.compact([annotation.skeleton, ...annotation.volumes]);
return Object.fromEntries(layers.map((l) => [l.tracingId, l.boundingBox]));
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ import {
rgbToHex,
} from "libs/utils";
import _ from "lodash";
import messages from "messages";
import React, { useEffect, useState, useMemo } from "react";
import { useDispatch } from "react-redux";
import { type APIDataLayer, type APIJob, APIJobType, type VoxelSize } from "types/api_types";
Expand All @@ -56,7 +57,11 @@ import {
getMagInfo,
getSegmentationLayers,
} from "viewer/model/accessors/dataset_accessor";
import { getUserBoundingBoxesFromState } from "viewer/model/accessors/tracing_accessor";
import { hasEmptyTrees } from "viewer/model/accessors/skeletontracing_accessor";
import {
getTaskBoundingBoxes,
getUserBoundingBoxesFromState,
} from "viewer/model/accessors/tracing_accessor";
import {
getActiveSegmentationTracingLayer,
getReadableNameOfVolumeLayer,
Expand Down Expand Up @@ -636,6 +641,19 @@ function CollapsibleSplitMergerEvaluationSettings({
children: (
<Row>
<Col style={{ width: "100%" }}>
<div style={{ marginBottom: 24 }}>
You can evaluate splits/mergers on a given bounding box. <br />
By default this is the user defined bounding box or the bounding box of a task.{" "}
<br />
Thus your annotation should contain
<ul>
<li> either one user defined bounding box or the bounding box of a task</li>
<li>
with at least one neuron (sparse) or all neurons (dense) annotated as
skeletons.
</li>
</ul>
</div>
<Form.Item
layout="horizontal"
label="Use sparse ground truth tracing"
Expand Down Expand Up @@ -1054,9 +1072,10 @@ export function NucleiDetectionForm() {
export function NeuronSegmentationForm() {
const dataset = useWkSelector((state) => state.dataset);
const { neuronInferralCostPerGVx } = features();
const hasSkeletonAnnotation = useWkSelector((state) => state.annotation.skeleton != null);
const skeletonAnnotation = useWkSelector((state) => state.annotation.skeleton);
const dispatch = useDispatch();
const [doSplitMergerEvaluation, setDoSplitMergerEvaluation] = React.useState(false);

return (
<StartJobForm
handleClose={() => dispatch(setAIJobModalStateAction("invisible"))}
Expand Down Expand Up @@ -1099,6 +1118,31 @@ export function NeuronSegmentationForm() {
doSplitMergerEvaluation,
);
}

const state = Store.getState();
const userBoundingBoxCount = getUserBoundingBoxesFromState(state).length;

if (userBoundingBoxCount > 1) {
Toast.error(messages["jobs.wrongNumberOfBoundingBoxes"]);
return;
}

const taskBoundingBoxes = getTaskBoundingBoxes(state);
if (Object.values(taskBoundingBoxes).length + userBoundingBoxCount !== 1) {
Toast.error(messages["jobs.wrongNumberOfBoundingBoxes"]);
return;
}

if (skeletonAnnotation == null || skeletonAnnotation.trees.size() === 0) {
Toast.error(
"Please ensure that a skeleton tree exists within the selected bounding box.",
);
return;
}
if (hasEmptyTrees(skeletonAnnotation.trees)) {
Toast.error("Please ensure that all skeleton trees in this annotation have some nodes.");
return;
}
return startNeuronInferralJob(
dataset.id,
colorLayer.name,
Expand Down Expand Up @@ -1126,7 +1170,7 @@ export function NeuronSegmentationForm() {
</>
}
jobSpecificInputFields={
hasSkeletonAnnotation && (
skeletonAnnotation != null && (
<CollapsibleSplitMergerEvaluationSettings
isActive={doSplitMergerEvaluation}
setActive={setDoSplitMergerEvaluation}
Expand Down
2 changes: 2 additions & 0 deletions unreleased_changes/8678.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
### Changed
- Before starting a neuron segmentation with `Evaluation Settings` enabled, it is checked that a useful bounding box was selected and that some skeletons exist within the annotation, preventing the job from failing.