Orientation handling of input volumes affects inference results

[hassanelsheikh]

Hello,

I wanted to share an issue I encountered that took me quite some time to figure out. I was working with nnU-Net for bone segmentation, but my dataset did not have a consistent orientation across volumes. At first, I wasn’t aware that nnU-Net does not normalize or enforce a fixed orientation during preprocessing.

During training, the validation results looked great, but when running inference on new data, the performance dropped significantly. After some investigation, I realized that the variation in orientation between training and inference volumes was the root cause.

To resolve this, I had to manually reorient all my volumes to a consistent orientation before training and inference. This extra step was crucial for achieving reliable results, but it was not obvious at the start.

Suggestion:
It might be helpful if nnU-Net provided an option (or at least a warning) to enforce a fixed orientation during preprocessing/training. This would save users time and prevent confusion when working with datasets that come in different orientations.

Thank you for the great tool, and I hope this feedback helps improve the workflow for future users.

[ykirchhoff]

Hi @hassanelsheikh,

this is indeed something, which can result in unwanted behaviour. There are specific image reader/writer classes for this, namely NibabelIOWithReorient and SimpleITKIOWithReorient. You can set these in your dataset.json to overwrite the default (usually SimpleITKIO). However, this should only affect datasets, where you have some symmetry in the labels, for example left and right kidney, and don't train with the normal nnUNetTrainer, which does mirroring during training as well as inference. I still agree that the documentation on that could be improved.

Best,
Yannick