single-model vs 5-fold cross-validation

[mgarbade]

Has anyone compared nnU-Net trained as a single model on 100% of the training data vs. the default 5-fold CV ensemble (5×80%)? I wonder how much accuracy is actually lost or gained, given the big inference speed advantage of using just one model.

I expect CV ensembles to help on small datasets by stabilizing hyperparameter choices, but on very large datasets you probably don’t need to hold out 20% for validation. In that scenario the benefit should vanish, and a single model would be much faster to train and for inference. Any benchmarks or experience with this?

[vmiller987]

Are you referencing the all-fold option? This puts all of the training data into validation as well and will provide an overfit model.

Otherwise it would basically be a 1-fold training, where 80% in training and 20% in validation. The issue is you might not have the best images in validation which is why the cross fold helps.