What could cause validation loss to be erratic with a multi-label image model?

[Savannah27]

I’ve been trying to implement a custom loss function for a multi-label image classification model, but my validation loss is behaving erratically even though the training loss decreases smoothly. I’m not sure if my issue is with the gradient flow or something in my **loss function implementation** itself.

[KyleT]

I ran into that too. I had a custom multi label loss that summed over classes with a per class weight, and I forgot to use the same reduction in train and val. The validation loss went nuts while train kept dipping. Once I made the same reduction (mean) apply everywhere and kept the same sign convention, it quieted down a bit.

[Scott_C]

I did notice that one or two rare classes were pulling the gradient the wrong way. I tried clipping per class gradients and a tiny epsilon in the log term, and the validation curve steadied for a few epochs, then wandered again.

[Kenneth_R]

Could it be data distribution? I tossed in a quick check and found the val set had different augmentation intensity and it looked like the model was okay with training data but not test; loss still bounced.

[Eric59]

Not sure this is the real issue, but I keep thinking maybe the loss isn't the only thing that matters; I once abandoned a weighting scheme after no change in metrics and felt the training looked different; after that I kept that change and nothing resolved.