How can I verify my gradient for a custom multi-label loss with class imbalance?

[Ryan_M]

I’ve been trying to implement a custom loss function for a multi-label classification problem, but my model’s validation metrics are barely moving even after several epochs. I’m wondering if my gradient calculation is correct or if the issue is with how I’m handling the class imbalances in the penalty.

[AnthonyNB]

I dug into my custom loss last month, thought the gradient was fine because the training loss dropped but validation barely budged. I re-implemented the penalty with a per-class weight and still saw no change. Maybe I'm overfitting training noise.

[Oliver.L]

I did a quick sanity check by printing the weight updates for a couple of layers, and the norms looked fine. I tried a few small tweaks to the learning rate and batch size, nothing moved the validation metrics.

[Benjamin92]

I keep staring at the data, though; some labels hardly show up at all. I added class weights but it moved nothing, and the macro metrics barely budge. Maybe the issue is not the loss but how the labels are distributed.

[HannahIJ]

I even chased a random loader quirk for a day that turned out to be unrelated, then tried a simple BCE baseline to sanity check and it performed a bit better, but the custom penalty still felt off. It makes me wonder if the real problem is something else entirely.