What could be wrong with my custom loss function for multi-label training?

[Aria_W]

I’m trying to implement a custom loss function for a multi-label classification problem, but my training loss is converging while validation metrics are barely moving. I’m wondering if my gradient calculation is correct or if there’s something subtle about the backpropagation through this function that I’m missing.

[BenjaminP]

I ran into that pattern once—training loss fell nicely but the validation score barely moved. I double checked data splits for leakage and distribution mismatch; ended up discovering the val set had a different label mix, so the metric stayed flat even though the model learned something.

[Joshua66]

I did a lot of digging by printing per class grads. A couple of rare labels mostly got tiny gradients, so those heads never learned much. I tried adding class weights and a tiny bit of label smoothing; the gradients got a bit healthier, but the validation numbers still didn't cooperate.

[Matthew_G]

If you are backpropagating through a sigmoid per label with BCE, the math should give you p minus y as the gradient wrt the logit. If you accidentally detach inside the loss or mix up which variable you call backward on, you can end up with almost no signal for some classes.

[Oliver_M]

I wonder if the real bottleneck isn't the loss at all but how we compute the metric. Calibrating thresholds or optimizing for the metric directly sometimes changes things. Have you checked whether simply tuning the decision thresholds on the validation set changes the score?