Login

I’ve been trying to implement a custom loss function for a multi-label classification problem, but my model’s validation performance is completely stagnant no matter how I adjust the learning rate or architecture. I’m starting to wonder if my gradient calculation is correct, or if the issue is with how the function handles class imbalance. Has anyone else run into this kind of plateau when moving beyond standard cross-entropy?

I’ve chased a similar plateau with a custom loss for multi label too. I added a per class weight and a BCE term that changed over time, and the val loss just sat there. I dumped some prints on the backward pass and gradient norms looked sane, no huge spikes or vanishing updates. Changed learning rate schedule a bunch of times, nothing moved. It felt like the signal wasn’t getting through the loss even when the model was learning.

I played with a few emphasis variants like dice-ish components and adjusted class balance inside the loss. On one project it helped a bit early, but the validation curve just froze after a few epochs. It seems like the model found some local pattern it was confident about and refused to budge, regardless of the loss math.

I keep thinking maybe the issue isn’t the math but the data. I wasted weeks chasing a tweak to the loss when the val set was suddenly easier or the labels were noisy. Sometimes the model learns to memorize coarse signals and the loss keeps changing but the eval stays flat.

Could the real issue be data quality or label noise rather than the loss?

Login
Username:
Password:	Lost Password?
	Remember me