improving multi-label business text classification with imbalanced data: adjusted bce weighting and threshold optimization for rare labels in bert models

Multi-label classification of business texts in the presence of imbalanced label distributions remains a significant challenge in natural language processing. tail labels, which are associated with very few training samples, typically exhibit weak predictive performance even when advanced transformer-based models such as bert are employed. this limitation hinders the reliable identification of rare but potentially valuable business opportunities within large-scale textual data. the present study aims to enhance tail-label performance by introducing an adjusted weighting strategy into the binary cross-entropy (bce) loss function. the proposed approach consists of two main components. first, a label-specific weight is calculated as the ratio of negative to positive samples for each label and then constrained within a predefined range to prevent excessive dominance of either frequent or rare labels. second, an optimal decision threshold is determined through grid search over the interval [0.1, 0.9], enabling improved balance between precision and recall across labels. experiments are conducted on an english multi-label dataset containing 1,000 samples and 20 imbalanced labels, with label frequencies varying from 180 to 5 instances. the data are split into 80% training and 20% testing sets. results show that the weighted bert model achieves a hamming accuracy of 0.623, a macro-f1 score of 0.091, and a tail-label f1 score of 0.025. notably, using only one twenty-eighth of the baseline dataset size, the model retains approximately 70% of baseline accuracy while improving tail-label performance compared to the unweighted setting. the method offers a practical, computationally efficient solution for data-scarce and resource-constrained environments.