Statistical Language Model for E-Commerce Content Categories

Abstract

Proper classification of e-commerce content is necessary for successful product discovery, catalogue management, and consistent navigation. Product titles and descriptions create certain obstacles owing to abbreviations, incomplete sentences, overlapping terminology, and promotional language. The study proposes a field-aware statistical language model incorporating word probabilities and sequences specific to particular categories and character-level patterns. The research evaluates whether separate treatment of titles and descriptions can improve classification without using sophisticated architecture. The evaluation procedure includes a corpus of 12,000 English-language product records distributed into six categories with training, validation, and test sets. Preliminary results demonstrate an accuracy of 89.50% in comparison with 85.83% of a word unigram baseline and 91.17% of a contextual transformer comparator. The category-level analysis reveals that clothing is the best-performing category and that sports products are the most difficult category. The proposed framework provides an interpretable way to analyze the relationship between lexical evidence, classification accuracy, and catalogues' review requirements.