MODERN TEXT CLASSIFICATION METHODS BASED ON MACHINE LEARNING ALGORITHMS

Authors

         Nizamitdinov A.I. – Philosophy Doctor(PhD) in Statistics, The lecturer of Digital Economy department, Polytechnic institute of Tajik technical University, Khujand, Republic of Tajikistan,ahlidin@gmail.com

         Jaborova Sh.A. – The 4^th year student of specialty technology and information systems in economy, Polytechnic institute of Tajik technical university, Khujand, Republic of Tajikistan,jaborova232000@gmail.com

Annotation

        The article deals with the application of machine learning in text analysis tasks. Particular attention is paid to those approaches that can be effectively used to extract information from natural language text. Various stages and levels of text analysis and the possibility of using machine learning methods on each of them are considered. When solving text classification problems, machine learning algorithms are used, such as logistic regression, K-nearest neighbors, decision trees, Random Forest, boosting algorithms (CatBoost, XGBoost), Linear Discriminants and Neural Networks. It is concluded that the solution of text classification problems based on these algorithms has a fairly high score compared to other approaches to classification problems. To evaluate the efficiency of the algorithms used, used an Confusion Matrix, which shows the accuracy of model prediction and the degree of errors in classification problems. The machine learning procedure demonstrated an efficiency of about 60-86% in the analysis of parts of speech in sentences of various thematic orientations (using the example of the Russian language) using data from the information site lenta.ru.

Key words

  algorithm, machine learning, text classification, data processing, text analysis, text preprocessing