Журнал «Современная Наука»

Russian (CIS)English (United Kingdom)
 MOSCOW +7(495)-142-86-81

METHODS FOR ASSESSING DEEP SEMANTICS EXTRACTION BY LARGE LANGUAGE MODELS

Gromozdov D. R.  (postgraduate student, BMSTU)

Gapanyuk Yu. E.  (Candidate of Technical Sciences, Associate Professor, BMSTU)

Afanasyev G. I.  (Candidate of Technical Sciences, Associate Professor, BMSTU)

The widespread use of textual data and the gradual increase in the complexity of natural language processing tasks that can be solved with large language models leads to the discovery of non-trivial facts, or underlying semantics, in the data. Creating false facts to produce an answer for the user is also a common characteristic of language models, known as hallucinations. It is necessary to develop a system for evaluating the truth of a fact obtained from a model. The aim of this paper is to identify a metric suitable for the evaluation of deep semantics. A statistical semantic content estimation method and a method based on machine learning were applied to the output of a large language model, which was tasked with identifying non-trivial fact from texts. The possibility of combining the approaches, to obtain a more accurate and interpretable metric, is discussed. The conducted research defines the limits of applicability and interpretability of modern metrics for the tasks of semantic analysis solved by means of large language models. We propose possible ways of solving the problem of selecting a semantic metric based on the widely spread method of intelligent agents, vector databases used for retrieval augmented generation, and other software solutions for analyzing the truth of facts or their adequacy to the information contained in the corpus of texts.

Keywords:large language models, metric, evaluation models, deep semantics, metagraph.
 

Read the full article …

Citation link:
Gromozdov D. R., Gapanyuk Y. E., Afanasyev G. I. METHODS FOR ASSESSING DEEP SEMANTICS EXTRACTION BY LARGE LANGUAGE MODELS // Современная наука: актуальные проблемы теории и практики. Серия: Естественные и Технические Науки. -2025. -№07. -С. 73-79 DOI 10.37882/2223-2966.2025.07.10
LEGAL INFORMATION:
Reproduction of materials is permitted only for non-commercial purposes with reference to the original publication. Protected by the laws of the Russian Federation. Any violations of the law are prosecuted.
© ООО "Научные технологии"

Terms Publications

Issues of the journals (Archive)

ECONOMICS AND LAW
HUMANITIES
NATURAL AND TECHNICAL SCIENCES
COGNITION

Серия - Natural and Technical Sciences