Natural evaluation: The reconstruction of the academic Publication evaluation model driven by artificial intelligence
Author:
Affiliation:

Clc Number:

TP18;G311

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    As the basis for the evaluation of scientific research projects, talents, journals, and institutions, academic publication evaluation is an important part of the academic evaluation system and a key step in scientific research management. The current academic publication evaluation model has some disadvantages such as "black box of peer review", "ambiguous motivation of citation" and "indirect and one-sided evaluation data", which hinders academic innovation and also runs counter to the current strategy of building an independent knowledge system. The breakthrough development of AI in data, algorithms, and arithmetic power in recent years, especially the increasingly mature application of discriminant and generative models in machine translation, text classification, text summary, dialogue system, etc. has brought a great opportunity for innovation of the traditional academic publication evaluation model and provided a possibility to realize a more scientific, pluralistic, and intelligent academic publication evaluation. From the analysis of the inadequacy of the existing evaluation models and the fit of AI applied to it, the paper proposes a new model of academic publication evaluation, namely Natural Evaluation. Natural Evaluation is an integration of AI and academic publication evaluation, which is both a critical inheritance and a breakthrough of the traditional evaluation models such as peer review, bibliometrics, webometrics, and alternative metrics. Specifically, Natural Evaluation is an evaluation model based on the fully quantified academic trace data naturally generated by the academic community in various scholarly activities, and AI technology is applied to the semantic understanding and automatic analysis of the knowledge content of academic results and academic trace data of the community, to dynamically form evaluation judgments and serve knowledge innovation. The paper discusses the technical logic, value compliance, and prospect of Natural Evaluation driven by AI from the doctrinal level. In terms of technical logic, Natural Evaluation is based on the naturally generated trace data from various scholarly activities, and the algorithm supports the intelligent extraction of semantics in the data and generates evaluation judgments, and enhances the accuracy and efficiency of evaluation through arithmetic power empowerment. In terms of value compliance, Natural Evaluation upholds the value principles of quality first, impartiality, and comprehensiveness, and seeks to break out of the trap of "digital discipline", and break the shackles of "human favor dominance" and "one-sided evaluation". In terms of the future outlook, Nature Evaluation presents a picture of responding to the trend of the open science era, optimizing the ecological environment of academic innovation, and promoting the construction of independent knowledge systems. Finally, although the technical logic, value compliance, and prospect of Natural Evaluation have been systematically discussed from the theoretical level, and it can be theoretically proved to present more scientific evaluation results, the specific difficulties of AI technology implementation, academic community evaluation incentive, human-machine relationship coordination, and evaluation empowerment of different subjects are yet to be further researched in the future.

    Reference
    Related
    Cited by
Get Citation

杨红艳,卢思佳,徐拥军.自然评价:人工智能驱动下的学术成果评价模式重构[J].重庆大学学报社会科学版,2023,(4):101~114

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:
  • Revised:
  • Adopted:
  • Online: September 08,2023
  • Published: