Retrieving, Classifying and Ranking Relevant Scientific Papers Using Machine Learning (ML) Algorithms

Abstract

The most important phase of research is literature review. Scientists must identify the gaps and challenges on the specific area of research, and get already generated knowledge to base further exploration. The problem is how and where to find the already generated literature that guarantee to ascertain the state of the art on that specific domain. A feasible literature review consists on locating, appraising, and synthesising the best empirical evidences in the pool of available publications, guided by one or more research questions. But it is not assured that searching electronic databases will retrieve the most relevant content. The existing search engines recommend the articles by only looking for the occurrence of given keywords. In fact, the relevance of a paper should depend on many other factors as adequacy to the theme, specific tools used or even the test strategy, making automatic recommendation of articles a challenging problem. Our approach will allow researchers to browse huge databases quickly retrieve appropriate publications on the specific by using Machine Learning (ML) techniques. The proposed solution automatically classifies and prioritises the relevance of scientific papers. Using previous samples, manually classified by domain experts, a Naive Bayes Classifier is applied to get predicted articles from real world journal repositories. The model developed and tested revealed that it can substantially recommend, classify and rank the most relevant articles of a specific scientific field of interest. The authors they achieved 98.22% of accuracy in recommending articles that are present in an expert Classification list, indicating a good prediction of relevance. Authors envisage to expand their model to accept user’s filters and other inputs to improve predictions.