Linked Open Literature Review using the Neuro-symbolic Open Research Knowledge Graph

The way scholarly knowledge and in particular literature reviews are communicated today rather resembles static, unstructured, pseudo-digitized articles, which are hardly processable by machines and AI. This demo showcases a novel way to create and publish scholarly literature reviews, also called semantic reviews. The neuro-symbolic approach consists of extracting key insights from scientific papers leveraging neural models and organizing them using a symbolic scholarly knowledge graph. The food information engineering review case study will allow participants to see how this approach is implemented using the Open Research Knowledge Graph (ORKG). The real-time demo will allow participants to play with the ORKG and create their own living, semantic review.


INTRODUCTION
The scientific community of today faces the problem of scientific paper overload [1,7].There is an increasingly large number of currently 3 million papers published every year in addition to the approx.200 million ones already published.This gives rise to the research question: "How can we provide a reliable and living scientific knowledge base that empowers researchers to query, synthesize and analyse the vast body of scholarly knowledge?"Traditional syntheses are performed by researchers collecting relevant research contributions, synthesizing them, and publishing them as unstructured, static literature review articles.However, these traditional reviews are static and updated only by creating new articles, thus rendering them obsolete shortly after publication due to the continuous daily output of new research, especially evident in fast-evolving fields like web technology.Platforms such as arXiv 1 , HAL 2 and research square 3 provide a way to link several versions of papers.However, these are still static resources generally in PDF, the new version is only linked to the previous version and not to key insights provided in the paper.
Similar to the transition from static web pages (Web 1.0) to more dynamic web technologies (web 2.0 or 3.0), the current practice of literature reviews (Review 1.0) is poised for a transformation.We introduce an innovative approach to literature reviews, termed Review 3.0 or semantic review, which envisions a network of interconnected insights drawn from scientific papers.This approach necessitates the 'semantification' of scientific literature, wherein key findings are not only extracted but also interconnected within a knowledge graph, paving the way for applications like question answering, semantic search, information retrieval, educational tools, and more [2,3].On the one hand, the semantification of scientific knowledge allows for recording, indexing, and disseminating semantic information and facilitates the answering of research questions.On the other hand, the semantification of key insights helps researchers to save time due to the condensed representation of the findings.
During this demo, the participants will learn how scientific papers can be semantififed using Open Research Knowledge Graph and used to write linked open literature reviews.Actually, ORKG allows researchers to organize scholarly knowledge involving the research domain, research problem, research methodology, methods, models, algorithms, processes, data source, data sets, tools, evaluation measures, results achieved, limitations of the research, future directions, etc. in a KG, making them comparable [1,8].ORKG proposes a new paradigm for writing literature reviews or related works.It consists of linking literature reviews to the semantic description of related work stored in a scholarly KG.This semantic description consists of ORKG resources such as the author's research contributions, comparison tables, smart reviews, etc. published using a Creative Commons license.Once written, this review can be published in any journal or conference.The main advantage of this approach is the fact that the research community will be provided with dynamic resources such as comparison tables and smart reviews which can continue to be updated with the evolution of the domain.These dynamic resources, which are open to the research community, can be updated as new scientific papers addressing the research problems or research domains are published, forming a linked open literature review.
This demo includes two parts: (1) during the first part, the food information engineering review case study will be presented.The participants will browse the resources created and being used for food information engineering review; (2) during the second part, the participants will have a chance to play with an ORKG testing environment 4 .This consists of organizing key insights extracted from scientific papers of their domains into research contributions, comparing these research contributions, and writing a linked open review.
In the rest of this paper, the Open Research Knowledge Graph is presented in Section 2, the description of the demo in Section 3, and the conclusion in Section 4.

OPEN RESEARCH KNOWLEDGE GRAPH
This section presents an overview of ORKG (Section 2.1) and the main features that will be used during this demo (Section 2.2).

Overview of ORKG
ORKG is an open research infrastructure empowered by Artificial Intelligence and designed to acquire, publish, and process structured scholarly knowledge published in the scholarly literature [1,8].It is built according to the principles of Open Science, Open Data, and Open Source [10].All the data ingested in ORKG are freely available on the Web and via API, dump, and integrations, can be identified by a Uniform Resource Identifier (URI), and can be accessed via HTTP or using a software library 5 .ORKG entities are linked to resources in external KG such as Wikidata, DBpedia, etc.
The ORKG is a neuro-symbolic knowledge graph designed to semantically structure and interconnect scholarly information, blending the strengths of symbolic AI with neural network approaches.In this hybrid model, symbolic AI is used to represent and reason over structured scientific knowledge, such as research findings, methodologies, and concepts, in a formal, interpretable manner.Meanwhile, neural network techniques are applied to enhance the graph's capabilities, such as improving information retrieval, recommendation systems, and the automated extraction and linking of knowledge from unstructured text.This combination allows the ORKG to provide a rich, dynamic, and semantically interconnected repository of scientific knowledge, facilitating advanced scholarly communication and analysis.
To date, ORKG comprehensively describes more than 25,000 research papers addressing more than 6,000 research problems (in 700 research fields), resulting in more than 1,500 state-of-the-art comparisons, 400 templates, contributed by 1,600 users, and 80 smart living reviews6 .

ORKG features used during the demo
To facilitate the demo, we will demonstrate various features of ORKG with a particular focus on the AI assistance functions.

Add papers. ORKG represents an article with the following two parts [1]:
• Article metadata: The article metadata involves the bibliographic information such as article title, authors, journal, book title, etc., and will be automatically retrieved and added from Crossref when a DOI is provided; • Semantic description of the research contribution: The semantic description of research papers consists of the annotation of these papers with key insights extracted from them and organizing these elements into research contributions.This allows us to put the paper in a machine-readable form following the RDF paradigm.In ORKG, each paper consists of at least one research contribution that addresses at least one research problem and is further described with contribution data including materials, methods, implementation, results, or other key insights.These contributions can be compared between them or by other contributions from other papers in an ORKG comparison table [8].
To semantify a paper, ORKG provides users of a wizard 7 to add papers in the ORKG system and create research contributions.Once the paper is added, ChatGPT is used for recommending properties and paper annotations to users.They can decide to use the ChatGPT suggestions directly or to refine them before using them.This allows users to get started with the process of describing a structured contribution of a paper.

Templates.
Templates are used during the description of research contributions to facilitate data entry and ensure comparability.To build templates, classes, and properties allowing to describe research contributions of the domain should be identified.An example of a template is a template used to describe food images dataset 8 .

Comparing research contributions.
The structured content descriptions of scientific papers into research contributions is done in such a way that the contribution becomes comparable with other contributions addressing the same research problem [8].To make it easy for researchers to compare research contributions, ORKG provides comparison table wizard 9 .Given that tabular research contribution comparisons are valuable resources, they can be published with a DOI, exported in various formats such as RDF, LaTeX, PDF, CSV, and integrated into a living dynamic literature review.The comparison table link can be shared with other researchers so that they can improve the comparison in a crowd-sourcing manner by correcting errors or adding missing information.An example10 is presented by the Fig. 1.

Smart Reviews. ORKG provides a "What You See Is What
You Get (WYSIWIG)" editor11 for helping researchers to write reviews [9].This editor facilitates the searching and linking of this review with all the resources created during the semantification of scientific papers of the domain.For instance, a smart review can be linked to a comparison table, to research contributions, ontology/templates, etc.If new literature is published, it is easy to continuously expand the comparison tables and smart reviews, which thus continues to reflect the state-of-the-art of the domain.To this end, a whole community of researchers can gather and collaboratively build the state of the art of a research problem.To make this easy, tabular contribution comparisons and smart reviews are versioned so that all changes can be discussed by the professional community, updated, and new revisions published.

DEMO
The demo will consist of (i) demonstrating how the food information engineering review is currently being done using ORKG (see Section 3.1); (2) a real-time demo (see Section 3.2).

Food Information Engineering case study
Food information engineering [4] relies on statistical and AI techniques for collecting, storing, processing, diffusing, and putting food information in a form exploitable by humans and machines.A huge number of research papers have been published in the domain.However, these papers are scattered on the Internet in different formats and are difficult to exploit.
To solve this problem, ORKG is currently being used to organize these data, ensuring high-quality standards.Unlike the current state-of-the-art on the subject [4][5][6] which provides static resources in the form of HTML or PDF documents, the ORKG approach provides dynamic resources stored in a KG which will be continuously updated by the researchers of the domain.Resources created and being used to date involve around 230 papers, 11 templates, 65 comparison tables, and 9 reviews.Fig. 2 presents an excerpt of these resources.During this demo, the attendees will be able to browse through these resources.
Templates.The templates are made as generic as possible to facilitate their reuse for other purposes.Fig. 2 presents some templates for documenting papers related to retrieval systems, recognition systems, methodologies, methods, and tools for ontologies and knowledge graph construction, image datasets, and question answering.These templates were used to describe food recognition systems, food retrieval systems, ontologies, knowledge graph construction, food image datasets, and food question answering.
Comparison tables.Comparison tables related to different research problems of food information engineering are also presented in Fig. 2. In addition, comparisons of food knowledge graphs (see Fig. 3) are also provided.These comparisons compare methodologies, methods, tools for food KG construction, and food KG integration in software.The fact that the food KG is using an ontology, the knowledge source, the application domain, etc. is highlighted.
Smart Reviews.Finally, smart reviews presenting an overview of the different topics of food information engineering research are provided.Currently, nine smart reviews are provided.Food information engineering [4] introduces food information engineering, the research methodology being used to curate the observatory and link to collecting, organizing, processing, and using food information.

Realtime demo
During the real-time demo, the participants will play with ORKG and create linked open reviews.They will be invited to identify three to four key insights from at least two scientific papers.Therefore, the participants will learn how to create templates and use these templates to annotate papers.Research contributions obtained will be used to create comparison tables and smart reviews.For this second part of the demo, participants require a laptop and wifi connection.

CONCLUSION
This demo presents how a novel living semantic literature review can be done by extracting, organizing, and comparing research contributions of authors using Open Research Knowledge Graph.Thereafter, these resources can be used to write smart reviews.These resources break down information silos that exist between various papers and can be used to write and publish literature reviews in conferences or journals.Peer review can be used to make the paper trustful and online resources continue to be updated when new papers in the domain are found or published.This facilitates the extension of the literature review and allows easy updates.

Figure 1 :
Figure 1: Comparison of machine learning models used to estimate calorie from food image

-Figure 2 :Figure 3 :
Figure 2: An overview of resources being used to semantify food information engineering research papers