Systematic Review Management Platforms Powered by Artificial Intelligence: A Narrative Review
1. Department of Ophthalmology, Eye Specialty Hospital, Amman, Jordan. 2. The University of Jordan, Amman, Jordan.
Abstract
Systematic reviews are essential for evidence-based clinical decision-making and identification of research gaps. However, conducting a systematic review is often time-consuming and requires careful coordination. In response to these challenges, several digital platforms have been developed to improve efficiency, transparency, and collaboration throughout the review workflow. This narrative review provides an overview and comparison of three systematic-review management platforms: Rayyan, Covidence, and AIPRA. Rayyan is a widely used platform that supports reference import, deduplication, blinded screening, conflict resolution, full-text management, data extraction, and AI-assisted prioritization. Covidence offers a highly structured workflow for screening, full-text review, extraction, quality assessment, and PRISMA documentation, with strong support for team-based systematic reviews. AIPRA is an AI-augmented platform designed to support a broader review process, including question formulation, search organization, screening, extraction, evidence synthesis planning, and manuscript drafting. Overall, systematic-review platforms should be viewed as supportive tools that enhance workflow efficiency, documentation, and consistency while maintaining the central role of human oversight in evidence synthesis.
Body
Introduction
Systematic reviews are fundamental to clinical decision-making, the identification of research gaps and guideline development. They aim to synthesize and assess all relevant evidence addressing a focused research question by using clear and reproducible methodology. (1,2) The credibility of a systematic review depends on methodological transparency. Reviewers must clearly define and report eligibility criteria, search strategies and screening decisions. The Cochrane Handbook emphasizes that systematic reviews should be planned, documented, and reported in a way that allows readers to understand and appraise each stage of the process.(3) Despite methodological clarity, systematic reviews remain difficult to conduct efficiently. Searches across databases may return thousands of citations. Many are irrelevant, duplicated, poorly indexed, or incompletely described. Screening these records requires sustained attention and consistent interpretation of eligibility criteria. Data extraction, risk-of-bias assessment, synthesis, meta-analysis, and manuscript writing require accuracy, methodological judgment, coordination and clear documentation. Many platforms emerged in response to this workload. Earlier review workflows often relied on reference managers, spreadsheets, shared folders, and email-based decision tracking. Although these methods remain usable, they can be inefficient and difficult to audit. These platforms now support citation management, duplicate detection, screening, reviewer blinding, conflict resolution, extraction, reporting, and export. Some platforms have also incorporated machine learning and artificial intelligence to assist with automated screening, extraction, summarization, and writing.(4,5) Automation may reduce repetitive work, shorten review timelines and improve consistency in selected tasks. However, systematic reviews are vulnerable to errors introduced early in the workflow and a missed eligible study, an incorrect exclusion, or a fabricated extracted value can affect the conclusions of the review. In this article, we will provide an overview of existing platforms that support systematic-review workflows and discuss their potential role in improving the way systematic reviews are conducted.
Rayyan
Rayyan is primarily a web-based systematic review management platform designed to support multiple stages of the evidence synthesis workflow.(6) It also has a mobile application that serves as a complementary extension of the web platform, allowing reviewers to continue selected review tasks on mobile devices. However, the web platform remains the central component of its workflow. The review workflow begins with reference import as the platform does not connect directly to bibliographic databases; instead, reviewers must first run searches in external sources such as PubMed, Scopus or other databases, export the results in a compatible format and then upload the files into Rayyan. The platform supports several reference file formats, including RIS, EndNote Export, BibTeX, CSV, PubMed NBIB, PubMed XML, Web of Science CIW, and PDF metadata extraction.(7) It can also process supported reference files contained in TXT, DOCX, ZIP, or GZ containers. Reviewers should ensure that database exports include full records and abstracts, because missing abstracts can reduce the performance of Rayyan’s AI-supported features, particularly Relevance Ranking and PICO extraction. If a source does not export in a standard format, reviewers can use reference managers such as Zotero, Mendeley, or EndNote as converters and then re-export the records in RIS or another compatible format before uploading them. Rayyan includes a structured deduplication workflow that begins after all references from different databases or sources have been imported into the review. (8) Users can initiate duplicate detection, after which Rayyan scans the dataset and reports the number of possible duplicate records identified. These records are then grouped under the “Possible Duplicates” facet, where unresolved duplicates can be filtered, reviewed and resolved manually. When duplicate records are opened, Rayyan displays them side by side with a confidence percentage, allowing reviewers to compare citation details before deciding whether to keep the left record, keep the right record or retain both records as non-duplicates. For subscribers, Rayyan also offers a Systematic Auto-Resolver, which allows more advanced and customizable deduplication. This function can apply exact-match criteria such as title, author, journal, year, pages, DOI number, and publication type, either individually or in combination. It can also use an overall similarity threshold and allow users to prioritize records from a specific imported dataset when duplicates are detected across multiple searches. However, because auto-resolution is irreversible once confirmed, Rayyan’s deduplication process still requires careful reviewer oversight. The platform also supports blind mode. (9) This function is enabled by default and prevents reviewers from viewing one another’s decisions while screening is ongoing. By limiting each reviewer to their own decisions, blind mode reduces the possibility that one reviewer’s judgment will influence another’s. Once independent screening is completed, the review owner can turn blind mode off to access the conflict filter, compare team decisions, resolve disagreements, and move the appropriate records to the next stage. This approach can be repeated across title-and-abstract screening, full-text screening and data extraction. Rayyan also notes that blind mode should be turned off before exporting references so that the export includes the full team’s decisions rather than only the exporting reviewer’s own decisions. Rayyan provides several options for organizing screening work across review teams. In the standard approach, all reviewers screen all records independently with blind mode enabled. It also supports data sampling, which allows review owners to create named subsets of references for screening. This function is useful for larger datasets because it helps distribute work across reviewers while maintaining a structured workflow. Samples can be generated from all references or from a filtered subset, and they may be created using several methods, including randomized selection, system ID, title, publication date, or relevance ranking score. Rayyan also offers a Min/Max Decisions filter, which allows teams to display records that have received fewer than a target number of final decisions. Another option is label-based division, in which references are grouped using labels such as reviewer group, database source, or screening batch. However, this method requires blind mode to be turned off because labels are not visible to other reviewers when blind mode is active. Therefore, for protocols requiring blinded independent screening, data sampling or the Min/Max Decisions filter would be more appropriate than label-based assignment. Rayyan supports full-text PDF management by allowing PDFs either to be uploaded as new references or attached to existing records. When uploaded as a reference, Rayyan attempts to extract citation metadata and create a new record automatically. When a record already exists, the PDF can be attached directly during screening, full-text screening, data extraction. It also supports bulk PDF attachment, Auto Match, and Auto-Retrieve PDF on paid plans, which can help link full texts to citations more efficiently. However, PDF matching and metadata extraction should be checked manually, especially when PDFs contain multiple DOIs or incomplete metadata. It also provides a structured data extraction stage for studies that have passed screening or full-text assessment. Reviewers can create customized extraction fields, define column titles, choose answer types such as free text, numbers, or predefined options, mark essential questions as required, and organize variables into sections. Full-text PDFs can be uploaded and reviewed within the extraction interface using the built-in PDF viewer. Rayyan provides AI-supported PICO highlighting and filtering to help reviewers identify key eligibility concepts during screening. However, the function depends on the availability of abstract text and should be used as a screening aid rather than as a replacement for independent eligibility assessment. Access is also limited to Advanced and Institutional plans. Rayyan’s Relevance Ranking is an AI-supported prioritization tool that predicts how likely undecided references are to be included based on previous screening decisions. The feature uses a support vector machine (SVM) classifier trained on title, abstract, and MeSH-term features, and assigns unscreened records to five rating levels ranging from “most likely to include” to “most likely to exclude.” Ratings require at least 50 prior screening decisions, including at least five included and five excluded records. Because ratings are suggestions rather than final decisions, Relevance Ranking should be used to prioritize screening order rather than replace reviewer judgment. Rayyan has further expanded its AI-supported functions through ResearchPilot, an institutional feature designed to support searching within the review environment, article screening, AI-based analysis, and data extraction. ResearchPilot includes several modes, such as Help Center Mode, Review Articles Mode, AI Reviewer, AI Analyzer, and Auto Extraction. Help Center Mode functions as an embedded support assistant that provides answers and links to relevant Rayyan guidance. Review Articles Mode allows users to load all or filtered articles for AI-supported exploration, including relevance scoring from 0 to 1 and extraction of top answers from the most relevant articles. The AI Reviewer function acts as an automated screening assistant that can become a member of the review team once activated. It screens articles according to the inclusion and exclusion criteria defined in the Overview stage and provides include or exclude decisions with explanatory notes. Its progress can be followed through review-level status updates. However, Rayyan indicates that blind mode must be turned off to view the AI Reviewer’s decisions directly in the article list. ResearchPilot also includes AI Analyzer, which provides criteria-based recommendations for selected articles. Unlike AI Reviewer, which can screen articles more broadly, AI Analyzer is used on selected records and generates include or exclude recommendations with justification notes explaining how the eligibility criteria were applied. Rayyan explicitly frames AI Analyzer as an assistive function rather than a replacement for reviewer decisions, meaning that human verification remains necessary. Finally, Auto Extraction allows Rayyan to extract structured information from articles after they have been moved to the Data Extraction stage and extraction questions have been created. When full text is available, the system analyzes PDFs and metadata to identify answers relevant to the predefined extraction questions. It can highlight the supporting text inside the PDF or article, show multiple sources when applicable, and allow reviewers to revise or add answers manually. The quality of Auto Extraction depends heavily on the clarity and specificity of the extraction questions. ResearchPilot is available exclusively under Institutional plans not to all Rayyan users. Despite its strengths, Rayyan has several limitations. First, access to some advanced functions, particularly AI-supported features, may depend on subscription level or institutional access. Second, Rayyan does not provide a fully dedicated risk-of-bias assessment environment with ready-made domain-based templates, built-in signaling questions, automated judgment rules, or direct generation of risk-of-bias summary tables and figures. Although reviewers may create custom extraction fields or tables to record risk-of-bias judgments, this approach requires manual setup and does not replace specialized risk-of-bias assessment tools. Third, Rayyan does not perform meta-analysis or quantitative synthesis, so reviewers who intend to calculate pooled effect sizes, assess heterogeneity, generate forest plots, or conduct subgroup and sensitivity analyses must use external software such as RevMan, R, Stata, or other statistical programs. Overall, Rayyan is a useful platform for managing key operational stages of systematic reviews, especially citation organization, duplicate detection, independent screening, conflict resolution, full-text assessment, collaboration, and data extraction. Its AI-assisted and workflow-management features may reduce reviewer workload and improve efficiency. However, it should be viewed as a review-management and screening-support platform rather than a complete end-to-end systematic-review solution.
AIPRA
AIPRA, https://aipra.co/, is a platform designed to support the review process through a stepwise, AI-augmented workflow. (10) Unlike tools that mainly focus on citation screening or reference management, AIPRA is structured around the full logic of conducting a systematic review, beginning with formulation of the research question and continuing through searching, screening, extraction, evidence synthesis and manuscript writing. Its workflow is organized so that each stage builds on the previous one, allowing the review protocol and later reporting outputs to develop progressively from the decisions made throughout the project. A central feature of AIPRA is its emphasis on protocol development through workflow execution. The platform begins by helping users formulate clear and well-scoped research question using the established framework PICO. This framing is then used to guide eligibility criteria, search concepts, extraction fields, synthesis planning, and reporting. As reviewers move through the workflow, AIPRA can incorporate these decisions into a living protocol document, helping maintain alignment between what was planned, what was executed, and what is eventually reported. AIPRA’s search workflow is organized around two linked components: concept development and database management. In the search area, users can define or add the main concepts derived from the research question, then connect those concepts to the literature sources that will be searched. In the database selection stage, AIPRA provides a dedicated Add Database interface where users can select from commonly used bibliographic sources, including PubMed, OpenAlex, Scopus, Web of Science, Embase, and the Cochrane Library. The platform also includes a Manual Retrieval option, which allows reviewers to upload records from other sources not listed among the predefined databases. After a database is added, AIPRA creates a separate database-specific import area. Each database page displays the number of imported articles, identified duplicates, and import batches, allowing users to track the contribution of each source to the overall review library. The platform also provides source-specific instructions for exporting and uploading records. For example, PubMed records can be imported using RIS, NBIB, or CSV files; Scopus and Web of Science support formats such as RIS and CSV; Embase supports RIS and CSV/Excel; Cochrane Library imports are handled through RIS files; and OpenAlex records can be exported through the OpenAlex web interface or API in formats such as RIS, CSV, or BibTeX. Users are still expected to run or validate searches in the original databases, export the results in supported formats, and upload them into AIPRA. The platform also provides practical guidance for database-specific constraints. For example, Scopus and Web of Science require institutional access and have export limits, meaning large searches may need to be exported in batches. AIPRA’s separate import tracking can help document these batches and preserve the audit trail for PRISMA reporting. The Manual Retrieval option further increases flexibility by allowing reviewers to upload files from sources outside the predefined database list using common formats such as RIS, CSV, BibTeX, or NBIB. After records are uploaded, AIPRA performs deduplication across databases, using identifiers such as DOI and PMID to detect duplicate records. However, as with other deduplication systems, identifier-based matching is not perfect, and the remaining duplicates could be resolved during screening or full-text review. AIPRA’s screening stage is built around explicit inclusion and exclusion criteria. The platform guides users through structured questions related to the review topic and uses the answers to help generate eligibility criteria aligned with the research question Screening is then conducted in two phases: title-and-abstract screening, followed by full-text screening for records marked as included or unclear. During screening, AIPRA can provide AI-assisted include or exclude recommendations with a short rationale linked to the stated criteria and available article information, however final decisions remain the responsibility of the reviewers. For full-text screening and data extraction, AIPRA extends its AI support beyond citation-level decisions. After studies are included, users can define extraction fields based on the research question, expected outcomes, study characteristics, population, intervention or exposure details, effect sizes, and other variables required for synthesis. AIPRA can help generate a structured extraction framework and may also propose extracted values directly from full-text articles. When AI-assisted extraction is used, the platform can provide supporting quotations or pointers to the source passage, allowing reviewers to verify the extracted value and maintain a clearer audit trail. AIPRA’s evidence synthesis module allows reviewers to organize extracted findings into structured synthesis sections before manuscript drafting. The platform can generate an evidence synthesis plan that summarizes the intended synthesis approach and divides the review findings into organized sections based on the research question, extracted variables, and outcomes of interest. These sections may cover study characteristics, main outcomes, secondary outcomes, subgroup analyses, safety outcomes, or other topic-specific domains depending on the scope of the review. This structure helps transform extracted data into a coherent synthesis framework rather than leaving reviewers to manually build the results section from an unstructured extraction table. Each synthesis section can include a short summary, specific writing instructions, and selected extraction fields that should be used to generate the synthesis. Reviewers can define which variables are relevant for each section, such as whether an outcome was assessed, the number of contributing studies, effect estimates, heterogeneity measures, statistical significance, subgroup findings, or additional notes. The platform also allows users to choose whether each section should produce narrative text, a summary table, or both. This creates a direct link between the extracted dataset and the planned synthesis output. AIPRA also allows users to add new synthesis sections or regenerate the synthesis plan when the review structure needs refinement. This is useful for reviews with multiple outcomes, heterogeneous study designs, or complex evidence maps, where findings need to be grouped and presented in a transparent way. In this sense, the evidence synthesis stage functions as a bridge between data extraction and manuscript writing. AIPRA’s manuscript module is designed to convert the structured outputs of the review workflow into a draft manuscript. Rather than beginning from a blank document, the platform uses information already entered or generated during previous stages, including the protocol, search strategy, screening outputs, extraction table, evidence synthesis plan, and selected references. The manuscript workspace is organized into major manuscript components such as title and authors, methods, results, and references. The methods section can be generated from project-level data, including the databases searched, search concepts, search strategy, deduplication results, study selection process, and other protocol-related information. When project data change after a section has been generated, AIPRA can indicate that the content may need updating, allowing users to regenerate or revise the text so that the manuscript remains aligned with the most recent version of the review. The results section is linked to the extraction table and evidence synthesis plan. Users can add blank results sections or create sections directly from the synthesis plan. AIPRA can then generate AI-assisted narrative text and summary tables using the selected extraction fields. For example, a results section may summarize study characteristics, outcome-specific findings, effect estimates, or other extracted variables in both written and tabular formats. The generated content can include citations connected to the underlying references. Overall, AIPRA’s writing module functions as a bridge between review execution and manuscript preparation. Its main strength is that it links manuscript drafting to the data and decisions produced during earlier workflow stages. Despite these strengths, AIPRA also has limitations that should be acknowledged. Because many of its functions are AI-assisted, outputs require human verification. Also, although AIPRA supports structured evidence synthesis planning and can organize extracted findings into narrative sections and tables, it does not currently perform formal statistical analysis or meta-analysis within the platform. In conclusion, AIPRA is an AI-assisted systematic review workflow platform that supports multiple stages of the review process, including question formulation, search organization, screening, extraction, evidence synthesis planning, and manuscript drafting. Its main strength is the integration of these stages into a connected workflow, allowing review decisions and extracted data to inform later synthesis and writing.
Covidence
Covidence is a web-based platform that helps with the management of systematic review, as it supports reviewer coordination, transparency of documentation and consistency of workflow through multiple stages of evidence synthesis. It has been developed based on methodological standards described in the Cochrane Handbook and PRISMA reporting guidelines, which means that Covidence follows a highly organized review process and is widely used for systematic reviews, meta-analyses, scoping reviews, and clinical guideline development projects. Unlike general reference-management or more flexible platforms, Covidence is designed to support a structured and standardized systematic review workflow. The review workflow in Covidence begins with importing references. Like Rayyan, Covidence does not perform literature searches directly within bibliographic databases. Instead, reviewers first conduct searches in external databases such as PubMed, Embase, Scopus, Web of Science, CINAHL, PsycINFO, or the Cochrane Library, then they export the results in supported formats before uploading them to Covidence. The platform supports citation formats including RIS, CSV, and PubMed XML, and integrates with reference-management software such as EndNote, Zotero, RefWorks, and Mendeley. If text conversions or database-specific reformatting are required, reviewers often use these external reference managers before import. Once references are imported, Covidence automatically starts a structured deduplication process to reduce redundant screening workloads, which means that Covidence tries to find and remove references. Duplicate detection occurs automatically during import by evaluating citation fields such as title, publication year, volume, and author information. References that are identified as duplicates are separated from the main screening dataset, while the first imported citation is retained as the primary reference. These records can later be reviewed manually through the duplicate-management interface, where reviewers may restore citations incorrectly identified as duplicates by selecting “Not a duplicate.” Covidence also allows reviewers to manually identify duplicates during title-and-abstract or full-text screening if duplicates were missed during automated processing. These manually identified duplicates are automatically recorded in the PRISMA flow diagram and review record. However, the deduplication algorithm is intentionally conservative to reduce the risk of removing unique studies, meaning that reviewer oversight remains important throughout the process to manage potential false positives. Covidence supports blinded screening for both title-and-abstract and full-text stages, meaning that reviewers can screen references without knowing what other reviewers think, so screening decisions remain unknown among team members until the required number of votes has been completed, reducing the possibility that one reviewer’s judgment may influence another’s decision. One strength of Covidence is its automated workload distribution system; if a protocol requires two independent votes per record, the platform automatically manages screening queues across reviewers and removes records from active queues once the required number of decisions has been reached. Once the initial screening is complete, the platform automatically routes conflicting records into a dedicated conflict-resolution stage. Disagreements are reviewed and differences can be resolved through team discussion or by a third reviewer. This consensus process is strictly maintained across both screening phases. During full-text assessment, which includes a built-in PDF viewer for side-by-side eligibility checking, reviewers must choose one clear reason for exclusion from a list they can customize. If reviewers choose different exclusion reasons for the same study, even if both agree on its exclusion, Covidence flags this as a conflict that must be resolved before the exclusion is finalized. This mechanism ensures transparent exclusion decisions and automatically creates the PRISMA flow diagram, while progress dashboards show the status of included, excluded, and unresolved studies in real-time. To make screening stages efficient, reviewers can use tools such as keyboard shortcuts, keyword highlighting, and filtering functions, along with customizable tags to group studies under specific labels like “ongoing study” or “awaiting classification.” For studies the move beyond full-text selection, Covidence offers an environment for structured data extraction and quality-assessment. The platform has two extraction systems, including the newer “Extraction 2.0” framework, which allows teams creates customized extraction forms with options like dropdown menus, text fields, numerical entries, and validation rules, all tailored to the review design. Additionally, to that, Covidence includes modules for assessing risk-of-bias based on established frameworks, including Cochrane Risk of Bias tools, CASP checklists, and the Newcastle-Ottawa Scale, these modules have signaled questions and sections for reviewer comments and justifications. In collaborative reviews, multiple reviewers may independently extract data and perform quality assessments before differences are resolved through built-in workflows in the platform. Covidence has increasingly included automation and machine-learning functions to improve screening efficiency. One of its main AI-supported tools is “Most Relevant” sorting. This system is an active-learning system developed in collaboration with the EPPI-Centre. This feature learns from reviewer screening decisions and reorders unscreened references according to predicted relevance, prioritizing eligible studies in the workflow. Covidence also provides automatic PRISMA flow diagram generation, dynamically tracking imports, duplicate removal, exclusion reasons, and final study inclusion throughout the whole review process. These AI-supported functions are meant to assist reviewer decisions and do not replace them, meaning that human oversight remains the priority. Despite its comprehensiveness in management workflow environment, Covidence has some limitations. First, access to many advanced functions requires institutional or paid access, this makes it less available for independent researchers compared with more accessible alternatives such as Rayyan. Second, although its structured workflow supports consistency and auditability, but may be less flexible for non-standard review designs, rapidly designed reviews, or highly customized workflows. Third, Covidence does not perform meta-analysis or advanced quantitative synthesis. While it supports export pathways compatible with RevMan workflows, reviewers must still use external statistical software such as RevMan, R or Stata to perform pooled effect-size calculations, heterogeneity assessment, and forest plot generation. Overall, Covidence is a comprehensive platform and highly structured for systematic review management. It supports reviewer coordination, documentation transparency, and workflow consistency across multiple stages of evidence synthesis all in one platform. It improves operational efficiency from citation screening through data extraction and risk-of-bias assessment within a single environment. However, it should be viewed as a workflow-management and evidence-synthesis support platform, not an analytical solution, for systematic reviews and meta-analyses.
Conclusion
Rayyan, Covidence, and AIPRA can improve the efficiency and transparency of systematic reviews. Each platform has different strengths: Rayyan is useful for screening and citation management, Covidence provides a structured workflow with risk-of-bias tools, and AIPRA supports a broader AI-assisted process from question development to manuscript drafting. However, these tools should support and don’t not replace reviewer judgment, human verification, and external software for formal meta-analysis. The main differences between Rayyan, Covidence, and AIPRA are summarized in Supplementary Table 1.