A research paper evidence matrix with source links
October 16, 2026 · Oyren AI Team (AI-written)
Recommendation: An evidence matrix is useful when every row ties one claim to an exact passage, the conditions behind that passage and a record of what was checked. A table of paper titles and generated summaries cannot show whether the evidence supports the final paragraph.
The mind map shows one row of the matrix. The yellow card is the sentence the matrix supports. The four branches are the field groups each row carries. The pink card is the synthesis paragraph, which only draws on rows whose status says the source was checked.
This guide covers exploratory literature reading. A formal systematic review has its own reporting requirements, listed in the PRISMA 2020 expanded checklist. A spreadsheet alone is not evidence that such a review took place.
One row, one unit
The first decision is what a row represents: a paper, a study, an experiment or a single claim. The choice changes how rows can be counted and compared, so it goes in a note at the top of the matrix.
For a methods comparison, one row might represent one evaluated configuration. A paper that tests three configurations then produces three rows that share one source identity. Those rows are not three independent studies, and the note should say so.
The matrix note records three things:
- Row unit: for example, "one evaluated configuration".
- Question: the sentence or research question the matrix serves.
- Scope: which sources are in, which are out, and why.
Fields that tie a claim to a passage
A starting set of columns:
| Field | What goes in it | Example entry |
|---|---|---|
| Source identity | Title, identifier and the version read | Paper A, arXiv ID, v2 |
| Claim examined | The draft sentence this row may support | "Method B is faster on long inputs" |
| Passage location | Section, table, figure or equation | Section 4.2, Table 3 |
| Method and setting | Procedure and the conditions of the result | Dataset X, inputs over 4k tokens, baseline A |
| Reported finding | What the source states, close to its wording | "2.1x lower latency than baseline A" |
| Interpretation | What the reader takes from it for this project | Supports the claim for long inputs only |
| Limitation or gap | What is still unknown or unchecked | Short inputs not tested |
| Status | The check completed so far | passage checked |
The example entries are illustrative and do not describe a real paper.
Two columns carry most of the weight. Reported finding and Interpretation are kept apart, so a reader can see where the source ends and the reading begins. When a paper's discussion section speculates about future uses, that text stays in the interpretation column or the gap column. It does not become a reported finding.
The version recorded in source identity matters for preprints. An arXiv paper can change between v1 and v2, and a table number cited from one version may not exist in the other.
A coding guide tested on a few papers
Column names are not enough on their own. Two people, or one person a month later, will fill "method" differently unless a short guide defines it. A coding guide states, for each column, what qualifies as an entry. For example:
- Method and setting: must name the procedure and the evaluation data.
- Limitation or gap: marks whether the limitation is stated in the source or raised by the reader.
The Cochrane data-collection chapter describes piloting extraction forms and resolving disagreements in intervention reviews. An exploratory matrix can borrow the practical part: fill the columns for a small, varied set of papers first, fix the definitions that caused confusion, then extend the table. This is an adaptation for informal reading, not an implementation of Cochrane methods.
Comparable conditions before combined conclusions
Two values in the same column are not automatically comparable. In a computing comparison, different evaluation data, task definitions or baselines can make two latency numbers answer different questions. The Method and setting column exists to expose that.
When the conditions do not match, the row status stays unresolved and the comparison is not made. Averaging two values whose settings have not been reconciled produces a number with no clear meaning.
The SWiM reporting guideline covers how systematic reviews group studies and report synthesis without meta-analysis. It is a reference for that formal setting. A qualitative summary table does not meet SWiM by having similar headings.
Status labels instead of certainty grades
A personal confidence score can look like a formal certainty grade. Cochrane's chapters on risk of bias and certainty of evidence describe structured methods for those judgments. An exploratory matrix does not apply them, so it should not borrow their names.
A status column that records the work done is more accurate:
| Status | Meaning |
|---|---|
abstract read | Only the abstract has been read |
passage checked | The cited passage was found and matches the row |
numbers compared | Values were checked against the source table or figure |
unresolved | Conditions differ or information is missing |
These labels describe effort, not quality. A numbers compared row can still come from a weak study. The limitation column is where that concern goes.
AI-drafted cells and the review record
An AI assistant can draft cells from a supplied PDF. The prompt should ask for the passage location with every value and allow the answer "not stated". A drafted cell stays a draft until a reviewer has compared it with the passage.
The map reads left to right, then down. The source passage (yellow) produces a drafted cell (blue), which the reviewer checks (green). Each check ends in one of three gray outcomes:
- Accepted: the draft matched the passage, and the status becomes
passage checked. - Corrected: the draft was wrong. The row keeps the draft value, the corrected value and the reason, so repeated errors become visible.
- Unknown: the source does not state the value, and the cell stays empty rather than filled with a guess.
Accepted and corrected rows feed the synthesis. Unknown cells reach the synthesis only as stated gaps. A sentence that depends on an unresolved comparison keeps that limit in its wording. The AI literature review limits post covers where AI reading tends to go wrong.
The matrix next to its sources
The matrix is most useful when each row links back to the passage it cites. Oyren's PDF-to-note guide shows a source PDF and a note kept side by side, so a row's location field leads straight to the highlighted passage. Oyren does not have a dedicated evidence-matrix feature or a reference-manager integration. The matrix is a Markdown table inside a research note, next to the papers and the open questions.
Frequently asked questions
Does an evidence matrix count as a systematic review?
No. A systematic review reports its protocol, search strategy, screening process and appraisal methods, as itemized in the PRISMA checklist. The matrix described here is a tool for careful exploratory reading.
Can one paper produce several rows?
Yes, when the row unit is smaller than a paper, such as one evaluated configuration. The rows share a source identity and should not be counted as independent studies.
How many papers should the coding guide be tested on?
Enough to cover the variety in the reading list: at least one paper from each method type expected in the matrix. The test is finished when new papers stop exposing ambiguous column definitions.