How to Respond When Reviewers Ask for More Experiments
How to Respond When Reviewers Ask for More Experiments You Can't Do
Requests for additional experiments are among the most stressful reviewer comments because they appear to leave no options: you either do the experiment (expensive, slow, sometimes impossible) or you refuse (risking rejection). The real situation is more nuanced. Most experiment requests can be handled without new data collection — but only if you understand what the reviewer is actually worried about.
Understand the Request Before Responding
Reviewers rarely ask for experiments because they want more experiments. They ask because an experiment represents, in their mind, the cleanest way to resolve a specific concern. The concern is usually one of the following:
- Confound uncertainty: "Another variable might explain your results."
- Mechanism uncertainty: "You've shown X correlates with Y but not why."
- Generalizability doubt: "This might only work in your specific setup."
- Construct validity concern: "Your measure might not be measuring what you think."
- Effect robustness question: "This could be an artifact of your analysis choices."
If you can identify which of these is the underlying concern, you have several paths forward that do not require new data collection.
Response Strategy 1: The Reanalysis
Many experiment requests can be addressed with a reanalysis of your existing data.
Example:
> Reviewer 2, Comment 4: The reported effect might be entirely explained by baseline differences in anxiety between groups. The authors should run a separate experiment with pre-matched groups to rule this out.
> Response: We appreciate the reviewer's concern about baseline anxiety as a potential confound. While we cannot run a new experiment within the revision window, we have addressed this concern directly with a reanalysis. We reran the primary analysis as an ANCOVA with baseline anxiety (BAI score, measured at session 1) as a covariate. The experimental effect remained significant and of similar magnitude (F(1,84) = 18.2, p < .001, partial η² = 0.18, vs. original η² = 0.21). This analysis is now reported in Section 3.1 (page 12). The finding suggests the effect is not explained by pre-existing anxiety differences, addressing the reviewer's core concern without requiring a new matched sample.
Key elements: restate the concern, explain what you actually did, give the numbers, cite where it appears.
Response Strategy 2: The Existing Literature Argument
If the reviewer wants you to demonstrate something that prior literature has established, you can argue from the literature rather than from new data.
Example:
> R1.6: It is unclear whether the paradigm the authors use is sensitive to the construct they claim to measure. A manipulation check experiment would strengthen this considerably.
> Response: The validity of the [paradigm] as a measure of [construct] has been established in prior work. [Author A, Year] used this paradigm to validate against [criterion measure] in n=... participants (r = .78). [Author B, Year] replicated this validity evidence in a clinical sample. We have added a paragraph to the Methods section (page 8, lines 3-12) reviewing this validation evidence and explaining why we consider the paradigm established for use here. We note that requiring original validation work on an established paradigm would expand the scope of this paper substantially, which we believe is beyond the present manuscript's focus.
The last sentence performs important work: it contextualizes the scope argument without being dismissive.
Response Strategy 3: The Robustness Check
When the reviewer questions whether your findings depend on specific analytical choices, a series of robustness checks can substitute for a new experiment.
Example:
> R2.2: The dependent variable operationalization is unusual. Results may be specific to this measure. The authors should replicate with a standard measure to demonstrate robustness.
> Response: We understand the reviewer's concern. While we do not have a matched dataset collected with the standard measure, we have run three robustness checks on our primary operationalization. First, we recalculated our primary outcome using the scoring approach of [standard reference], which yields comparable estimates (β = 0.43 vs. original 0.41). Second, we correlated our measure with the three-item short-form version of [standard scale] administered at session 3 (r = .71, p < .001), supporting convergent validity. Third, we conducted a sensitivity analysis excluding the 15% of trials where [potential artifact condition]; the effect size was unchanged (Cohen's d = 0.62 vs. original 0.64). These checks are reported in Section 3.3 and Supplementary Materials B. Together, we believe they address the robustness concern without requiring a new data collection effort.
Response Strategy 4: The Honest Scope Statement
Sometimes none of the above alternatives exists. You simply cannot address the concern empirically within this paper. In this case, honest scoping is the right move — but it must be substantive, not dismissive.
The weak version (avoid):
> "We agree this would be interesting but it is beyond the scope of the current paper."
The strong version:
> R1.4: This study uses an undergraduate convenience sample. The proposed mechanism should be tested in the relevant clinical population before any translational implications can be discussed.
> Response: We agree that a clinical sample would provide stronger evidence for the translational implications we discuss. This study was conducted as a conceptual proof-of-principle to establish the basic phenomenon under controlled conditions — a standard first step before moving to clinical samples, where confounding from medication, symptom severity, and treatment context would complicate interpretation of initial null results. We have revised the Discussion to be explicit about this: we have removed language suggesting direct clinical applicability (pages 17-18) and reframed the translational implications as a research agenda rather than a clinical recommendation. A sentence noting the undergraduate sample as a primary limitation has been added to the Limitations section (page 20). We believe these changes appropriately scope the paper's claims to what the current evidence supports.
Note what happens here: you made real changes (removed overreaching language, added the limitation), which means you addressed the underlying concern (overclaiming) even though you could not fulfill the specific request (new sample).
Response Strategy 5: Pilot Data or Archival Data
If you have any related data — from a pilot study, an archival dataset, or a different project — that speaks to the reviewer's concern, even imperfectly, consider whether it can be included as exploratory or supplementary evidence.
Caveat: Do not run new exploratory analyses and report them as if they were pre-planned confirmatory tests. If data are exploratory or collected post-hoc relative to the main study, say so. Reviewers who discover undisclosed outcome switching are a more serious problem than reviewers who asked for an experiment.
When to Say No Without an Alternative
Some requests are genuinely outside what a revision can accommodate, and it is acceptable to say so clearly — with a clear explanation.
Example:
> R3.1: The study needs to be replicated in three additional countries to establish cross-cultural validity before publication.
> Response: Cross-cultural replication is an important direction, but conducting equivalent studies in three additional countries within a revision window would require resources and time that are not available to us, and would constitute a substantially new research program rather than a revision of this paper. The current study is appropriately framed as an initial investigation of [phenomenon] in [population], not as a claim of universal cross-cultural validity. We have ensured this framing is clear throughout the manuscript, particularly in the Introduction (page 3) and Discussion (pages 17-18). We note that this limitation is now explicitly acknowledged (page 20) and that our detailed methods description facilitates replication by other groups.
The critical element: you have made a substantive change (tightened the framing) that addresses the underlying concern (overclaiming cross-cultural generality), even though you refused the specific request.
Do/Don't Summary
Do:
- Identify the underlying concern before deciding how to respond
- Offer a reanalysis, robustness check, or literature argument as an alternative
- Make real changes to scope and framing when you cannot do the experiment
- Be specific about what you did and where it appears
Don't:
- Say "beyond scope" without explaining why or making an alternative change
- Promise an experiment in a future paper (reviewers know this is not binding)
- Run exploratory post-hoc analyses without disclosing their status
- Dismiss the request without acknowledging what concern prompted it
Getting Your Response Drafted
Responses to experiment requests often require the most careful framing — you need to refuse without appearing dismissive, and to propose an alternative without appearing evasive. [Reviewer2](/#demo) can draft a baseline response to any reviewer comment, including experiment requests, which you then revise for accuracy and specificity.
For the broader revision process, see the [revise and resubmit strategy guide](/guides/revise-and-resubmit-strategy) and the [complete response-to-reviewers template](/guides/response-to-reviewers-template) for how to structure your response letter.