AAO-HNSF
  • Register Now
  • Posters
  • Meeting Coverage
  • Annual Meeting Program
  • OTO Experience
  • How to Claim CME
Topics
  • Meeting Coverage
  • Annual Meeting Program
  • OTO Experience
  • How to Claim CME
Resources
  • Register Now
  • Posters
Facebook iconInstagram iconTwitter X icon YouTube icon
Oct 4th, 2021

Peer Review of Database Studies: Rigors in Reams of Results

Peer review of research is fundamental to journal publication, and peer review of database studies is more complicated.


John H. Krouse, MD, PhD, MBA
John H. Krouse, MD, PhD, MBA

Peer review has been the standard approach to vetting scientific articles for centuries. It remains a critical aspect of all types of review and consideration for publication in medical and scientific journals. It is the subject that was examined by John H. Krouse, MD, PhD, MBA, Editor in Chief of Otolaryngology–Head and Neck Surgery and OTO Open, and Jennifer J. Shin, MD, SM, the Deputy Editor.

During the on-demand session “Peer Review of Database Studies: Rigors in Reams of Results,” Dr. Krouse explored the steps in peer review for articles in the AAO-HNSF journals, while Dr. Shin dug into the intricacies of large database studies.

“Research studies need to undergo an evaluation of its rigor, of its scientific content, of its ethical nature, to look at whether or not it is relevant and something the scientific community would value,” said Dr. Krouse. “It is an incredibly important part of the acceptance process for papers in science.”

In considering a manuscript, the reviewer follows this checklist:

1.  Relevance to mission: Is the information consistent with the mission of the journal?

2.   Internal validity: Are the study design, conduct, and analysis described in a manner that is unbiased, appropriate, and reproducible?

3.   External validity: Was the study sample chosen appropriately and described in adequate detail for results to be generalized?

4.   Level of evidence: Does this manuscript significantly improve the knowledge base beyond what is already published on this topic?

5.   Ethical conduct: Is the manuscript original, approved by an Institutional Review Board (if applicable) and free of undisclosed conflicts?

“Our journal has a very high rejection rate (roughly 80%) due to its high levels of expectation for the manuscripts,” he said. “Being rejected is not a reflection of the work you have done or the value, but perhaps more of the methods used.”

Papers that are not rejected then typically go through a major revision. The reviewers may have found significant but addressable concerns about study design, reporting of results, generalizability of the findings or composition of the manuscript, or significant but addressable ethical concerns. The revision may result in acceptance, rejection, or request for an additional revision.

After all the revisions have been completed and the reviewers and editorial team feel that these revisions are satisfactory, an acceptance decision will be given. 

Dr. Shin explored peer review of database studies, which has infinitely more intricacies to consider as a reviewer or an author. In particular, she focused on the database evaluation process, analytics performed, and database properties. Jennifer J. Shin, MD, SMJennifer J. Shin, MD, SM

Within that database evaluation process, there are three areas of focus: transparency, reproducibility, and validity, she said.  “Transparency and reproducibility mean that any cleaning processing and extraction of the data are made clear within the paper. It's typically not enough to just say you used a certain database. You should describe the specifics of how you used the database,” she said. 

“It is important to have a clearly defined time frame of reference as that provides a temporal anchor and reference point within the data for both exposures and outcomes,” she said. “That's important regardless of whether you're looking at a cohort from a certain starting point and following it forward or if you are looking backward.”

Using “An Original Report Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies” as a guide, she reviewed key parameters that would be used to implement a database study and a checklist of items to consider when reporting or reviewing it.

She also reviewed database analytics. She recommended that authors focus on a primary outcome.

“This often can be a challenge for authors when they are presented with a huge amount of very tempting data,” she said. “Even if you are looking at a very large data set, you ideally still want to focus in on one, or a very small number of, primary outcomes. If you don't, you can be subject to error for multiple comparisons,” she said.

She concluded by discussing database properties.

“Different databases have different strengths and weaknesses. If you understand these, you realize what the pros and cons and potential limitations of an analysis are,” she said.

This session is available on demand.


More Content
The team from Massachusetts Eye and Ear Infirmary/Harvard Medical School takes first place.
AAO-HNS 2021
Winners of the 14th Annual AAO-HNSF Academic Bowl
Oct 5th, 2021
Laura Brainard, MD
AAO-HNS 2021
The Right Diet Can Be Good for Physicians and Patients Alike
Oct 5th, 2021
(From left to right) Richard W. Waguespack, MD, Pratyusha Yalamanchi, MD, MBA, Michael J. Brenner, MD, and Hardy Shah
AAO-HNS 2021
APPs Changing the Landscape of Otolaryngology
Oct 5th, 2021
Hall of Distinction inductees (from left to right): Gavin Setzen, MD, K.J. Lee, MD, Eugene N. Myers, MD, FRCS Edin (Hon), and James L. Netterville, MD.
AAO-HNS 2021
Hall of Distinction Panel Tackles Past-Future Question
Oct 5th, 2021
Nikhila Raol, MD, MPH, and Jennifer J. Shin, MD
AAO-HNS 2021
Big Data Comes to Otolaryngology
Oct 5th, 2021
Richard M. Rosenfeld, MD, MPH, MBA
AAO-HNS 2021
A Systematic Review Can Help You and Your Patient Get the Most from Their Care
Oct 5th, 2021
Will Harrill, MD
AAO-HNS 2021
Value-Based Healthcare a Future Reality in Otolaryngology
Oct 5th, 2021
Rachel A. Georgopoulos, MD
AAO-HNS 2021
The Importance of Developing Professional Expertise
Oct 5th, 2021
Merry E. Sebelik, MD
AAO-HNS 2021
Can Global Surgical Skills Training Be Done Remotely?
Oct 5th, 2021
Sarah N. Bowe, MD, EdM, FACS
AAO-HNS 2021
The Real-World Impact of Disparities in Healthcare
Oct 5th, 2021
SIM Tank first place winner Alexis Graham-Stephenson, MD, pictured second to left, with Kelly Malloy, MD, Chair of the AAO-HNSF Simulation Committee; Jeffrey P. Simons, MD, MMM, AAO-HNSF Coordinator for Education; and Ken Yanagisawa, MD, AAO-HNS/F President-elect.
AAO-HNS 2021
SIM Tank Results
Oct 4th, 2021
Ellen M. Friedman, MD
AAO-HNS 2021
Cotton-Fitton Lecture Brings New Views to Otolaryngology
Oct 4th, 2021
AAO-HNSF
Facebook iconInstagram iconTwitter X icon YouTube icon
© 2025 Ascend Media. All rights reserved.