Building a Scalable Research Integrity Framework in the Age of AI

The Challenge

ASM’s editorial teams were already under pressure when a wave of out-of-scope and paper mill submissions began hitting their submission queues, nursing papers in immunology journals, fish papers in virology journals, seemingly submitted through automated or AI-assisted mechanisms. The volume was significant enough to affect acceptance rates and distort the editorial picture, yet the submissions were arriving faster than any manual process could handle.

ASM had a strong ethics team, highly capable at handling post and pre-publication cases with depth and nuance. But the nature of this new threat required something different: the ability to recognize patterns and signals at scale, before papers reached editors at all. The existing toolset wasn’t built for that. Image integrity screening via Image Twin was already in use but it was being applied after acceptance, a stage at which flagged papers had already consumed significant editorial resources. It was a good check in the wrong place.

The core problem was threefold:

The Approach

ASM partnered with KGL to co-develop a workflow-integrated integrity framework, one that treated research integrity not as a reactive investigation triggered by suspicion, but as an infrastructure component embedded throughout the editorial process.

Together, KGL and ASM established a risk evaluation process at the triage stage, drawing on KGL’s multi-factor analysis engine to assess up to 80 individual indicators across eight categories including author credentials, ORCID consistency, reference integrity, formulaic research patterns, and contextual AI usage signals. Rather than relying on a single tool or score, each submission was assessed holistically, with findings synthesized into a clear, tiered report designed to be actionable for whoever received it, whether that was an ethics team member, a peer review associate, or a senior editor.

KGL’s risk evaluation reports were correlated with ClearSkies’ Paper Mill Alarm flags to create a calibrated picture of risk, one that could be interpreted differently depending on context. A red flag from ClearSkies might, on closer inspection, represent manageable risk; an orange flag, combined with other KGL-identified indicators, might warrant immediate escalation. The goal was a decision tree: clear guidance on who should see what, and what action to take at each risk level.

To address the timing issue, image integrity screening was moved upstream, applied earlier in the workflow where it could prevent resource expenditure on manuscripts that should never have progressed. Policies were tightened, including a strict position on AI usage in figures, giving the team a clear framework against which to evaluate incoming work.

The Solution in Practice

The framework gave ASM’s teams a structured way to act on risk without bypassing editorial independence. Where risk scores were high enough and the signals clear — out-of-scope submissions, falsified credentials, evidence of automated submission — papers could be withdrawn before editor involvement, saving volunteer editors from wasting time on work they would never have wanted to see. Where signals were ambiguous, the report provided enough context for the appropriate person — ethics team, peer review associate, or editor — to make an informed, consistent decision.

Critically, the system was built to be flexible. ASM’s policies, risk thresholds, and escalation criteria were reviewed and updated regularly — sometimes monthly — as AI capabilities evolved and new patterns of misconduct emerged. The framework wasn’t static; it was designed to learn alongside the challenge.

KGL’s risk evaluation of the JVI batch categorized the flags as follows: 39 were identified as Low risk, 25 as Moderate risk, and 12 as High risk. This evaluation was carried out from an initial count of 64 Orange flags and 15 Red flags. The results indicate that most flags were resolved significantly below the initial alarm threshold after the full evaluation was completed.

In Their Own Words

“This adventure we’ve been on together has been super collaborative, and I think that’s the most important piece. The ability to communicate, collaborate, and iterate has been great — and I think that’s critical to mitigating risk and moving science forward.”
— David Haber, Publishing Operations Director, ASM

Ready to strengthen research integrity across your publishing workflow? Connect with us to explore how our modular, end-to-end services can support your journals today—and scale with you tomorrow.