Stop Asking ChatGPT to Summarize Papers. Here’s What to Do Instead.

1. Introduction

You have a paper open in one tab and ChatGPT in another. You paste the abstract. You ask: can you summarize this? A confident, well-structured response appears. Three or four paragraphs, clearly written, covering the contribution, the method, the result. You read it. You feel like you understand the paper. You close the tab and move on. This workflow has, over the last two years, become almost universal among researchers. It feels productive. It saves time. It produces output that is technically correct in most cases. And it is quietly, systematically making you worse at research. This post is about why that is true, what the failure modes actually look like, and what to do instead. It is not anti-AI. AI is going to be a permanent part of how researchers engage with the literature. The argument is more specific: the particular workflow of pasting a paper into a general-purpose chat assistant and asking for a summary is the wrong way to use AI for reading, and there are better alternatives.

2. Recent Advances

What’s Actually Wrong With Asking ChatGPT to Summarize Several things, in increasing order of seriousness. The summary is based on what the model thinks the paper says, not what it actually says. When you paste an abstract or even a full paper into a general chat assistant, you are asking the model to compress text. The compression is based on language patterns in the model’s training data, not on careful reading of the specific paper. The result is a summary that is usually approximately right, but whose specific claims are often subtly distorted in ways the reader cannot easily detect. The model fills in plausible-sounding details

from similar papers it has seen. The output reads fluently and feels reliable. It is, on close inspection, often wrong about specifics. You cannot verify what is missing. A summary tells you what the model decided was important. It does not tell you what the model decided to omit. The result is that you do not see the contributions of the paper that the model did not recognize as central — which, in research, are often the most interesting parts. A paper’s headline contribution may be straightforward to summarize; its subtler implications, the assumptions it questions, the methodological choices it makes that experienced readers would notice — these are exactly what tend to disappear in machine summaries. The act of reading was what produced understanding, not the act of receiving a summary. This is the deepest problem. Reading a paper is not just acquiring information. It is constructing a mental model of the work — arguing with it as you go, noticing tensions, comparing it to other things you know, sitting with the questions it raises. None of this happens when you read a summary. You receive a compressed conclusion without the reasoning that produced it. You feel like you understand the paper. You actually understand only the model’s interpretation of the paper, which is a different and shallower thing. Hallucinated content is hard to catch. Recent research has documented that large language models routinely invent details when summarizing complex material — claims that sound plausible, numbers that look reasonable, even citations to papers that do not exist. When the model summarizes a paper you have not read, you cannot tell what it invented. The summary becomes a black box that you have no way to audit. You are training yourself to read worse. This is the slowest-acting consequence and the most damaging over time. Every time you substitute an AI summary for actually reading a paper, you are practicing not reading. The skill atrophies. The instinct that experienced researchers develop — the ability to skim a paper and rapidly identify what is genuinely new, what is overstated, what assumptions are doing the work — requires repeated practice with the actual texts. Outsourcing the reading to a model makes you progressively worse at the thing that defines research expertise.

This argument is largely an illusion, for two reasons. First, you should not be reading everything. As earlier posts in this series have argued, the right move in the current literature is to read fewer papers more deeply, not more papers more shallowly. The papers that warrant your engagement at all warrant being read properly. The ones that don’t warrant a proper read should be dismissed, not summarized. Using AI to skim papers that don’t deserve attention is not a productivity gain; it is a way of feeling productive while accomplishing less than nothing. Second, the apparent time savings are paid back later. A paper you read carefully sticks. A paper whose summary you read evaporates. The next time you encounter a reference to it, you will half-remember the summary, not the paper, and either you will go back and read it properly (paying the time cost you thought you avoided), or you will cite it shallowly based on the half-remembered summary (which is its own problem, as covered in earlier posts). The math does not work out the way people assume. Time saved at reading is usually time owed at writing, citing, or being confronted with a gap in your understanding.

When AI Reading Tools Are Actually Useful This is not a blanket rejection of AI in the reading workflow. AI tools have legitimate uses. The problem is that the wrong workflow has become the default. Here is what works. Use AI for triage, not for substitution. An AI summary is useful for deciding whether to read a paper, not as a replacement for reading it. “Is this paper worth my time?” is a question an AI can help answer in thirty seconds. “What does this paper actually contribute?” is a question only careful reading can answer. The summary should function as an extension of the abstract, not as a substitute for the paper. Use specialized tools, not general assistants. ChatGPT was not built for academic reading. Tools like Elicit, SciSpace, and Scholar GPT are specifically designed for engaging with research papers. They have access to verified document content, citation context, and academic-specific features. Their outputs are still imperfect, but they are substantially better calibrated than asking a general chat assistant the same questions. If you must use AI in your reading workflow, use the tools built for it. Ask narrow questions, not broad ones. “Summarize this paper” is the worst possible prompt. It invites the model to produce confident-sounding but unverifiable output. Narrow, specific questions produce better results: “What dataset does this paper use?” “What

baseline are the authors comparing against?” “What is the effect size of the main result?” These are questions the model can answer reliably because the answers are in the text. They also serve a useful function in your reading — they make you specify what you actually want to know, which is itself a form of engagement with the paper. Use AI to extract, not to evaluate. AI tools are good at finding specific pieces of information in long documents. They are bad at evaluating significance, judging novelty, or assessing how a paper fits into a broader landscape. Use them for the first kind of task. Do the second kind of task yourself.

immediately after reading. A few sentences. What the paper does, why it matters to your work, what questions it raises. This is the step that turns reading into knowledge. It is also the step that cannot be outsourced to a model — your interpretation of a paper, framed in your terms, is the actual product of reading.

3. Discussion

Fix the upstream problem, not the downstream symptom. PaperRadar delivers AI-ranked, personalized research paper summaries to your inbox every morning — so the papers reaching you are already pre-filtered and worth a proper read, not noise to compress. Get started free at paper-radar.com