Story by Bec Roldan
Photography by Ridwan Karim Dini-Osman
When large language models like chatGPT jumped onto the scene in 2022, many people feared these tools would render careers like journalism and science writing obsolete. But at the ScienceWriters2024 national meeting, a panel of science communicators outlined that generative A.I. cannot perform to the level of human communicators, but that doesn’t mean we shouldn’t continue interrogating their uses in our field.
The "Tools, Toys, or Trouble: Can A.I. Help Science Communicators?" session was co-organized by Abigail Eisenstadt, a writer for AAAS’s SciPack; Matthew Wright, deputy director of SciPak; and Matthew Shipman, assistant director of research communications at North Carolina State University.
“The title [of this panel] is not a discrete choice,” said Matthew Wright. “We know that the answer could be some of all three, it could be none of those three.” The panelists interrogated this rhetorical question by discussing their experiences experimenting with large language models (LLMs) in their own work, either formally or informally, in the various pockets of the science communication ecosystem.
Before thinking about how to use these tools or determining their accuracy, Shipman raised legal and ethical concerns regarding generative A.I. and copyright issues. Aside from legal concerns, Shipman said points raised by creators “pose serious questions about the ethics of using content generation tools that make use of creative work without compensation.”
In the informal science education and museum space, Danielle Olson has experimented with chatGPT informally to determine if the tool could help her with her writing, like generating article titles. Olson, a managing editor for the Ocean Portal at the Smithsonian National Museum of Natural History, gave an example of a chatGPT-generated title about coral reefs:
As coral reefs face demise or degradation, their intricate ecosystems stand to vanish irreversibly.
“I don't know about you guys, but we all agree that that is not something a human would write,” she said. Olson notes that the Smithsonian has a policy against using chatGPT.
Miriam Fauzia speaks on stage at ScienceWriters2024 in Raleigh on Nov. 10. (Ben Young Landis/NASW)
In the journalism space, Miriam Fauzia, a science and medical reporting fellow at The Dallas Morning News, examined how LLMs summarize academic papers after coming across a YouTube video about the topic. This could be a useful application of the tool, given that science journalists often have to write study stories on topics outside of their expertise. “It turns out that they can help summarize, but they can also make a lot of mistakes, too,” she said, especially when interpreting the results of a study. When she asked chatGPT to find citations on scientific topics, the tool generated fake citations, something Shipman said is called “hallu-citations.” Using LLMs in science journalism can introduce a lot of inaccuracies, Fauzia said. “We’re living in a time when trust in the media is so low, so we have to be really careful about how we’re using these tools.”
In the scientific journal press office space, Abigail Eisenstadt and the SciPak team have been experimenting with chatGPT+ — a paid version of chatGPT that can ingest new content — to see if the LLM could generate the short, concise paper summaries SciPak is known for. The team picked papers across all six Science family journals, including research papers, perspectives, policy forms, and more and fed them into chatGPT+.
The SciPak team found that ChatGPT+ can summarize one main finding and does well distilling non-research papers, but if nuanced methodology is introduced in the research paper and nuanced contextualization is needed, the tool falls short. “It can summarize and it can’t synthesize,” said Wright quoting Abigail Eisenstadt, who was unable to attend due to illness. Wright added, “We think these summaries don’t meet the bar for quality and style that we really strive for, at least not yet.”
Generative A.I. is “being used in just about every way that you can imagine it’s possibly being used,” said Shipman. The panel urged science communicators in the audience to engage with generative A.I. in an experimental manner, to better understand these tools and their potential applications, but advised against using these tools in a professional capacity in science writing and journalism.
A packed conference hall listens to the panel on generative A.I. at ScienceWriters2024 in Raleigh. (Ben Young Landis/NASW)
Miriam Fauzia meets with fellow conference attendees Adithi Ramakrishnan and Lila Levinson during the ScienceWriters2024 weekend in Raleigh. (Ridwan Karim Dini-Osman for NASW)
Bec Roldan (www.becroldan.com) is an independent science journalist based in Brooklyn, N.Y. They received their Ph.D. in chemistry from the University of Michigan in 2024.
Ridwan Karim Dini-Osman (linkedin.com/in/ridwan-karim-dini-osman-67417183) is a multiple award winning development journalist working for Ghana's GHOne TV as a senior reporter and news anchor. He is a recipient of the 2018 Lorenzo Natali Media Prize, a global prestigious award run by the European Commission; and 2020 best African TV Journalist in Environmental and Change Reporting by the Pan African Climate Justice Alliance (PACJA).
This ScienceWriters2024 conference coverage article was produced as part of the NASW Conference Support Grant awarded to Roldan to attend the ScienceWriters2024 national conference. Find more 2024 conference coverage at www.nasw.org
A co-production of the National Association of Science Writers (NASW), the Council for the Advancement of Science Writing (CASW), and the Science Communicators of North Carolina (SCONC), the ScienceWriters2024 national conference featured an online portion Oct. 16-18, followed by an in-person portion held in Raleigh, N.C., Nov. 8-11. Follow the conversation via #SciWri24 on Bluesky and on LinkedIn.
Founded in 1934 with a mission to fight for the free flow of science news, NASW is an organization of ~2,400 professional journalists, authors, editors, producers, public information officers, students and people who write and produce material intended to inform the public about science, health, engineering, and technology. To learn more, visit www.nasw.org and follow NASW on LinkedIn and Bluesky. And join us in celebrating #NASW90th.
Credits: Reporting by Bec Roldan; edited by Ben Young Landis. Photography by Ridwan Karim Dini-Osman; edited by Ben Young Landis
