Credit Where Credit is Due
This summer, Santiago Schnell, the provost of Dartmouth college, wrote an article for The Washington Post titled
Universities are fighting AI cheating. But there’s a deeper problem.
The byline says it was authored solely by Schnell.
On August 26, Semafor published a report in which they ran the AI detector Pangram against 310 guest articles in major newspapers and found 50 which appeared to be written by LLMs.
Schnell’s op-ed was among them.
The Dartmouth, Dartmouth’s inventively-named student newspaper, conducted
a follow-up investigation a few weeks later, and found that the same tool identified nine of Schnell’s recent written works as being almost entirely generated by an LLM.
I do not believe that getting flagged by Pangram should be considered definitive proof of LLM use.
LLM-generated text is increasingly hard to avoid, so its ticks are likely to embed in your writing even if aren’t using them yourself.
I also have no patience for the treadmill of identifying typographic features or turns of phrase which supposedly indicate their influence.
LLMs are trained on text from the internet, and vast swaths of internet content is marketing copy written by non-native English speakers;
a lot of LLM “tells” are the hallmarks of, e.g., Kenyan English.
Pangram’s possible bias against non-native speakers like Schnell (who was born and raised in Venezuela) is irrelevant here, however, because Schnell admits that he uses LLMs extensively in his writing process.
In his own words:
In an email statement to The Dartmouth on Sept. 18, Schnell wrote that he uses artificial intelligence for “language refinement, copyediting, improving clarity and organization and making writing processes more efficient,” but “review[s] the final text and take[s] responsibility for its content.”
“I develop ideas and arguments, conduct the research, evaluate the evidence and sources, determine the structure, prepare drafts and decide what appears in the final text,” Schnell wrote.
Or at least, what I assumed were his own words—an assumption that someone at The Dartmouth had the insight to check.
Schnell’s Sept. 18 email statement to The Dartmouth was itself “100% AI-written,” according to a Pangram test.
In a follow-up statement about the composition of the email, Schnell wrote that he “developed the substance and wrote a draft” of the email, then “used an Al tool to assist with language and clarity, taking on several suggestions (including reworded sentences), before reading it through, making minor changes and sending it to you with my signature.”
“I am fully responsible for any text under my signature,” Schnell added.
Let’s put aside how astonishing it is to use LLMs to write an email insisting that you are not overly reliant on LLMs.
I’d like to focus on the implicit argument that Schnell is making: that you shouldn’t care how much of the article he actually wrote, as long as he is willing to sign his name to it.
I recently encountered an identical argument in the programming community.
In a post on his blog, software engineer Indradhanush Gutpa writes that he is no longer crediting Anthropic’s Claude on his code changes, because adding an LLM as co-author gives you a way for you to shirk responsibility for mistakes.
Pressed by commenters that this sounds like a pretext for hiding LLM use, Gupta insists that it is not.
He explains that he is exploring a setup where the project has a broad disclaimer that LLMs were involved, but individual changes bear his sole authorship when they reflect work I did myself and with an LLM agent but I slowed down enough to understand what it did.
If there is any field in which LLM-generated, human-approved content should be acceptable, it is software engineering.
Most people do not consider code a form of human expression in the same way that, say, a poem is.
If a computer can automate writing code, why shouldn’t it?
Society probably does not require software that inspires—it requires software that works.
In addition, software engineering has already gone through multiple disruptive transformations in which a difficult, manual code-writing process was replaced by a more convenient, automated one.
Programming languages themselves are an example of this.
Below is a sample of x86 assembly that declares a routine for squaring numbers, then uses it to square the number 4.
"square":
push rbp
mov rbp, rsp
mov DWORD PTR [rbp-4], edi
mov eax, DWORD PTR [rbp-4]
imul eax, eax
pop rbp
ret
.LC0:
.string "%d\n"
"main":
push rbp
mov rbp, rsp
sub rsp, 16
mov edi, 4
call "square"
mov DWORD PTR [rbp-4], eax
mov eax, DWORD PTR [rbp-4]
mov esi, eax
mov edi, OFFSET FLAT:.LC0
mov eax, 0
call "printf"
nop
leave
ret
Assembly instructions are the raw inputs to a CPU.
The push, mov, and pop operations control individual bytes of memory.
Writing assembly is highly empowering because you are, in essence, manually controlling the computer.
It is also highly inconvenient for complex tasks.
Instead, computer scientists invented languages that are easier to read, write, edit, and reason about.
A software program called a compiler transforms the high-level code into assembly instructions, which can be understood by the processor.
The following snippet of the C programming language, courtesy of Compiler Explorer, produces the x86 assembly instructions above:
#include <stdio.h>
int square(int num) {
return num * num;
}
void main() {
int result = square(4);
printf("%d\n", result);
}
I have never seen anyone claim that, after writing C code and running it through the compiler, they become the “author” of the assembly that the compiler generates.
People sometimes inspect what assembly instructions their code produced.
They may even adjust it to produce a desired result.
But that's not what they authored—they authored the C code.
Properly demarcating one's role in producing a program makes it easier to write correct software.
Compiler errors are rare—so rare that a programmer’s instinct to blame the compiler instead of their own code is a common joke, even (especially) when it turns out to be true.
When compilers do have bugs, however, it is incredibly helpful to know the difference between what the human wrote and what the computer generated.
If we did to compilers what Gupta proposes that we do for LLMs—peel off the sticker and claim the product is homemade—we would have a far more difficult time identifying the problem at its source.
The introduction of LLMs is not a novel psychological trap that makes programmers more likely to shirk responsibility by blaming their dependencies.
Programmers have always done that and it has always been wrong.
Jeff Atwood, the founder of StackOverflow, wrote about this in 2008, in The First Rule of Programming: It’s Always Your Fault.
The flip side of code ownership is code responsibility. No matter what the problem is with your software – maybe it’s not even your code in the first place – always assume the problem is in your code and act accordingly. If you’re going to subject the world to your software, take full responsibility for its failures. Even if, technically speaking, you don’t have to. That’s how you earn respect and credibility. [emphasis his]
You don’t need to steal robot valor to be responsible for your product. You can just… choose to be responsible for it.
My friend Carson Gross, a computer science professor at Montana State, offers a more robust solution that builds on existing best practices:
Software engineers who use LLMs should save their prompts alongside the code that was generated from them.
Doing so separates what the human intended (as captured in the prompt), what the human reviewed and edited from LLM generation (when both are credited with the change), and what the LLM generated alone (when the LLM is the sole author).
Theoretically, it also allows the code to be re-generated later, based on the same prompts, but with better models.
Rather than run from the implications of relying on machine assistance to produce things, we can improve the machines and ourselves by properly documenting it.
LLM-assisted authors who do not wish to be labeled LLM-assisted authors
deploy a variety of analogies to justify eliding the role that LLMs play in their work.
In one email, Provost Schnell wrote: I regard these technologies as assistive tools. They do not […] substitute for my research, intellectual judgment or authorship.
Two days later, in a different statement, he claimed that if a professional editor, ghostwriter or software editor (machine learning or AI-based) is used, they will not be acknowledged in the publication.
First, the role that LLMs play in your writing process cannot be analogized to both an assistive tool
and a professional editor
at the same time.
Either they function like a pen and a dictionary, tools which do not operate autonomously, or they operate like a second person reviewing your work, one who might exercise independent judgement, offer new ideas, and influence the final product.
The latter is more consistent with how Schnell describes his own LLM use: improving clarity, organizing the text, and offering suggestions on how to express ideas.
Schnell’s mixed metaphors suggest that he himself does not have a firm grasp on the extent to which AI is influencing the work he produces.
To the detriment of Schnell’s credibility, this is a widely relatable feeling—most of us have working experience with these products and we know how eagerly they produce text.
LLMs are more than willing to substitute for your authorship and judgement;
it requires substantial effort and personal restraint to keep them from doing so.
Insisting that your undisclosed LLM use remains circumspect is a bit like insisting that your covert nuclear program is only for energy production.
Maybe that’s true, but then why did you hide the facility?
Can we inspect it?
You sure seem to have a lot of centrifuges.
Suppose, however, that it was relatively straightforward to limit LLMs such that they only provide the kind of assistance that a professional editor might.
Schnell argues that employing an uncredited editor or ghostwriter is the the rule of thumb for scientists or academics who are non-native speakers.
Would employing an LLM in the same manner—without credit—be acceptable?
I propose the opposite:
Concerns that academics might be submitting low-quality, LLM-generated work provide us an opportunity to re-evaluate whether the existing norms of academic publishing are serving their intended purpose.
I don’t know how much academic work is actually being ghostwritten, but if a professional editor or writer is involved in the production of a paper, I would argue that this is at least worth mentioning in the acknowledgements.
There are many fields in which significant contributors to a published work are not given the credit they are due.
Federal judges publish decisions under their own name that are often written largely by clerks.
Executive assistants send emails from their boss’ address.
Airport fiction mogul James Patterson proudly employs a stable of collaborators who don’t get top billing, just as the Dutch master Rembrandt did before him.
None of these mis-attributions are sanctioned exceptions to the laws of decency—they are, for the most part, marketing gimmicks.
Omitting the contributions of a less-prestigious collaborator attempts to enhance the appeal of a work outside its intrinsic merits.
It serves a purpose orthogonal to integrity.
In some fields, one is expected to document all the people who helped bring a creative work into being.
Movies have extensive credits, which illuminate how complex they are to produce and give everyone involved a moment of recognition.
Newspaper articles rarely credit editors, but novels sometimes do, and translation is usually regarded as an interpretive act worthy of front-cover representation.
A budding typographer will quickly discover that books often note the typeface with which they were printed—in a roundabout sense, publishers really do acknowledge their pen.
No creative work is impoverished by further documenting its creation.
This is not to say that every single article necessarily needs to have movie credits, but in cases like this one, where existing norms might not be capturing something important, giving more credit to collaborators (human or machine) is a great first step.
As with code, establishing a clear chain of provenance for prose helps us better understand each other and the work we produce.
Corporate inboxes now let you send an email on behalf of
someone else;
the Dutch government funded an effort to identify which of Rembrandt’s paintings should actually be credited to his students;
even James Patterson now routinely puts his ghostwriters on the cover (albeit in smaller print).
If editing with LLMs becomes commonplace, it likewise makes sense to acknowledge its role—especially since LLMs are far more willing to completely rewrite your work than a human would be.
Suppose we engage with Schnell’s work on his terms.
What do we find in the op-ed for which he assumes full responsibility?
Dartmouth professors Andrew Levin and Micheal Herron allege substantive errors and omissions […] that would be practically inconceivable in a column written by a distinguished human expert.
Schnell misrepresents the methodology of an academic study, implying that he only read the abstract;
he omits more recent studies which would support his argument, suggesting his research was cursory;
when describing how AI is deployed in classroom, he reproduces a sentence straight from a Dartmouth press release, only with clauses switched and synonyms swapped in—exactly the way a high-school plagiarist would.
To the analysis of Levin and Herron I add my own: the article is simply bad.
It provides no insight, only pablum and cliché.
Universities are fighting AI cheating. But there’s a deeper problem does not even justify its title, let alone offer a compelling take on it.
As best I can tell, the “deeper problem” is that widespread deployment of AI makes it difficult to determine what students are capable of;
that would seem to be the fundamental problem with cheating, not a deeper one.
Naturally, Schnell does not have a solution to his problem statement.
He says that universities need to go “beyond” in-person examinations, but his examples of how Dartmouth ensures that students understand their work are all in-person examinations.
He says that universities should decide how much their degrees reflect independent mastery versus intelligent use of AI, but offers no suggestions on how to separate these.
He summarizes cited studies as revealing that students should use AI for questions, feedback and extra practice, rather than to supply a finished answer;
this echoes Schnell’s explanation of how he used AI to produce this article, and is just as nonspecific.
Whatever the actual extent of Schnell’s reliance on LLMs, he successfully reproduced their core function: to regurgitate things people already know in prose just unreadable enough to mask that nothing novel has been said.
Many op-eds have bad takes, but at least they are bad takes with a human point of view.
At least I know which deli meats David Brooks thinks are beyond the grasp of his friend.
At least I know where Tom Cotton wants to send the troops.
Schnell’s op-ed compares especially poorly with one that his boss, Sian Beilock, the president of Dartmouth, published in The Atlantic a week later, under the punchy headline Why I Want More AI at Dartmouth.
Beilock references the various measures that other schools have taken to curb LLM use but, unlike Schnell, she offers a clear take on them.
Beilock argues that fear of AI is driven by unfamiliarity with the technology and that integrating it deeply into the curriculum will better prepare Dartmouth students for the job market that awaits them after graduation.
Her article includes a number of pertinent examples that demonstrate how she has encouraged this approach at the school.
I do not agree with Beilock’s strategy, but I can acknowledge that it is philosophically coherent.
The article clearly articulates her worldview and is supported by the evidence included in the text.
It has an argument.
It doesn’t waste my time.
The two most profound academic experiences of my life occurred when I was an undergraduate at Dartmouth.
They both taught me the same lesson: at the highest levels, you cannot talk your way out of not having thought the problem through.
My sophomore winter, I took Professor Prasad Jayanti’s CS31: Algorithms.
Contrary to my expectations, the course was mostly not about learning algorithms, but proving them.
I did quite poorly on the first few assignments, because when I didn’t really know how to prove something, I would talk myself into believing that I had figured it out, then write a proof that sounded close enough.
It was trivially obvious in my writing that I lacked the key insight to actually prove the algorithm, and my score reflected it.
The midterm for this class was a take-home assignment with just five questions.
You had a full week to complete it.
You were not allowed to work with your classmates.
As the week melted away, I got more and more desperate.
I paced the third floor of the library, glancing forlornly at my classmates, allowing myself only the occasional did you get it yet?
(Usually they hadn’t.)
I stayed up well past midnight consuming a horrifying number of sweetened iced cappuccinos.
I hadn’t cracked the key insight, but I had at least learned to stop bullshitting myself about it, and I was going to keep drinking sweetened icedcaps until I figured it out.
And eventually I did.
My senior year, I signed up to write an honors thesis in Government, under the advisorship of Professor William Wohlforth.
I had aced Gov85: Grand Strategy, his senior seminar, two years prior, and I had worked for him as a research assistant, so I thought we had a great rapport.
I put together what I thought was a compelling and slightly cheeky argument, along with a couple basic examples that seemed to demonstrate the state behavior I sought to prove.
I cruised through the fall thesis workshop and emailed him my working draft.
The reply he sent me is long gone, but I vividly recall how I felt reading it.
He basically told me he had no idea what I was talking about.
My argument was shapeless, my evidence was thin, and I was quite behind where he expected me to be.
The overwhelming sentiment I detected was disappointment.
That was when it hit me that the thoroughness I had been forced to put into mathematical proofs was expected of me in the social sciences, too.
In fact, it was expected of me everywhere.
No level of writing skill would ever hide from a discerning eye whether I actually knew what I was talking about—and if I wanted to work with the best of the best, knowing what I was talking about would take an immense amount of hard work.
I believe Provost Schnell didn’t acknowledge his AI assistance because he was afraid doing so would invite scrutiny—and reveal that he does not actually have anything to say.
He doesn’t know what to do about AI in colleges;
he just knows he’s supposed to have an opinion about it.
If he has given the issue any thought, it is not reflected in his writing, and it was not sufficient to generate novel insight.
LLMs are more than happy to paint over that hole, the way a landlord turns over an apartment with some trusty cans of beige.
Four days after The Dartmouth published its findings, Provost Schnell published his response,
What I’ve Learned About AI and Academic Integrity.
It ends like this:
One of the reasons I was called to Dartmouth is because this community is eager to take on hard questions and does not stand by and wait for others to do the difficult work first. I am working with senior vice president for community and campus life and chief student affairs officer Jennifer Rosales to think about how we engage our community very soon in a dialogue on when AI should or should not be used in class assignments, public writing and scholarship. These are not questions that can be answered by one individual or members of one academic discipline alone. I invite you to join this shared endeavor — your input is important and will be valued.
As a proud alum and a lifelong community member, I would like to accept this invitation to join the shared endeavor.
I offer the following proposal as a starting point for discussion:
- The use of AI products for any coursework or publication must be disclosed, alongside a short description of the manner in which they were employed. When in doubt, disclose; there are no penalties for being overly explicit.
- Using LLMs for translation purposes is not considered distinct from using LLMs to write and edit English-language prose.
The use of a dictionary does not need to be disclosed, but tools used to translate larger strings of text do.
- The use of LLMs for research is permitted, and does not necessarily need to be disclosed, so long as the LLM is used to search for citations, not summarize them.
LLMs summaries of a text cannot be cited in a manner that suggests the writer has actually read the text.
- Many web services, such as Google Docs, now have built-in LLM features which can read your work and suggest edits.
Use of these features must be disclosed in the same manner that use of a standalone tool, like Claude, would be.
- Emails drafted with LLM assistance should include a disclosure that LLMs were involved in drafting the email.
If this is your standard practice, the disclosure can be automated by including it in the email signature.
These are not my personal principles for LLM use, nor are they necessarily the ones that I would choose for Dartmouth if I were made president of the college tomorrow.
I would rather see Dartmouth enforce rules that create space for the deep learning I experienced in the best courses I took there—space which is, for the time being, becoming rarer in both academia and the workplace.
Absent that, however, I believe that these proposals are realistic, aligned with President Beilock’s vision, and would do a lot of good.
The goal of these principles is not to restrict the adoption of LLMs, but merely to illuminate it.
If widespread use of these products really does produce better work and more capable students, full disclosure only serves to highlight that.
Dartmouth can blaze a path for academic integrity and help us better understand the capabilities of LLMs by requiring transparency about the extent of their contributions.
Then we can all asses how useful they really are.
–Alex Petros ’19
Thanks to Parker Richards ′18 and Katie Petros for their feedback on a draft of this post.
