The value of asking questions: Of podcasts and AI writing

A longform read

Introduction: Podcast ambience

Since the pandemic, I have been listening to podcasts when cooking. Rather than think about the meal at hand, I listen to a podcast. Most cooking is quite boring. Chop onions. Peel carrots. Dice celery. Rather routine. This habit followed for other chores. If I had to do laundry, I popped in a podcast. If I took my dog for a walk, podcast in the ears. Driving? Podcast. Grocery shopping? Podcast. Every chore could be sprinkled with a podcast. Sometimes, I spent more time finding an episode than it took to do the actual chore. I once put on a podcast to plunge the toilet after one of my kids clogged it. This decision proved unwise.

Podcasts ease the interminable practices of everyday life. A bandage for boredom. They provide an escape from mindfulness.

Most podcasts are such drivel that I remember little about them. With the bad ones, I recall no details. Even with good episodes, the ones I discuss with friends or family, I can remember only a detail or two. I give an episode a second listen so rarely that I consider it a monumental event. I tell my friends, “It was so good, I listened to it twice!” I have only said these words a handful of times.

Part of what makes podcasts so obnoxious is their length, at least the ones that aren’t carefully edited and tightly scripted. Joe Rogan’s episodes are over three hours. Erza Klein’s comes in generally above an hour. I don’t need four-hour podcasts about videogames I already played. I don’t need two-hour podcasts about the previous week in whatever sportsball is being played (Bill Simmons). No one has time for these herculean listening tasks unless they multitask. We pay half attention to a podcast, half to the chore at hand, neither one getting done especially well. Podcast listeners are always doing something else, not just listening.

I think that’s the secret. Podcasts are something like ambience. And because of this, podcasters meander.

It’s the meandering that frustrates me most. While I can hear what keeps listeners coming back—easy relaxed banter, sometimes a cacophony —the imprecise nature of podcasts makes me frustrated. Podcasts are wordy without much being said.

The meandering nature of podcasts has created a plague of what conversation analysis might call dropped threads. Dropped threads are moments when people stop talking about a topic. Everyone does this in their lives, often unintentionally, sometimes intentionally. It happens all the time on internet discussion forums. In podcasts, dropped threads are glaring if you’re a careful listener. Almost every episode—even rehearsed ones like “The Daily” from The New York Times—a speaker will say, “We can get to that” or “We can talk about that later.” But they rarely get to it. The point never arrives. The thread dropped. Speakers often drop the thread precisely when the conversations get difficult. The discussion stops precisely when conversations require precision.

Dropped threads make me wonder if unproduced, unrehearsed conversations aren’t meant to be recorded. Podcasts are often too unformed. (In fact, this is why companies want to monetize them: they’re cheap to make. No scripts or editing. The only cameras are those for someone’s face.) People say things that aren’t true. They’re glib, facetious. They might be lying loquaciously but they also might be shooting the breeze, not even realizing they are lying. Because of this breeziness, listening to auditory records of casual conversation isn’t the best way to take in information, especially if you aren’t part of the conversation. People just saying—that’s probably not a good ambience to have around. Meandering leads to overstating. To being disorganized. To being incoherent.

It’s better that I focus on plunging a clogged toilet without earbuds.

Removing the earbuds

Recently, though, I decided to put my phone away for the entire day. I had gotten frustrated with my phone, feeling anchored, burdened by my listening addiction. This feeling was exacerbated by the banality of podcasts. After leaving it aside for the entire day, I decided to take a more drastic step.

This past week, I stopped listening to podcasts entirely, ridding myself of conversational ambience. It’s left me with…many unplanned thoughts. I went for a walk a few evenings ago with a good friend and colleague. We both left our phones behind, remarking about how untethered we felt. Being unlocatable made us naughty.

Lest you think this blog post is going to turn into an anti-screen rant, not entirely. See, I didn’t spend much time looking at my phone in the first place. My screen time was only about two hours a day, which is must less than the typical American. That said, I’m proud of these screen time numbers from my phone (below). Yet, since stopping podcasts, I’ve spent more time on my laptop, so it’s not entirely about screentime. I’m not an ascetic monk.

But. Yet. Hmmm. There…is something else that has me enjoying the lack of podcasts. It has nothing to do with screens. I found myself…asking more questions.

The importance of asking questions

If you know me, I am a chatty person by disposition. After giving up podcasts, I found myself talking in significantly different ways. Ripping the chatter from my ears made me realize how much I have been talking simply to fill a room. To banter without goals.

I found myself asking my kids more questions, asking them about their world. I found myself asking my students more questions. Free of headphones, I found myself saying hello to people around town. The earphones were a fortification I’d erected, inadvertently.

I found myself less solipsistic: pondering the lives of others rather than letting myself be dominated by non-interactive sound. Podcasts cannot be asked questions, yet this is the best feature of talking: the interaction.

I’m reminded of a legendary exchange in Greek philosophy, which occurs in Plato’s Phaedrus. In the dialogue, Socrates famously expresses his skepticism of writing. Most people key into the criticism of writing as an aid to memory. For Socrates, a tight memory is good because it is internal to the body whereas writing is external:

[Writers] will cease to exercise memory because they rely on that which is written, calling things to remembrance no longer from within themselves, but by means of external marks.

Orality—speaking, listening, and dialogic exchange—are crucial because writing becomes a substitute for remembering. Socrates says:

[The writing] which you have discovered is an aid not to memory, but to reminiscence, and you give your disciples not truth, but only the semblance of truth; they will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.

I’ve always found this passage ironic because it’s written. I’ve always had to imagine how Socrates delivers these words. Does he say them prophetically, with righteous anger? Calmly? We never know because the dialogues are written.

It’s important, though, to recall why Socrates thought writing was a crutch. He believed writing was bad because people cannot ask questions of it.

I cannot help feeling, Phaedrus, that writing is unfortunately like painting; for the creations of the painter have the attitude of life, and yet if you ask them a question they preserve a solemn silence. And the same may be said of speeches. You would imagine that they had intelligence, but if you want to know anything and put a question to one of them, the speaker always gives one unvarying answer.

This is one of my favorite passages (the solemn silence is too good). The passage reveals, for Socrates, the key weakness behind writing and the strength of orality: we can ask people questions of their assertions when we are having a conversation. In other words, it’s not necessarily that speaking is inherently better than writing. The real-time nature of speaking allows a dialogic exchange to occur. The person asking questions can push on assertions—can challenge claims if the speaker meanders into a quagmire.

This is exactly the problem of podcasts: you can’t ask them anything about their banter. While it’s true that you can’t ask questions of TV or movies, these are typically not impromptu casual banter. Even with news shows, the talking between people is highly choreographed, with defined time limits and producers in their ears. But podcasts are configured differently. The casual banter leads me to consistently—often angrily—ask, “Where’s the evidence for that?” Podcasters speak so many words that I get overwhelmed. I can’t challenge them on their claims, good or bad, evidenced or not.

When we can ask questions about the claims people make, we have a better sense not only of the claims themselves but also if the person understands those claims. If someone wants to talk about the woes of capitalism, their perspective about markets informs what they understand capitalism to be. You can learn a great deal by not only what a person’s answer is but also in how they answer the question.

AI imprecision

The podcast problem appears most starkly as a problem of precision—or more precisely, imprecision. When people talk about a topic they don’t know much about, they’re likely to prattle on in various directions, free associating. AI writing kinda does the same thing. When you ask AI questions, it’ll give you an answer, often in bullet points unless you force it to answer in paragraph form. After a while, these bullet points get very same-y. It appears to be bantering without goals. It’s like a speaker who can talk for a long time but not actually say anything—like a podcaster.

The filler of podcasts sounds remarkably like the filler of AI writing. If we get used to one, it seems to me like it becomes easier to accept the other.

This is where length seems critical, for speakers or AI. Given a two-minute time limit, most people can speak coherently on a variety of topics. Give them 5 minutes, the level of coherency goes down. Give them 60 minutes, and force them to stay focused, most people can maybe speak on only a handful of topics. (If you want to know mine: writing, swimming & water polo technique, teaching, Willa Cather). Being precise in short bursts is easy because there is only so much one can say or write in a limited situation. If you broaden the scope by increasing the time (talking) or length (writing), then people suddenly become much more general.

In the same way, when I read AI writing, it’s like a podcaster filling up the required time slot. AI tools produce enormous amounts of text. They produce too much text, most of it meaningless. Vacuous and vapid. When I read AI writing, I feel like I’m losing my mind. Others feel the same ways, as this thread from Reddit can tell you. Below are some highlights from the Reddit thread of teachers describing what it’s like reading AI-written essays:

“I told my students that AI writing is on the level of a college senior, but one that doesn’t know the topic. It’s excellent in conventions but talks in circles, saying nothing, trying to blow smoke up my butt.”

“The grammar is better than the thinking.”

“AI essays often say a lot without actually saying much at all. The writing sounds sophisticated and the grammar is correct, but when you try to figure out the actual ideas, it falls apart. An example from a recent project in which students were supposed to create a playlist to accompany a movie adaptation of a novel: Lots of elegant writing about the “sentiment evoked by the scene” or whatever, but they never define what that sentiment actually is.”

The AI writing problem is not one of writing, these comments illustrate, but of thinking. AI writing is like having a car with the body of a Ferrari but the engine inside is one of those scooters people leave on the side of the road.

From judging writing to ideas

I don’t know what your profession is, but mine is writing and the teaching of writing. The problem of the podcast and AI writing, when taken together, has made me reflective. What the heck am I doing when I read an essay? What kind of response is expected of me?

These questions inspired me to return to an old research paper about feedback on student essays, “Effective faculty feedback: The road less traveled” (2006). This is a great paper, synthesizing a variety of studies in a literature review about how teachers provide students with feedback on their writing, as well as conduct their own study. In their conclusion, they write:

…faculty are providing feedback on technical writing components such as spelling, grammar, and word choice. These are types of errors that are fairly easy to detect and take relatively little effort to correct. Although these are important aspects of writing, many types of meaningful comments were surprisingly absent. The lack of written comments (either positive or negative) for the students’ support/evidence for claims, paper structure/organization, voice, and creativity was perplexing at the least. For the most part, students needed to improve in these areas, yet there were no comments telling them so.

This paragraph explains to me why Silicon Valley bros—like the newest models of Mark Zuckerberg or Sam Altman—see writing as merely a vehicle or conduit for communication. When most people go through school, they experience writing as a mechanical skill. Teachers correct their papers but rarely do the writing teachers provide them with broader suggestions. Let’s look at two devastating paragraphs from the study. I quote them at length because I feel like this explains what’s going on culturally with writing.

…only one-third of the papers received feedback that addressed the quality of at least one specific thought/claim in the paper. About half of these comments were positive while the other half were negative. Given that papers are about communicating specific ideas, it was surprising to see that 68% of the papers did not contain a single comment on at least one specific idea or claim in the paper….Very few papers (14%) contained comments that addressed the sufficiency or quality of the evidence, supporting ideas, or thoughts that were used to back up a claim (2/3 of the 14% were negative comments; 1/3 were positive) (p. 35)

6% of the papers contained at least one comment that provided scholarly advice to students. The “scholarly advice” category was created to include things such as references to additional sources of information, advice on how to continue on with a line of research, and instructor comments on the students’ patterns of errors and how to improve them systematically. Yet, after coding the data, we found no explicit comments that addressed “patterned errors/correction advice.” Only a few times did we find the error pattern identified. Therefore, the findings include mostly references to additional references and future lines of thought and inquiry. Although the frequency of these kinds of comments was rare, when they were utilized, they seemed to reflect a great deal of thought and personal investment by the faculty members. (They were often quite lengthy-even though we did not count the number of words for them, that feature was very salient.) (p. 36)

These paragraphs might be best summarized, simply, that teachers are identifying errors of the students but rarely engaging with the ideas. Only a third addressed the quality of anything!

The reason is clear in the second paragraph: teachers only write lengthy responses when they can give a “great deal of thought” and are personally invested. Otherwise, we don’t respond to the ideas of a student. Teachers need to be genuine readers. If students don’t have genuine readers, they won’t be genuine writers.

Until AI, most writing teachers could get away without needing to be discerning about a student’s ideas. They could default to spelling, grammar, or sentence structure. If a student had a bad idea related to an interpretation of Hamlet or the cause of World War I, usually that manifested in mechanics. Bad writing that was conceptually flawed could be first indicated by mechanical faults, which meant the teacher never needed to ultimately engage with the conceptual apparatus of the paper.

All of that ended with AI writing. We now have perfectly legible sentences and paragraphs, written in designed syntax. Most comments from teachers are thus rendered obsolete. (Note: I almost enjoy spelling or grammar errors now, if only because I sense the student actually wrote it themselves. Artisanal writing, perhaps.)

We need to engage with an essay’s ideas. We need to judge those ideas, not let them bounce around, like the words in a podcast.

Conclusion: What’s left?

I ask this question sincerely: what is left for me as a writing teacher? Rather than commenting on mechanics, I’ve found myself focused on two high level concepts: organization and examples. I ask students to improve their organization, which is surprisingly hard for an AI to do. Humans can improve the order of ideas, often providing examples or at least specifics. But even these facets will eventually be solved by the machines, maybe not in a year but in five or ten.

What’s left, then?

I think the answer lies in what I’ve replaced podcasts with: asking questions. I’ve started asking students a lot more questions. Questions that come from a genuine place. This has the benefit of showing I’m an engaged reader. But it also reveals how much they’re thinking about their writing. Very few writers have the opportunity to be asked about their work. Recall that Socrates claimed writing could only ever offer a solemn silence. Not true in our world. Writing is plentiful. Writing can be interacted in real-time, with rapid exchanges similar to orality.

If we teach our students to ask questions of their writing, that might just be good enough to improve their thinking and writing. It certainly has improved my own.