Noam Scheiber at NYT writes about the Ariely / Gino falsification controversy and the generalized cesspool of behavioral science: Effectively, the accusation is that they either willfully or negligently caused data to be altered that supported their surprising, novel, pop-sci-book-type conclusions.
I am strongly inclined to take these criticisms seriously, and to look at much of the soft science around human behavior, decision-making, and mindfulness with a very jaundiced eye, because of an experience I had while at Harvard around the year 2000. In short, it was strongly implied to me by a grad student that a micro-celeb prof was ginning up completely spurious anecdotal nonsense in order to bootstrap a pet theory. And I had, it turns out, personally seen it happening.
(Note: I reserve my agita here very specifically for the soft “science” of social / psych / behavior stuff — that is, people who are purporting to apply rigor to the measurement and theoretical explanations of it. If you derive benefit from meditation or visualizing your desired outcomes, or you feel counting to 10 helps you make better food choices, good for you and keep it up!)
Around 1998, I got a work-study job at the computer help desk of the Faculty of Arts and Sciences, which is roughly speaking the main body of Harvard, including the College — everything except the professional schools like Law, Business, etc. By luck of the draw I got assigned to a small offshoot of FAS — the William James Hall help desk. WJH was a white mid-century modern tower that housed the Psychology department among other things, including an entire armored floor of monkey (or some other animal) experimentation that drew occasional protestor ire, and a sweet penthouse conference room where one could sometimes score leftover hors d’oeurves and opened bottles of white wine from the profs’ confabs. For reasons I never fully understood, but probably relating in similar measures to both the needs of Cat-5 cable runs and the egos of psychologists, WJH got its own standalone IT manager and help desk. The WJH subset of the help desk only served clients inside WJH, which meant importantly that during Cambridge winters we never had to travel further than an elevator ride. In short, it was a cushy gig. And, since I needed to post up hours as part of my financial aid package, I stuck with it as long as I could (I think I started in 1998 and stayed until 2000 or maybe 2001).
Maybe 5% of the job was mobile — all inside the building — to go power-cycle a professor’s computer or swap out a network cable. 95% was spent at the desk, which overlooked a nicely appointed computer lab mainly used by grad students, and otherwise was open to the hallway that ran past the elevators and to rows of offices of faculty and grad students. We’d answer the occasional phone call or email ticket, but mostly it was paid time to do your classwork or goof off (the WJH workload was also much lower than the FAS helpdesk workload from what I gathered).
One semester, I noticed a couple of new grad student faces on the floor. Reader, forgive me, and remember that I was probably technically still a teenage lad: the newcomers were a blonde and a redhead as I recall, somewhere deep enough into their 20’s that they were clearly untouchably beyond our undergraduate social world, and attractive enough that I and at least my male colleagues definitely took notice. I maybe spoke to them a handful of times over the year in the course of my duties, but never became familiar. (I mention their looks only because it’s part of why I remember the sequence of events — honest. Well, that, and it’s nice color for an otherwise dismal story.)
About a year had gone by — maybe it was 6 months, maybe 12-15 months — but I’d been aware of Blonde and Red for some time, to where their presence had faded into the background routine. Then something weird happened. I was at the desk, when down one of the usually very quiet hallways I heard and sort of observed some commotion. I looked over and there was a small dog and a few people. I think Blonde and Red were there – but it was a ways away, not not my business. There might have been more than one dog, but I only recall one, something small and golden. So I went back to whatever I was doing.
Some time later that day I was sitting alone at the desk when one of the grad students — Blonde, I think — came over and spoke to me.
“Hey.” She looked a little deflated, and didn’t appear to have a crashed laptop in hand.
“Hi. What’s up?”
“Did you see what was going on in the hallway earlier?”
“No,” I honestly replied. I mean, I’d seen something, but had had no idea what was going on.
“Do you know who that professor was?”
“No.” This was getting weird.
“That was Ellen Langer.” She kind of paused a beat. It was clearly a name that had some weight for her, but meant very little to me — I’d seen the name on directories, maybe on some help desk tickets or emails. “She was making us call her dogs.”
That seemed weird, but not extraordinarily weird. “Oh, ok.”
“It was supposedly an experiment.” She was, I think, so disenchanted or angry with the situation that she needed to vent to someone — clearly, pretty much anyone, including the IT help desk undergrad. “She made us sit there and do math problems in our head, and then, we both called the dog. She was trying to see if the dog would go to which of us did the more complex problem.”
“Did it?”
“No! The dogs just wandered around.” Her exasperation was apparent. “It’s supposedly part of mindfulness, but …” she kind of trailed off. We chit-chatted a bit more. I think I learned where she went to undergrad and what her name was, but I forget those things. The only other thing I remember about that exchange was that she seemed to be re-evaluating whether it was a good thing to have signed on to work with a domain-famous psych professor anyhow, if it was going to result the ignominy of being a prop for dog-obedience anecdata.
I kind of filed this away as a weird experience in my head. Later, I ran into the name Langer in a few different contexts, including the copy machine study, which seems sort of surprising but plausible (basically, giving a spurious reason, “because I need to make copies,” to cut the line for the copy machine, seems to work if it’s a small imposition on the people in front).
When, about 10 years later (and now 10 years ago), the Bem controversy came out (basically: a tenured Cornell professor published a peer-reviewed paper proving either the existence of ESP or the deficiency of social science peer review, depending on your view of reality) I remembered this little anecdote but I was a tad busy with a few startups and children, so I never wrote it up.
It would also be disingenuous to leave out here the fact that I am a little hesitant to tattle on, effectively, a co-worker, in writing. The sausage-making in almost any field of endeavor is a messy process, and especially having been a bootstrapped entrepreneur, I have respect for trying things that don’t scale (including sometimes, things that don’t have adequate experimental design or plausible mechanisms of action).
But I also studied History of Science (History and Science), and one of the key things you learn there is that the stuff that actually happened is not the highly abstracted and purified result of published papers and 4-color textbooks. The real stuff is messier and for career reasons, you’ll never see the tenured elite opening up as to what kind of tomfoolery led on the winding road to their (legit or falsified) results. You might get more of the real story from the lab techs, or the relatives of your experimental subject, etc. I used to think that it was tedious that historiographically we were always being pushed to inject class or power analyses into our writing, but now I think it’s axiomatic that you need to find people who are outside of or below a particular power structure to help fill in the truth if you really want to understand what happened in that structure.
Anyhow: the lesson here is that publishing, tenured professors even at the “best” university, not only in living memory but likely still to this day, may think it’s a good use of their mandate and their grad students’ time and IQ to perform worthless, un-designed, un-controlled, crap “experiments” to advance their woo-woo whims. It’s a fundamental unseriousness and egotism and, I fear, the corollary is that many who have already achieved positional power in the social sciences are likely to advance their intuitions and pet [sic] theories with total disregard for the ways for discerning and illuminating actual truth that our species has painfully earned over the centuries.
And: to that grad student who felt the need to unburden herself of her deep skepticism about Langer’s dog-calling junk “experiment,” I hope you found a niche of psychology where your efforts produce truth and understanding first, and hopefully also human well-being, whether or not they sell a bunch of bullshit self-help books and seminars. Your skepticism was warranted. And if you know more than I do about weaknesses in the corpus of 20th century quantitative social science, please consider speaking out now.