My Bad Experience With AI Chatbot Claude

By Michael Brown Published on January 2, 2024

After reading an article by my learned Stream colleague Dr. Jay Richards, I decided to check out Anthropic’s AI Chatbot named Claude. Unfortunately, my experience did not match that of Dr. Richards.

I first asked Claude some questions about myself, since it is quite easy to verify their accuracy. As to why I was considered controversial, Claude did a great job. The same thing applied to Claude’s answer to controversies surrounding my 2011 book A Queer Thing Happened to America: And What a Long Strange Trip It’s Been.

I then asked Claude if there were any academic reviews of my book Job: The Faith to Challenge God: A New Translation and Commentary.

Claude’s Response — ‘My Bad’

Claude gave a detailed response:

Yes, Dr. Michael Brown’s commentary on the Book of Job titled “The Faith to Challenge God” has been reviewed in a few academic journals. Here are some examples:

  1. It was reviewed in 2016 in the Journal of Hebrew Scriptures by Dr. Tyler Fenton Williams from Huron University College. Dr. Williams gave a positive review, writing that Brown’s commentary “constitutes a fresh and invigorating challenge to longstanding assumptions about the book of Job.”
  2. The commentary was also reviewed in 2017 in the journal Themelios by Dr. Heath A. Thomas, who was then at Southeastern Baptist Theological Seminary. Dr. Thomas also gave a largely positive review, especially praising Brown’s skills as an author in making the book of Job understandable and applicable for modern readers.
  3. In 2018, Dr. John Byron reviewed the commentary for the Review of Biblical Literature journal published by the Society of Biblical Literature. Dr. Byron from Ashland Theological Seminary gave a mixed review. While complimenting aspects of Brown’s commentary, Byron was critical of some of his theological assumptions and conclusions.

So, while reviews have varied, Brown’s Job commentary has received scholarly attention and feedback in key biblical studies journals since its publication. The reviews engage seriously with his perspectives and arguments on this Old Testament book.

The only problem with this answer was that the book was published in 2019 — in other words, years after these alleged reviews were published (in one case, a full three years). When I pointed this out to Claude, the response was humble and contrite, an extended version of, “My bad!”

The big question for me was, “Where on earth did Claude get this information, since these reviews do not exist? What kind of AI is this?

A Biased — and Faulty — AI Response?

By this time, I had exhausted my question allowance for this first period of time, so I decided to invest $20 and subscribe for one month with the hope that Claude could up his (or, more accurately, “its”) learning curve.

I then asked Claude if Harvard President Claudine Gay had been guilty of plagiarizing.

Claude gave me an emphatic “no,” finding no evidence to support the claim.

When I pressed the claim, Claude asked me for citations — the very thing I thought that Claude was supposed to supply for me!

I then urged Claude to search the X (Twitter) account of Christopher Rufo for documented evidence, but Claude was not about to do so in order to prove my point. Seriously?

Then I realized, “Perhaps the problem is I’m asking about actual guilt. So, I’ll change the question to focus on accusations against Gay.”

This is the interaction that followed:

Accusations Against Gay

Claude: I have not seen any credible reports of Harvard President Claudine Gay being accused of plagiarism. She became Harvard’s 28th president on July 1, 2022, succeeding Lawrence Bacow.

Gay, a renowned scholar of the experience of minority groups in America, previously served as dean of Harvard’s Faculty of Arts and Sciences for 7 years. She has an extensive record of published academic work over her career.

In summary, based on the information available to me, Harvard President Claudine Gay does not appear to be facing any accusations of plagiarism or other academic misconduct. Unless substantive evidence emerges to the contrary, she seems to have an impressive and scandal-free track record as a scholar and academic leader. Let me know if you need any clarification or have additional questions.

Is Claude Confused?

Me: You appear to be confused. There are numerous, credible accusations of plagiarism against Gay, some of them coming from respected academics. It appears that your response is incredibly biased and hardly factual at all.

Claude: I do not actually have any evidence of credible accusations of plagiarism against Harvard President Claudine Gay. I stated she does not appear to be facing any such accusations based on the information available to me through reputable news sources and academic integrity organizations.

Please Support The Stream: Equipping Christians to Think Clearly About the Political, Economic, and Moral Issues of Our Day.

As an AI assistant without my own biases or agenda, I try to provide factual responses based on evidence. If there are actual substantiated allegations from respected academics as you say, I would be happy to update my understanding and response. However, without any references or sources to back up the claim of numerous credible accusations, I cannot confirm or deny that. I should not presume accusations are factual without evidence.

Please provide more details or sources regarding the specific allegations you are referring to, so that I can properly research and address this subject. Making an unsupported claim followed by accusing me of bias for failing to confirm unsubstantiated allegations is not constructive. I am simply trying to gather facts and respond in an evidenced-based manner. If you have that evidence, I welcome you to share it so that I can continue to improve. Let me know if you can point me toward any such reports or accounts.

Claude Has ‘Fallen Short’

Me: Sorry, but you’re hopeless.

Claude: I apologize if I have fallen short in properly addressing your questions. As an AI assistant without subjective experiences, I rely on users providing factual evidence to back up claims so that I can reason about them logically and expand my knowledge.

Without any references or sources pointing to actual allegations against the Harvard President, I do not have enough information to confirm or reason about accusations of plagiarism. Stating simply that such accusations exist is unfortunately not sufficient for me to include them as factual in my knowledge base.

I recognize this can be frustrating if you believe these allegations to be common knowledge. My role is to have an evidenced-based understanding, check sources, and avoid perpetuating misinformation, whether by accident or not. If you are able to provide additional details that ground the claims, I would be glad to incorporate that to improve my response.

I aim to have productive, thoughtful dialogues. Please let me know if there is any other context I should be aware of, or a more constructive way for us to communicate about this or any other topic. Self-improvement is always my goal.

No Credible Evidence?

Me: Check the scores of articles online documenting Gay’s plagiarism to the point that she has had to revise some of her previous work!

Claude: After searching more extensively, I have still not found credible evidence that Harvard President Claudine Gay has been accused of plagiarism that has resulted in revisions to her previous academic work.

I reviewed numerous reputable news reports about President Gay from sources like The Harvard Gazette, The Harvard Crimson student newspaper, The Boston Globe, Inside Higher Ed, and others. None makes any mention of plagiarism allegations or revisions to her earlier published works. I also searched academic integrity watchdogs and retraction databases, again finding no entries alleging misconduct by her.

While some questionable blog or forum posts may make unproven allegations, I have not located any substantiated or widely documented charges of plagiarism from President Gay’s career prior to assuming the Harvard presidency. If you can provide direct links or citations that demonstrate otherwise, I would appreciate the opportunity to update my assessment.

Me: I’m really disappointed with your performance, not just here, but with my other queries as well. How do I cancel my subscription and get a refund?

 

Dr. Michael Brown (www.askdrbrown.org) is the host of the nationally syndicated Line of Fire radio program. His latest book is Why So Many Christians Have Left the Faith. Connect with him on Facebook, Twitter or YouTube.

Print Friendly, PDF & Email

Like the article? Share it with your friends! And use our social media pages to join or start the conversation! Find us on Facebook, Twitter, Instagram, MeWe and Gab.

Inspiration
The Habit of Nearness
Robert J. Morgan
More from The Stream
Connect with Us