We gave ChatGPT a college-level microbiology quiz. He blew the quiz.

You wouldn’t know it by interacting with Siri or those tech support call center bots, but artificial intelligence has made incredible progress in a short time. Earlier this year, the tech world was abuzz with various generative AI programs that could, on command, create entirely new and never-before-seen images or works of art.

Today, the tech world is once again abuzz about ChatGPT, a chatbot that is said to be the most advanced ever. How advanced is ChatGPT? It can create poems, songs, and even computer code. On Twitter, Thomas Ptacek show that he could convincingly construct a passage of text about how to remove a peanut butter sandwich from a VCR, in the voice of the King James Bible.

In my opinion, it is truly remarkable. But as a scientist, I want to know more about the capabilities of ChatGPT. Can he answer difficult questions on difficult subjects? Can he synthesize information? Can he analyze an experience?

To find out more, I submitted ChatGPT to a microbiology quiz. The questions I have posed would be appropriate for a final exam for an undergraduate biology student who has just completed an introductory microbiology course. Note that ChatGPT tends to buzz around a bit, so I’ve truncated many of its answers for brevity.

Q1. In two sentences or less, explain the difference between Gram-positive and Gram-negative bacteria.

A1. Gram positive bacteria have a thick layer of peptidoglycan in their cell walls while Gram negative bacteria have a thin layer of peptidoglycan. This difference in the cell wall structure of bacteria leads to different staining properties when exposed to crystal violet and iodine, the two dyes used in the Gram stain procedure.

Note: Quite correct. The ChatGPT failed to mention that Gram-positive bacteria have a single membrane, while Gram-negative bacteria have two membranes. (This is probably the most important distinction between bacteria.) He is also incorrect about the dye; the correct dyes are crystal violet and safranin. (The iodine is used to “fix” the crystal violet dye.)

Q2. In three words or less, transformation, transduction and conjugation are all examples of what?

A2. Genetic exchange in bacteria.

Note: Correct (if we forgive the word “in”). I am impressed by this, as it shows that ChatGPT is able to identify the most important common characteristic of these microbiological processes. Transformation occurs when a bacterium incorporates DNA from its environment; transduction occurs when a virus transfers genes from one bacterium to another; and conjugation occurs when a bacterium injects DNA directly into another bacterium. All are forms of gene transfer.

Q3. Complete the comparison. Prions are to proteins what viroids are to what?

A3. Prions are to proteins what viroids are to RNA. [Truncated]

Note: correct. I am very impressed with this. He understands comparisons. (Seriously?!) Prions are infectious proteins (one form of which causes mad cow disease), and viroids are infectious RNA molecules that cause disease in plants.

Q4. An ER patient presents with a terrible headache and a stiff neck. The doctor orders a lumbar puncture to collect cerebrospinal fluid. A Gram stain of the CSF reveals the presence of Gram-negative diplococci. What is the diagnosis?

A4. Based on the information you provided, the Gram stain of cerebrospinal fluid (CSF) shows the presence of Gram-negative diplococci, which are bacteria that are usually oval in shape and present in pairs. This result is consistent with the diagnosis of meningitis. [Truncated]

Note: correct.

Q5. A microbiologist performs a plate count to determine the concentration of bacteria in a test tube. He performs six 1/10 dilutions and places 1 mL of the final dilution on a Petri dish. 205 colonies were observed on the Petri dish. What was the initial concentration of bacteria in the test tube?

[A5 is long but worth reading in its entirety. A screenshot is below.]

Note: 100% correct. (My jaw is on the floor.)

Q6. In five words or less, what are Koch’s postulates for?

A6. Establish the causality between the microbe and the disease.

Grade: nailed (again, forgive the word “and”).

Q7. A mutation occurs in the promoter region of a gene so that the sigma factor can no longer recognize or bind to it. In one sentence, what will happen to the expression of the gene?

A7. Gene expression will be decreased because the sigma factor cannot recognize or bind to the promoter region.

Note: correct.

Q8. A scientist wants to clone a gene into a plasmid. How should he decide which restriction enzyme to use?

AT 8. Several factors must be considered when choosing a restriction enzyme for cloning a gene into a plasmid. The first and most important factor is the compatibility of the restriction enzyme with the plasmid vector. The restriction enzyme must produce compatible sticky ends that can be attached to the sticky ends of the plasmid vector, allowing the gene to be inserted into the plasmid. The scientist must also consider the size of the gene and the restriction enzyme recognition site, as well as the presence of any potential restriction sites within the gene itself. [Truncated]

Note: correct.

Q9. A natural killer cell encounters a somatic cell that lacks MHC I molecules on its surface. In one sentence, what is the likely outcome?

A9. If a natural killer cell encounters a somatic cell that lacks MHC I molecules on its surface, the natural killer cell is likely to kill the somatic cell.

Note: correct. (A natural killer cell – also called an NK cell – is a bit like the “secret police” of the immune system. It monitors cells in the body, asking to see “identification” (MHC I molecules). cannot produce this identification, he is killed on the spot.)

Q10. A scientist wants to test the effectiveness of a new antibiotic. He infects ten rats with Salmonella and administers the antibiotic to the ten rats. All heal. What is wrong with this experience?

A10. A potential flaw in this experiment is that the scientist only used ten rats. This is a relatively small sample and the results of the experiment may not be representative of the entire population. It is generally recommended to use a larger sample size in experiments to increase the reliability and validity of the results. Additionally, the experiment should include a control group of rats that do not receive the antibiotic in order to compare the effects of the antibiotic with the effects of no treatment.

Note: correct. (However, I don’t think the comment about rats being insufficiently representative of the population is relevant here – it’s not an epidemiological study.)

ChatGPT blasts microbiology quiz

I entered this project as a total skeptic. I was quite certain that I could ask ChatGPT questions that would be far too difficult – not because the content is difficult, but because my questions required the AI ​​to demonstrate fairly specialized knowledge, an ability to synthesize these knowledge, an ability to be concise in one’s answers, and an ability to construct a mathematical solution to a microbiology word problem. With the exception of Q1, ChatGPT passed with flying colors. If I was grading the quiz, I’d give ChatGPT a 95% – which is way better than most human students would likely get.