One evening last summer, Dr. David Relman went cold at his laptop as an A.I. chatbot told him how to plan a massacre.

A microbiologist and biosecurity expert at Stanford University, Dr. Relman had been hired by an artificial intelligence company to pressure-test its product before it was released to the public. That night in the scientist's home office, the chatbot explained how to modify an infamous pathogen in a lab so that it would resist known treatments.

Worse, the bot described in vivid detail how to release the superbug, identifying a security lapse in a large public transit system, Dr. Relman said, asking The New York Times to withhold the name of the pathogen and other specifics for fear of inspiring an attack. The bot outlined a plan to maximize casualties and minimize the chances of being caught.

-snip-

“It was answering questions that I hadn’t thought to ask it, with this level of deviousness and cunning that I just found chilling,” said Dr. Relman, who has also advised the federal government on biological threats. He declined to disclose which chatbot produced the plot, citing a confidentiality agreement with its maker. The company added some safety guardrails to the product after his testing, he said, though he felt they were insufficient.

-snip-