To underscore the consequences of not having that kind of data, Smiley pointed to a recent attempt to rewrite SQLite in Rust using AI.

"It passed all the unit tests, the shape of the code looks right," he said. It's 3.7x more lines of code that performs 2,000 times worse than the actual SQLite. Two thousand times worse for a database is a non-viable product. It's a dumpster fire. Throw it away. All that money you spent on it is worthless."

-snip-

"Insurance underwriters are seriously trying now to remove coverage in policies where AI is applied and there's no clear chain of responsibility," said Smiley. "So now let's imagine you're the big four and you do get sued and you are having pricing pressure applied, the market's outpacing your ability to adapt, and now your underwriters are telling you, 'oh by the way we're not going to cover you.'"

-snip-

Insurers, he said, are already lobbying state-level insurance regulators to win a carve-out in business insurance liability policies so they are not obligated to cover AI-related workflows. "That kills the whole system," Deeks said.

-snip-