I assure you that you did not need an LLM to engage in, ahem, risky shenanigans, much before all this AI was ever a thing.
Sincerely, a former engineering student.
(Put another way - extracting for eg meth - or any such "dangerous"/illicit thing is stupidly easy for any engineering graduate who actually paid attention to their coursework. Hell, there are/were forums on one of the biggest red-colored, YC associated social media platforms that would tell you the steps for personal usage of these things.)
I don't doubt it. Bleach + ammonia is something anyone can make.
But I rather suspect there are improvements to be made in the realm that are a lot easier than building a uranium enrichment centrifuge hall under a mountain.
Do note that I'm not condoning lowering the bar. I'm merely pointing out that the bar was already quite low, and the current position of the bar is a small incremental change to anyone who actually knew where the bar truly lay to begin with.
I strongly recommend you read the book Amerithrax [0]. The book gives some historical examples of malicious groups [1][2] trying to use biological agents. Also, it is far harder to weaponize biological weapons than people think.
> In 1984, 751 people suffered food poisoning in The Dalles, Oregon, United States, due to the deliberate contamination of salad bars at ten local restaurants with Salmonella. A group of prominent followers of Rajneesh (also known as Osho) led by Ma Anand Sheela had hoped to incapacitate the voting population of the city so that their own candidates would win the 1984 Wasco County elections.[2] The incident was the first and largest bioterrorist attack in U.S. history.
Tried to take over a town by making all the voters too sick to vote on election day. This event is why all buffets & salad bars in the US now have sneeze shields.
> Aum Shinrikyo operated the most extensive biological weapons program by a non-state actor ever discovered. Aum considered a range of agents, but only seriously attempted to obtain and disperse Bacillus anthracis and botulinum toxin, the causative agents of anthrax and botulism. With the 2001 anthrax attacks, it comprises the only attempts to use anthrax as a weapon not attributed to a state program.
Tried multiple times to weaponize anthrax and failed. This was a group that made an automated factory to build AK-47s. Eventually, they spread sarin nerve agent in the Tokyo subway.
> Tried multiple times to weaponize anthrax and failed. This was a group that made an automated factory to build AK-47s. Eventually, they spread sarin nerve agent in the Tokyo subway.
What's most worrying is, Russia showed that you can use carfentanyl / fentanyl for the very same purpose, and that kind of stuff is something you can get shipped by the kilos as "research chemicals" from China or make it yourself.
I'm absolutely sure that even if claude gave me step by step instructions, I'd still be unable to produce a bio weapon. People fail at mixing milk and flour to produce a cake, and we expect them to produce weapons?
The ones with the required knowledge probably already know how to produce them, with nothing but public, easily searchable information.
Yes, but as with spam/phishing/abuse prevention, too much information about what does and doesn't trigger things can be very useful to attackers. An explicit error is something you can feed into another AI to find jailbreaks.
I think it's a fundamentally impossible thing to fix, though. There's no 100% correct answer.
I understand completely, and respect the tough spot they're in. They have a choice between human safety and cybersecurity/business needs here. I don't envy that position.
That said, this thing is in real production use with war fighters, doctors, and financial experts. Just YOLO'ing to a dumber model midway through a multi-step process and pretending everything is fine is not a real or defensible option. Someone is going to die, and its going to be the fault of whoever decided to make this the default rather than opt-in.
They are presumably building a KYC passport submission flow as we speak.
reply