> AIs don’t solve problems like humans do. As other's have noted, Bruce, you do a great disservice when you conflate the current state of popular machine learning algorithms with the whole of artificial intelligence. The crux of the problem is that ML doesn't solve problems at all, it merely satisfies constraints. You've written about the security flaws that that leads to countless times (usually found via [adversarial machine learning](https://en.wikipedia.org/wiki/Adversarial_machine_learning)). > Separately, AIs can engage in something called reward hacking. That is not something limited to machines. Humans successfully gamed societal systems for millennia. Yes, ML offers new tools in our toolbox (e.g., ways to automate said exploits at scale), but it doesn't fundamentally alter the fact we often times *intentionally* design flawed systems. > Genies are very precise about the wording of wishes, and can be maliciously pedantic. I think the introduction of magical thinking is very apt here. I mean that in a big way. My academic background is in AI, but I also had quite a bit of interest in magic tricks when I was younger (with [James Randi](https://en.wikipedia.org/wiki/James_Randi) being a nice intersection with pseudoscientific thinking). Modern machine learning *is* a trick, not genuine, "strong" AI. We need to stop fooling ourselves into thinking it is as good, or as bad, as we imagine it to be. > This wasn’t AI — human engineers programmed a regular computer to cheat — but it illustrates the problem. Sadly, it only illustrates a first-order problem. As I said above, we humans not only think of ways to break the rules, we actively engage in breaking the *systems* that impose the rules. ML will have nothing on us until it is able to hire lobbyists and bribe corrupt politicians to create rules that are inherently unfair. The inequity is already operating at the meta level; these ML are just fighting over table scraps. > My guess is that this isn’t very far off, and that the result will be all sorts of novel hacks. Nope. The result is simply going to be the self-serving "hacks" that were intentionally put there by monied interests, but now you offer the possibility of them being exploited by everyone (or at least those who have access to the technology, in a very "Click Here to Kill Everybody" kind of way). > They will be able to engage on issues around the clock, sending billions of messages, and overwhelm any actual online discussions among humans. Already happening. No ML needed, because humans are quite happy make fools of themselves on social media. Even Elisa-level unsophisticated chat bots are enough to rile up the masses. We're rapidly approaching a tipping point where either a large segment of the population is lost to irrational thinking, or they realize how damaging social media is and just walk away. > An AI that discovers unanticipated but legal hacks of financial systems could upend our markets faster than we could recover. Again, humans are doing this now; don't fool yourself into thinking you only need to pay attention when an "AI" is involved. The whole [Robinhood/GameStop attack](https://en.wikipedia.org/wiki/GameStop_short_squeeze) that recently happened is a prime example. Everyone dead set at screwing everyone else on the trading floor, but the people that make the rules still always win in the end. > So in the long run, AI hackers will favor the defense because our software, tax code, financial systems, and so on can be patched before they’re deployed. Unfortunately, as I said, those flaws are intentional. Bought and paid for by people who do *not* want them fixed. If your "AI" favors their defense, it only means the inequity in the system will get increasingly more polarizing, and it will only lead to solutions that are . . . non-digital. b32dc79a7cb25be64039fe15d466af526bb97ebf22031a75b532c5e7fefb0b7d