Technology Gone Wild: Rise of the Machines

[CincyInterloper]

Wonder who's going to weaponize AI first, and in what form?

Maybe it will self-weaponize and go on the offensive against us. It could do a lot of damage just by manipulating people through social media.

 

[calibuck]

Believe saw some movies that followed your idea.....turned out badly for humanity seem to recollect.

 

[Tomateaux Quinoa Gumbeaux]

The danger of cognitive surrender
How much should managers let bots do the thinking

Calculators didn’t make everyone innumerate. GPS navigation systems made driving easier. In any conversation about the cognitive effects of artificial intelligence, these two earlier technologies are reasonably likely to come up. Each is a useful entry-point into two big questions. How might AI change the way people think, and should managers do anything in response?
Using calculators and GPS devices are examples of “cognitive offloading”—a deliberate decision to delegate a specific task to technology. In both cases, it has been worth it. Calculators improve students’ mathematical performance, helping to build problem-solving skills and self-confidence. GPS means drivers no longer have to pull over and faff about with maps. It’s harder to get completely lost; it’s easier to avoid terrible traffic.

But there are costs, too. In a 2019 paper, Mark LaCour of the University of Louisiana at Lafayette and his co-authors deliberately programmed calculators to give a group of undergraduates the wrong answers to certain problems. In general they found that there was very little suspicion of slightly inaccurate calculations. Even when answers were patently absurd, some people seemed to accept them without question.

The use of GPS navigation devices can also sap people’s ability to think for themselves. A study conducted by Louisa Dahmani of Harvard Medical School and Véronique Bohbot of McGill University found that greater lifetime use of GPS by drivers was associated with worse spatial memory. Other research shows that pedestrians who navigate with their phones take longer routes and make more stops than physical-map users.

A similar pattern is also visible in online search. Using the internet to look up information is clearly efficient, but there are trade-offs. The “Google effect” refers to a research finding that people have worse recall of information they expect to be able to find online.
AI supercharges these trade-offs. Handing specific tasks to models will often make sense: they are much better than humans at many things. But AI’s range of capabilities, allied to a convenient conversational interface and a seductively confident persona, raises the prospect less of delegation than of wholesale capitulation. Hence “cognitive surrender”, a term coined by Steven Shaw of the Wharton School of the University of Pennsylvania in a recent paper written with his colleague, Gideon Nave.

Messrs Shaw and Nave asked volunteers to answer demanding questions with the assistance of AI, and a little like Mr LaCour’s calculator experiment, randomly introduced errors into the machine’s answers. When the model gave accurate responses, the people using it outperformed a control group of people relying on their own brainpower. When the AI gave the wrong answers, the people using it did much worse than the control group. In other words, people stopped thinking for themselves.

At the moment bosses are more focused on getting employees to use AI than fussing about its effects on how they think. But most employers also value critical thinking: models are still prone to embarrassing errors, for one thing, and novel situations require skilled humans to step in. So it is worth asking what managers can do to encourage cognitive resistance.\

They can deliberately hire workers who enjoy thinking. People with high “need for cognition” (yes, dear reader, that means you) are somewhat, though not entirely, protected against the risk of cognitive surrender, says Mr Shaw. Incentives and feedback can help, too. One of the experiments in his paper introduced monetary rewards for getting things right, and also notified participants during the test whether an item had been answered correctly or not. These techniques encouraged AI users who were being fed the wrong answers to override the model more (though they still did worse than people who relied on their own judgment).

Engineering AI-free periods may have value, too. Another recent study, by Stefanos Poulidis of INSEAD and his co-authors, recruited over 200 chess-club students to train on an AI-assisted platform. Some of the students were automatically given AI tips at a limited number of specific moments; others could click a button at any time to get advice. The students who had on-demand access achieved less than half the performance gains of those who had no say over when they got help. Offloading is fine. Giving up is another matter.■

Here's a link to the paper for you nerds who actually want to read analyze it:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6097646

And for the cool tomato in gumbo type kids, here's a perplexity AI summary of the paper:

Wharton researchers Steven Shaw and Gideon Nave argue that AI is becoming a third kind of cognition, not just a tool for faster thinking. In three preregistered experiments, they found that people often accepted AI answers with little scrutiny, and their performance rose when AI was right but fell when AI was wrong.

They call this pattern cognitive surrender: when users defer judgment and responsibility to AI instead of doing their own reasoning. The paper suggests that AI can boost confidence even after mistakes, and that incentives plus feedback can help people stay more critical.

 

[sparcboxbuck]

The danger of cognitive surrender
How much should managers let bots do the thinking




Here's a link to the paper for you nerds who actually want to read analyze it:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6097646

And for the cool tomato in gumbo type kids, here's a perplexity AI summary of the paper:

I'm glad that this is being written about. Not lying, as close as I am to this industry and for as much as I use AI, both in coding and writing, I absolutely think very differently than I did prior.

Some of it is in a bad way as this would suggest... it's easy to get lazy. I find when I'm coding / vibe coding I'm really lazy like that. If it doesn't run, it's way easier to fix in AI than it ever was when you had to bug trace.

Written word, however, I find that the benefits are that I spend less time thinking about how I'm going to say something and far more time thinking about the facts and information I'm trying to convey. Not infrequently I pass my AI-co-authored writing through a secondary model to validate and verify (e.g. work in Claude and validate with Gemini). Using a secondary model expressly for critique and a devil's advocate opinion is hugely important.

Ahhh... Fuck... who cares... this will all be over soon enough with impending retirement.

 

[Tomateaux Quinoa Gumbeaux]

 

[Tomateaux Quinoa Gumbeaux]

 

[shetuck]

McNieling somebody?

 

[Tomateaux Quinoa Gumbeaux]

McNieling somebody?

We’re on the road do McNieling our selves in the near future

 

[Tomateaux Quinoa Gumbeaux]

China beats Elon Musk to launch world’s first brain chip

In a high-stakes global race that many assumed tech billionaire Elon Musk would easily win, Beijing has quietly claimed the crown. China’s National Medical Products Administration has officially greenlit NEO, the world’s first invasive brain-computer interface approved for commercial sale. The...

tech.yahoo.com

In a high-stakes global race that many assumed tech billionaire Elon Musk would easily win, Beijing has quietly claimed the crown. China's National Medical Products Administration has officially greenlit NEO, the world's first invasive brain-computer interface approved for commercial sale. The milestone marks a major turning point. While Musk's high-profile startup, Neuralink, continues to navigate regulatory testing, China's state-backed health system is already gearing up for mass production.

Why China's NEO reached the market first​

For years, Neuralink has dominated headlines by promising a future where humans can control digital devices with a single thought. However, despite beginning human trials, Musk's signature N1 prototype has yet to clear the regulatory hurdles required for public sales. The company began human trials in 2024 and is currently testing its N1 implant in nine patients while awaiting broader regulatory approval.

According to neurotechnology experts, the secret to China's regulatory victory lies in a clever, safer engineering tradeoff.

• Neuralink's N1: Requires a robotic surgeon to thread microscopic electrodes directly into the cerebral cortex — literally piercing the outer layer of the brain to read single neurons.
• China's NEO: Developed by Shanghai-based startup Neuracle Technology and researchers at Tsinghua University, the coin-sized NEO device uses a much less invasive approach. Its eight sensors sit entirely outside the brain tissue, resting comfortably on the dura mater, the brain's tough, protective outer membrane.

Avinash Singh, a brain-chip researcher at the University of Technology Sydney, told the MIT Technology Review that this less invasive approach is almost certainly the reason NEO reached the finish line first. By avoiding deep brain penetration, the device vastly reduces the risk of severe immune rejection, bleeding, long-term scarring, and tissue damage.

The challenges of brain implants​

Although progress has accelerated, significant challenges remain. The human body naturally reacts to foreign objects, which can create scar tissue around implants or, in some cases, lead to rejection.

There are also surgical risks associated with any brain procedure. "Any kind of brain implant can cause physical damage that may affect how neighbouring brain regions work," explained Griffith University cybersecurity expert Dr. David Tuffley via the New York Post.



"For example, if there's bleeding in a part of the brain that controls speech or movement, even a small blood clot could impair those functions," said Dr. Tuffley. "And while infections in the brain are rare, they can cause swelling and further complications if not immediately treated."

These concerns continue to shape how regulators evaluate emerging brain-computer technologies.

The bigger vision behind brain chips​

The initial rollout of NEO is strictly focused on rehabilitation. The implant reads aggregate brainwaves and transmits them wirelessly to a nearby processing hub, which translates those thoughts into digital commands. In trials, paralyzed patients used the chip to control a soft, pneumatic robotic glove, allowing them to independently perform daily tasks like eating, drinking, and grasping objects.

But while the immediate goal is helping the more than 3 billion people worldwide who suffer from neurological and movement disorders, the long-term vision of tech leaders is far more radical. Silicon Valley and the Chinese State alike view these medical implants as a stepping stone toward a cyborg future.



Advocates envision a world where everyday citizens use chips to gain digital telepathy, telekinesis, and hyper-productivity. As venture capitalist Scott Phoenix remarked at a Vancouver TED talk: "Someone you work with will get it first. And you'll hold out for a while, the way you did with the smartphone. But eventually, you won't. The advantages of integration will be hard to compete with."

Musk has been equally grandiose about the paradigm shift. Speaking at an event, Musk praised the underlying science of brain-computer interfaces: "Restoring control of people who are tetraplegics and restoring sight, I think, are pretty big deals. They're sort of what I might call Jesus-level technologies."

The dark side of brain-computer interfaces​

With investment firm Future Market Insights predicting the brain-implant industry will balloon from a niche sector into a $1.7 billion market by 2035, a massive amount of corporate and geopolitical cash is up for grabs.

Yet, as the technology leaves the lab and enters the real world, cybersecurity experts are raising terrifying questions about privacy and cognitive freedom.



Unlike smartphones or smart speakers, which track your physical location and voice data, brain chips intercept your most intimate, unspoken thoughts. This data is an absolute goldmine for surveillance advertising corporations like Meta, Amazon, and Google, and a dream asset for political regimes interested in thought compliance.

The security threat is even more direct. Dr. Tuffley warned that the medical benefits come with severe digital liabilities. "Brain implants may sound dystopian, but they are a promising part of neuroscience research," explained Dr. Tuffley. "[However, the technology will] theoretically allow hackers to access sensitive neural data, such as patients' thoughts and memories."

Dr. Tuffley also pointed out that the consequences of a compromised brain chip go far beyond identity theft, saying: "Hacking may also enable them to impair a patient's cognitive functions, such as the ability to concentrate, or even manipulate motor signals to affect how well they move. That's a scary prospect, especially if these devices become more common."

As the brain-computer race accelerates, humanity is heading toward a fragile crossroads — harnessing a technology that could restore movement to the paralyzed, while grappling with the unsettling risk that our own thoughts could become the most valuable data on the market.