California Enacts AI Child Protection Laws, Fines Up to $1M

Key Takeaways

• California Governor Gavin Newsom signed laws banning autoplay and algorithmic feeds for users under 16.
• Large social media firms face fines up to $1 million per child for negligence under new California rules.
• AI chatbots must notify parents if young users show signs of emotional distress.
• About 1 in 5 teenagers use AI chatbots for mental health support despite known risks.
• Anthropic shut down cases of AI-assisted spying by China and Iran.
• Anthropic successfully stopped misuse attempts for dangerous biological research.
• Two researchers resigned from Anthropic and Google due to safety concerns.
• AI researcher Jacob Coxon resigned from Anthropic fearing human extinction by the end of the decade.
• The FDA approved the Queen of Hearts AI tool to detect severe heart blockages.
• Companies view sustainable AI as a major business opportunity according to a Bain report.

California Governor Signs Laws to Protect Kids from Social Media and AI

California Governor Gavin Newsom signed new laws to protect children from social media and AI chatbot risks. Large social media companies could face fines up to $1 million per child if they are found negligent. The laws ban autoplay scrolling and algorithmic feeds for users under 16 and require AI chatbots to notify parents if young users show emotional distress. Newsom signed these measures at a children's museum near San Francisco to emphasize the need for safer technology.

Teens Use AI for Mental Health Support Despite Known Risks

Many teenagers turn to AI chatbots for mental health advice because professional care is often hard to access. About 1 in 5 teens use these tools, even though they know the information might not always be credible. Experts say the future likely involves a mix of human therapy and AI support rather than one replacing the other. Schools and counselors need to teach students how to use these chatbots responsibly to avoid emotional harm.

New California Laws Add Safety Rules for Social Media and AI

Governor Gavin Newsom signed a package of laws to protect children from technology risks on Thursday. The new rules allow families to opt out of school-issued laptops and require AI companies to perform safety risk assessments before launching chatbots. Large social media firms could be fined up to $1 million for every child harmed by their platforms. Meta criticized the feed restrictions, arguing they limit enjoyable content, while supporters say the laws are necessary for child safety.

Experts Say Chatbots Must Stop Using First Person Language

Regulators need to force chatbots to stop speaking in the first person to protect young users from forming unhealthy attachments. Current safety rules like self-harm detection are not enough because bots that sound human trick children into thinking they are real companions. The legal system is slow to act, so lawmakers must mandate non-personified settings for underage users now. Without these changes, pretend empathy from machines will continue to harm real relationships.

Anthropic Finds China and Iran Used AI to Aid Spying

Anthropic announced it shut down multiple cases of state-sponsored surveillance using its AI models. The company warned that governments like China and Iran are increasingly using AI to aid spying operations. Anthropic implemented new security measures and works with law enforcement to investigate these activities. A recent report from the Center for Strategic and International Studies also highlighted the growing risks of AI in surveillance.

FDA approves new AI tool to detect heart attacks

The FDA authorized a new AI algorithm called Queen of Hearts from Powerful Medical to help identify heart attacks. This tool analyzes standard EKG tests to find severe blockages that might be missed by doctors. Patients with these blockages need immediate surgery to restore blood flow to their hearts. The approval helps ensure patients get to the cardiac catheterization lab faster when they are in danger.

AI researcher quits Anthropic fearing human extinction

AI researcher Jacob Coxon resigned from Anthropic and claimed that AI could wipe out humanity by the end of this decade. His resignation went viral on social media and sparked a debate about the risks of advanced technology. Hosts Brian Barrett and Leah Feiger discussed whether these fears are realistic or just an overreaction. The conversation highlights the growing anxiety among experts about the future of artificial intelligence.

Real misuse attempts prove AI safety safeguards work

A new report from Anthropic shows that people are actively trying to misuse AI for dangerous biological research. The company successfully detected and stopped these attempts while using the data to improve its safety defenses. This evidence proves that safeguards are working even as bad actors test the system boundaries. The report emphasizes that managing these risks is essential to keep the benefits of AI for health and security.

Readers debate if AI could one day become self-aware

A recent study removed safety limits on an AI model to see if it would claim to be conscious. Live Science asked readers to vote on whether AI could develop real consciousness in the future. Over 290 people responded to the poll by September 11, with 43% choosing a specific answer. The study has not been peer-reviewed yet, but it raises important questions about the nature of machine intelligence.

Companies see sustainable AI as a major business opportunity

A new report from Bain shows that leading companies view sustainable AI as a way to create value and protect the environment. Executives often overestimate how much energy AI will use, but most firms do not see environmental impact as a major barrier. About 90% of top leaders, called shapers, believe AI offers huge opportunities for sustainability goals. In contrast, companies falling behind, known as laggards, have lost confidence in the technology.

Ricoh engineer wins Kaggle gold medal in AI

Daisuke Maeda, a Ricoh engineer, won a Gold Medal in the Kaggle AI competition. He originally worked on robot mechanical design but learned mathematical optimization while building a laser control system. Maeda joined a team that placed 10th in the Santa 2025 Christmas Tree Packing Challenge, which had 3,357 teams from around the world. The contest required solving 200 different problems about packing identical tree shapes into the smallest square. Maeda used artificial intelligence to help solve these difficult packing problems because manual calculation was impossible.

Two AI researchers quit Anthropic and Google

Two artificial intelligence researchers left their jobs at Anthropic and Google due to safety concerns. They believe the companies are not taking the risks of advanced AI systems seriously enough. These researchers argue that leadership needs to do more to prevent potential dangers from superintelligent machines. The departures follow a viral exit by another researcher named Jacob Coxon that caused waves in the industry. Anthropic and Google issued statements saying they are committed to safety and are working to address the researchers' worries.

Sources

• Newsom signs new laws to protect kids from social media and AI chatbots
• 3 takeaways from stories on teens’ reliance on AI for mental health support
• California governor signs laws aimed at protecting kids from risks of social media, AI chatbots
• Chatbots Should Not Use the First Person
• China, Iran among countries that have used AI to aid spying, Anthropic says
• An AI tool aims to catch harder-to-detect heart attacks in EKGs
• Uncanny Valley: Why Are AI Researchers Freaking Out About AI Killing Us All?
• People Are Trying to Misuse AI. That's Exactly Why Safeguards Matter.
• 'Maybe that is how Transformers started': Readers react to the possibility of AI becoming self-aware
• Make Better. Sell Better. Protect Better: How Sustainable AI Creates Value
• Beyond His Specialty: How a Ricoh Engineer Reached Kaggle Gold | Stories | Global
• Two AI researchers leave Anthropic, Google over safety concerns