OpenAI’s Sora 2 is putting safety and censorship to the test with stunningly real videos

OpenAI’s Sora 2 is putting safety and censorship to the test with stunningly real videos


OpenAI's Sora 2 sparks AI 'slop' backlash

Fresh off a $6.6 billion share sale that made it the world’s most valuable private company, OpenAI’s TikTok-style video app, powered by its new artificial intelligence model, Sora 2, is going viral.

Despite the gated release that requires an invite code, the video creation tool has already shot to the number three spot on Apple‘s App Store and sparked a wave of deepfakes, including a viral clip of CEO Sam Altman shoplifting GPUs.

Internally, the rollout has reignited a long-running debate inside OpenAI about how to balance safety with creative freedom.

A person familiar with internal strategy at the company said leadership views strict guardrails as essential, but also worries about stifling creativity or being perceived as censoring too much.

That tension remains unresolved.

OpenAI’s culture has long favored speed, often shipping new tools ahead of rivals and letting the public adapt in real time.

One former employee, who asked not to be named to discuss internal matters, told CNBC that during their tenure, OpenAI leadership had a pattern of prioritizing fast launches. That strategy was on full display after China’s DeepSeek released a powerful model at the end of last year that was cheaper and faster to build than anything out of Silicon Valley.

OpenAI responded within weeks, debuting two new models in what was widely viewed as a defensive move to preserve its lead.

But OpenAI has a key advantage: Its growing institutional muscle.

Once a scrappy research lab in San Francisco’s Mission District, the company has since become more structured, enabling it to spin up cross-functional teams more quickly and accelerate the development and deployment cycles for products like Sora.

OpenAI said Sora includes multiple layers of safeguards meant to prevent unsafe content from being generated, using prompt filtering and output moderation across video frames and audio transcripts. It bans explicit content, terrorist propaganda, and material promoting self-harm. The app also uses watermarks and bans likeness impersonation.

But some users have already found ways to skirt those protections.

Sora 2, the AI model powering OpenAI’s app, is a sharp improvement over the first version. The new system generates longer, more coherent clips that look strikingly real.

Multiple viral videos feature Altman after he granted permission for his likeness to be used on the platform, while others depict popular cartoon characters like Pikachu and SpongeBob SquarePants in unsettling roles.

The content has fueled criticism that OpenAI is once again moving faster than its own guardrails. Its use of copyrighted material — unless rights holders opt out — is consistent with the company’s current policy, though that approach is being challenged in court.

Altman has brushed off concerns, saying in a post on X that Sora is as much about transparency — showing the public what the technology can do — as it is about building commercial momentum to fund OpenAI’s broader ambitions around artificial general intelligence.

The launch comes amid intensifying competition. Meta rolled out Vibes last week, a new short-form AI video feed inside its Meta AI app. Google has Veo 3, while ByteDance and Alibaba have also debuted rival systems.

OpenAI, meanwhile, just committed to fresh spending of $850 billion, deepening its push into infrastructure and next-gen models.

OpenAI hits milestone $500 billion valuation

Experts say the push into video isn’t just about drawing more users into the ecosystem with another sticky consumer app.

Professor Hao Li, a leading expert in video synthesis, told CNBC that most AI systems today are still trained on linguistic data like books and internet text. But to move toward general intelligence, he said, models need to learn from visual and audio information, much like a baby discovers the world through sight.

“We use AI to generate content to then train another model to perform better,” he said.

Li added that his lab already uses AI-generated video to enhance model performance, feeding synthetic data back into the system.

It’s part of a broader trend among researchers who see video generation as a way to simulate reality and help models reason more like humans.

Former OpenAI executive Zack Kass, whose forthcoming book “The Next Renaissance: AI and the Expansion of Human Potential” explores the societal implications of artificial intelligence, echoed that view.

On the broader question of how model makers should approach deployment, Kass argued that the trade-offs of releasing powerful technology early are worth it.

“There are two alternatives to building in the open: Not building at all, or building privately. And those alternatives, to me, are worse,” he told CNBC. “If we have a groundbreaking technology, I think people should know about it and use it so that we can all update to it.”

WATCH: OpenAI cements status as the world’s most valuable private company

OpenAI cements status as the world's most valuable private company



Source

Salesforce beats on earnings, issues better-than-expected revenue forecast
Technology

Salesforce beats on earnings, issues better-than-expected revenue forecast

Salesforce CEO Marc Benioff delivers the keynote address at the start of the Dreamforce conference in San Francisco on Oct. 14, 2025. Jessica Christian | San Francisco Chronicle | Getty Images Salesforce reported better-than-expected earnings on Wednesday and topped Wall Street estimates with its revenue guidance for the fourth quarter. The stock rose 5% in […]

Read More
Nvidia CEO Jensen Huang talks chip restrictions with Trump, blasts state-by-state AI regulations
Technology

Nvidia CEO Jensen Huang talks chip restrictions with Trump, blasts state-by-state AI regulations

Nvidia CEO Jensen Huang said he met with President Donald Trump on Wednesday and that the two men discussed chip export restrictions, as lawmakers consider a proposal to limit exports of advanced artificial intelligence chips to nations like China. “I’ve said it repeatedly that we support export controls, and that we should ensure that American […]

Read More
Dan Niles says his best idea right now is cash. Here’s what else he likes
Technology

Dan Niles says his best idea right now is cash. Here’s what else he likes

In a market that could soon be topping out, the best investment idea right now is cash, according to Dan Niles, founder of Niles Investment Management. The market moved higher on Wednesday, but Microsoft shares fell after The Information reported the company was slashing software sales quotas tied to artificial intelligence. Microsoft denied the report […]

Read More