OpenAI reveals AI voice assistant in ChatGPT.

OpenAI wrapped up its spring updates. We have a new model, free intelligence and the glimpses of HER.

What is going on here?

OpenAI announces GPT-4o and updates to ChatGPT’s free tier.

What does this mean?

These spring updates can be broken down into two main parts:

1) New model GPT-4o and HER. O stands for Omni. GPT-4o is trained on text, vision and audio from scratch and it can output these modalities natively. These fundamental capabilities make an assistant like HER possible.

If you don’t know HER, we’re talking about the 2013 movie where a man falls in love with Samantha—an AI assistant.

Like Samantha, the new model can allow ChatGPT to see you, hear you and respond with speech suitable to the situation. The model could change the tone and style of response in real-time, pause when you interrupt and understand emotions. It’s better to see and hear her 😉 .

More about the model: It’s 2x faster and 50% cheaper than the GPT-4 Turbo, but performs better on benchmarks.

Voice mode makes shiny demos, but the full model can understand videos & audio, create 3D animations, and answer more complex queries, which also opens up commercial possibilities. See these demos for example:

• Real-time translation with GPT-4o. [link]

• Solving Math Problems with GPT-4o. [link]

• Rock paper Scissors with GPT-4o. [link] (need some fun too, right?)

• Two GPT-4os interacting and singing. [link]

• GPT-4o for customer service support. [link]

• Bringing ChatGPT desktop to a meeting. [link]

More demos on OpenAI’s Twitter and YouTube accounts.

2) It’s a house party and we've got free intelligence. Most of ChatGPT Plus’ features are now available to free users. These include:

• web browsing

• code interpreter

• uploading files and images

• memories

• GPTs and the GPT store (huge!)

And take a breather, that’s not it. The new GPT-4o is also available for free users (approx. 16 messages/3 hours vs. 80 messages for paid users). So, the most intelligent AI model in the world is now free on ChatGPT.

Also, ChatGPT is getting a desktop app for Mac. Windows is coming later this year.

Why should I care?

GPT-4o is available on both ChatGPT and API but without the extra modalities right now. Image and video understanding are coming soon. It might be some time before we get audio though. Plus users will get “alpha access” to audio features before general rollout, but assume that everything is going free.

If you’re a Plus user, you might think it’s not worth paying anymore, but resist the urge to anger our agi-lord SAMA. The heavens of OpenAI will deliver more gifts soon. Mira Murati mentioned a major upgrade to their models later this year. The new desktop app also means Agents are closer than they appear.