What exactly is Apple Intelligence?

For the first hour of WWDC, I only heard intelligence three times. In the next 40 mins, I couldn’t keep count as Apple rebranded “Artificial Intelligence” to “Apple Intelligence”

So, here’s the TLDR:

OpenAI partnership is minor. Apple relies on its own models for a lot of the stuff.

Timmy wants Apple Intelligence to be “powerful, intuitive, integrated, personal and private”. Features go live this fall, we’ll see how powerful or intuitive they are.

AI will be integrated into three parts: Writing tools, Image generation and Siri which takes action. Nice, meh and woah—in that order.

Everything you do on your device gets stored in a semantic index that’s used to make AI personal. Apple Intelligence chooses between on-device models and in-cloud models. New Private Cloud Computing will make cloud inference private.

Apple’s Models and OpenAI Partnership

Apple has 3 levels of LLMs to power these features:

• Apple On-Device: A 3B model, similar in performance to Gemini 1.0 Pro.

• Apple Server: A larger model hosted on Apple's private servers (possibly Claude Sonnet-like performance).

• 3rd Party Models: Starting with GPT-4o via ChatGPT, but more like Google's Gemini models are expected.

Apple Intelligence decides which model to use based on your request. Before talking to ChatGPT, it will always ask for your permission.

With the OpenAI partnership, Siri can talk to ChatGPT without you needing to create a ChatGPT account. You can connect your existing account if you want to. The partnership, though, is limited to sending a question to ChatGPT via Siri or in very constrained writing and image generation widgets. This can grow if ChatGPT’s models drive Siri’s capability to take actions further down the line. OpenAI integration is coming later this year.

Most image-generation tasks are also done on-device with local models—though you can create images via ChatGPT if you’ve connected your ChatGPT account to Apple Intelligence.

Let’s dive deep into the features that these models enable.

Siri 2.0 preview- Taking actions

Siri's got a fresh new look and a serious upgrade under the hood, thanks to Apple Intelligence. The new design is sleeker, the language understanding is sharper, and you can even type your requests if you don't want to talk out loud.

But what makes me go Woah is the ability to take action across all apps using Siri. Mind you, it’s not gonna launch anytime soon but whenever it does it’ll be massive to have an assistant that isn’t just a dumb parrot.

Siri's also getting smarter about your personal context, thanks to a Semantic Index that stores all your activity on your devices. So, if you ask, "When is Mom's flight landing?", Siri can combine info about who your mom is, flight details from texts, and current flight status from the web to give you a precise answer.

On the near horizon, Siri will get:

• Screen awareness: Siri will understand and interact with what's on your screen. For example, if you see a new address in a text message, you can tell Siri to add it to your contacts.

• Product knowledge: Siri will become an expert on your Apple devices. Need help setting a timer or adjusting a setting? Even if you don’t know the exact feature name, Siri will guide you through it.

Writing Assistance

You get this tiny writing assistant window whenever you write across all your apps. It can proofread and rewrite your writing with built-in presets like friendly, professional, and concise, or convert existing content into summaries, key points, tables, or lists. You can also provide custom instructions—a much cleaner UX than copy-pasting to a chatbot window.

Another set of writing features summarizes your emails, calls, and voice notes, and suggests auto-replies. The baked-in features can be helpful if they work as demos, but I'm sceptical.

Last year, Google's AI features were powered by Gemini 1.0 Pro. Those features felt non-existent because the model wasn't smart enough to understand my requests. The writing and summaries were not up to the point. I don’t expect the on-device models to do much better than that. My hopes rest on the Apple Server model—it could serve well for a wide range of writing tasks.

Image generation

Image generation is also coming to Apple devices. It’s limited to three styles: animation, illustration, and sketch. The new Image Playground UI is stellar and feels magical, but the outputs and restrictions definitely look Meh. Or we’re all just spoiled by Midjourney I guess.

You can add elements you want (including contacts), choose a style and the on-device models create your image. It brings image generation to messages, notes and custom emojis that Apple calls GenMoji.

AI under the hood

Apple's new Calculator on the iPad is undoubtedly the biggest news from this WWDC. They call it machine learning but interactive calculations based on your inputs in notes and calculators are an insane use of AI behind the scenes.

Similarly, the writing LLMs also power a lot of focus & productivity features where notifications are prioritised or emails are auto-grouped. The image models power features to create automatic memory movies, search across photos and videos or remove unwanted objects from images.

Is Apple Intelligence a huge deal?

Yes, if discovery and action work as promised (like understanding your mom's flight info, searching through videos, adding events to the calendar etc.). The text and image tools are cool, but serious users probably won't be impressed by the performance of small, on-device models. They will likely go to other tools like ChatGPT, Claude and Gemini.

Apple is bringing the AI to the masses. Sure these are not powerful; features but they’ll make life easier for a lot of us.

There's the whole "coming later" thing as we see with Google too, but these features are supposed to come with the next OS versions for iPhone Pro and iPhone Pro Max, Macs with M1 and later chips and iPads.