Open AI adds voice chat and images to ChatGPT.

OpenAI, the creators of ChatGPT, announced new voice and image capabilities for the AI assistant to enable more intuitive interactions. The new features allow users to speak conversationally with ChatGPT and show it images to get relevant assistance. The image capabilities are powered by the multimodal GPT-4V (GPT-4 with Vision), which Open AI showcased earlier this year.

What's going on here?

Users can now have natural back-and-forth voice conversations with ChatGPT and show it images to receive relevant responses.

What does this mean?

The voice feature allows real-time voice chats. Users can choose from 5 different voices and ask follow-up questions conversationally. The text-to-speech is powered by a model that can generate human-like voices from text in seconds. OpenAI collaborated with voice actors to create the five voice options and hasn’t yet provided the option to create a custom voice.

The image feature lets users show ChatGPT single or multiple images to get assistance interpreting them. It can help with daily tasks like meal planning from fridge photos or graph analysis for work. On mobile, a drawing tool focuses images. The image understanding uses multimodal GPT-3.5 and GPT-4 models to apply language skills to diverse image types like photos, screenshots, and text+image documents.

Why should I care?

The intuitive voice and image modes allow for new ways to use ChatGPT in daily life. Have an interactive travel conversation about landmarks. Plan meals by taking pantry photos. Help kids with math homework by showing worked problems. The new modes make ChatGPT more useful as a virtual assistant.

Open AI is also venturing into voice generation now. Synthetic voices enable new risks such as impersonation. It seems like OpenAI has considered it by limiting voice to chat for now. Image abilities to analyze people directly are also restricted.

These features are to roll out in the next two weeks, and if it goes all smoothly, OpenAI might have beaten Google (once again) in the race.