Inflection-2 makes bold claims

Inflection, the AI startup by Reid Hoffman and Mustafa Suleyman has released their new LLM Inflection-2. They claim it’s the world’s second-best LLM after GPT-4 based on benchmarks.

What’s going on here?

Inflection AI releases its second LLM Inflection 2.

What does that mean?

Inflection-2 will soon power its consumer product PI (short for personal intelligence). Inflection-2 was trained on H100 GPUs (instead of A100s), resulting in lesser cost and higher speed while serving it, despite it being way bigger than Inflection-1.

They have not released details about model size, training size etc., just a bunch of selective comparisons with models. So, the claim of the second-best LLM is a bit fragile. But the performance on benchmarks is still impressive putting it in the top 3-4 LLMs alongside GPT-4 and Claude 2.

Why should I care?

Soon after formation, Inflection announced that it’ll create the largest cluster of H100 GPUs to train LLMs. And that move seems to be working out for them based on the claims about cost and speed. Though the team behind Inflection is pro-regulation and state certifications, so don’t expect the full power of Inflection-2 to be available via Pi.