Elon Musk’s AI startup X.ai has unveiled its latest generative AI model, Grok-1.5. Intended to power social network – at least judging by published results and benchmark specifications.
Grok-1.5 benefits from “improved reasoning,” according to X.ai, especially when it comes to coding and math-related tasks. The model more than doubled Grok-1’s score on a popular math test, MATH, and scored more than 10 percentage points higher on the HumanEval test on programming language generation and problem-solving abilities .
It is difficult to predict how these results will translate into real-world use. As we recently wrote, commonly used AI benchmarks, which measure things as esoteric as performance on higher-level chemistry exam questions, fail to capture how the average person interacts with models today.
An improvement which should lead to observable gains is the amount of context that Grok-1.5 can understand compared to Grok-1.
Grok-1.5 can process contexts of up to 128,000 tokens. Here, “tokens” refers to chunks of plain text (e.g., the word “fantastic” split into “fan,” “heap,” and “tick”). Context, or popup, refers to the input data (in this case, text) that a model considers before generating output (more text). Models with small pop-ups tend to forget the content of even very recent conversations, while models with larger contexts avoid this pitfall and, as an added benefit, better understand the stream of data they are absorbing.
“(Grok-1.5 can) use information from much longer documents,” X.ai writes in the blog post. “Additionally, the model can handle longer and more complex prompts while maintaining its ability to follow instructions as its pop-up window expands.”
What historically sets X.ai’s Grok models apart from other generative AI models is that they answer questions on topics that other models are typically off-limits, like conspiracies and more controversial political ideas. The models also respond to questions with “a rebellious streak,” as Musk described it, and with downright foul language if asked to do so.
It is unclear what changes, if any, Grok-1.5 brings in these areas. X.ai does not allude to this in the blog post.
Grok-1.5 will soon be available to early testers on X, along with “several new features.” Musk has previously alluded to the need to summarize threads and replies, and suggest content for posts; we’ll see if these arrive soon enough.
The announcement comes after X.ai’s open source Grok-1, but without the code needed to refine or train it further. Most recently, Musk said that more users on pay $16 per month).
techcrunch
