The NovaSky research team from the Sky Computing Lab at UC Berkeley has introduced a new prototype model, Sky-T1-32B-Preview, which is able to compete with early versions of OpenAI on key benchmarks. This is the first truly open understanding model that can be reproduced from scratch, as the team has shared the training data and the necessary code. Impressively, training Sky-T1-32B-Preview cost less than $450. For comparison, models of similar performance recently cost millions of dollars.
Sky-T1 uses synthetic data, which helped reduce costs. This is a model that effectively self-verifies, avoiding typical mistakes. Unlike many AIs, it requires a bit more time to solve tasks, but its solutions are more reliable in fields such as physics, science, and mathematics. NovaSky used the Alibaba QwQ-32B-Preview model to generate the initial data, and then processed it with OpenAI’s GPT-4o-mini.
Sky-T1 has 32 billion parameters and was trained for 19 hours on eight Nvidia H100 GPUs. The model outperformed the early o1 version on the MATH500 and LiveCodeBench tests. At the same time, it lags behind on GPQA-Diamond, which tests knowledge in physics, biology, and chemistry.
NovaSky plans to continue developing open models with advanced understanding capabilities. The team will focus on creating more efficient models that retain strong comprehension skills and on exploring techniques that improve efficiency and accuracy during testing.
