This AI-generated Minecraft could represent the future of real-time video generation

“What if you could say, 'Hey, add a flying unicorn here'? Literally talk to the model. Or “Turn everything here into the Middle Ages,” and then, boom, everything is medieval. Or “Turn this into.” star Wars' and that's all star Warssays Leitersdorf.

A major limitation at the moment is the hardware. They relied on Nvidia cards in their current demo, but in the future they plan to use Sohu, a new card Etched has in development that the company says will improve performance by a factor of 10. This gain would significantly reduce the cost and energy required to produce real-time interactive videos. This would allow Decart and Etched to create a better version of their current demo, allowing the game to run longer, with fewer hallucinations, and at higher resolution. They say the new chip would also allow more players to use the model at the same time.

“Tailor-made chips for AI hold the potential to enable significant increases in performance and energy efficiency,” says Siddharth Garg, a professor of electrical and computer engineering at NYU Tandon, who is not affiliated with Etched or Decart.

Etched says its profits come from designing its cards specifically for AI development. For example, the chip uses a single core, which supposedly allows complicated mathematical operations to be handled more efficiently. The chip also focuses on inference (where an AI makes predictions) over training (where an AI learns from data).

“We are building something much more specialized than any chips on the market today,” says Robert Wachen, co-founder and COO of Etched. They plan to do projects on the new map next year. Until the chip is deployed or its capabilities are verified, Etched's claims have yet to be substantiated. And given the level of AI specialization already present in the top GPUs on the market, Garg is “very skeptical of a 10x improvement through smarter or more specialized design alone.”

But the two companies have big ambitions. If the efficiency gains are close to what Etched claims, they believe they will be able to generate virtual doctors or tutors in real time. “All of this is coming to us and is based on better architecture and better hardware to operate. So that’s what we’re really trying to get people to understand with the proof of concept here,” says guards.

Currently you can try out the demo of their Minecraft version here.