Midjourney is building the Holodeck — new AI model lets you ‘enter’ 3D images

Leading AI image generator Midjourney is working on a new feature that completely changes how we engage with generated images. Using new forms of 3D technology, you’ll be able to effectively step into the image.

Announced during the most recent Midjourney office hours on Discord it’s just one of several of new features in the works that could be launched between now and the end of this year. Midjourney CEO David Holz also said we’d see v7 before the end of the year , a video model and an upgraded image editor that lets you edit external images for the first time.

The startup has been working on 3D technology for some time as part of a wider vision of creating something akin to a Star Trek Holodeck where you can generate a world and use that world to make movies or games or just spend time.

How will the Midjourney 3D model work?

According to generative AI expert Martin Nebelong , Midjourney’s 3D model will be an entirely new approach, building on NeRF technology widely used in game development. A NeRF is a neural network able to reconstruct 3D scenes from a 2D image.

Few specific details have been released but Holz has spoken several times in the past about wanting to build a virtual world where anyone can interact and build on. During the recent Office Hours he spoke of wanting people to easily convert a Midjourney image to a 3D environment.

It would “allow camera motion within certain limits” and could have 60 frames per second rendering and a camera path system to allow video output. This is independent of the planned future video model that would likely be closer to the likes of Runway or Sora.

Other companies like Luma Labs, Adobe, and Meta have text-to-3D models but these tend to be object-based, rather than environments. Roblox is working on AI-generated environments and startup Cybever launched a waitlist for an immersive text-to-3D world model.

What else was announced by Midjourney?

According to a post on X by Alfonso Rosenberg, the main focus of the recent Midjourney office hours was on more immediate updates including around personalization. For example, for some modes, it will be on by default and users will be able to more easily refine results simply by making a selection out of four generated images.

There will also be an updated image editor that will allow external images to be edited within Midjourney although this will come with more restrictive moderation and privacy protections.

Midjourney’s video model could arrive before the end of the year but it will be better at illustrative styles than photographic ones. Finally, Version 7 of the Midjourney model is in the training phase and will be out in more than a month but less than three.

Rosenberg, who was quoting a Discord thread by JamesGriffing added that there were two hardware projects in the works, a new explore page was coming and they were testing a storytelling tool due to release this year for "world-building rather than just image creation."

More from Tom's Guide

• Best food delivery services: Grubhub vs Uber Eats vs Doordash
• The best cast iron skillets 2024: Tested and rated
• How to make images with AI using Leonardo