Openai has released a new tool based on artificial intelligence: it can create 3d models from text, like dall-e with images or chatgpt with text. An ai that could well help creative people in their 3d productions.
Openai has announced the release of a new tool for creating images: point-e, which can create 3d point clouds from text. The start-up linked to elon musk and specialized in artificial intelligence is doing it again after dall-e and chatgpt, which have been very successful in the last few months, as the possible uses are impressive.
According to the team led by alex nichol, “Point-e can often produce coherent and high quality 3d shapes for complex queries”, and in color. If it’s not the best tool out there, its creators are certain: it’s the fastest by far.
An IA that requires far fewer resources than others to create 3d objects
For the openai engineers, the greatest feat of their tool is its speed and by extension the low computing power to make it work. In a scientific paper, they explain that for generating 3d objects via text, “The most recent methods typically require several hours of gpu power to produce a single sample.” this is a lot when you consider that dall-e can create images in seconds and meta can even create videos.
Openai therefore decided to use an alternative method “That produces 3d models in only 1-2 minutes on a single gpu”. So far, point-e has been trained on millions of 3d models, which could be converted into a standard computer format.
It is explained that the artificial intelligence first generates a single view using a text-to-image model (like dall-e). From this generated image, the program creates 3d points. At the end of this step, the model consists of 1024 points, which the program then refines to 4096 points.
Why dot-e does better than other artificial intelligences
These two steps take only a few seconds independently, which makes it possible to create 3d models much faster. Point-e intends to combine the advantages of two generation methods:
- The first, which involves training 3d generation models from matched data;
- The second one, which exploits pre-trained text-to-frame models to then represent these images in 3d.
It is precisely by detaching itself from 3d that point-e manages to reduce the necessary computing power, contrary to what other experimental ais are doing at the moment. Although other programs manage to generate three-dimensional elements, the developers explain that the main problem encountered is “The optimization procedures”, which require a lot of graphic resources. This is what prevents the creation of practical tools.
The other great strength of point-e is that it uses images rather than 3d objects as the basis for training: the latter are available in much larger quantities. This allows it to generate elements of all types and from more complex descriptions.
The possible uses of this openai tool
Point-e could revolutionize the creation of 3d content and thus help modelers in their work, whether for the cinema or for video games. One can imagine an integration in a game engine, like unreal engine, which would allow to integrate objects generated in a few seconds in a virtual universe, from a textual description.
Especially since its competitive advantage is probably the most practical: its speed of execution. Opposite him, there is also google’s dreamfusion, which also has great power in this field.
This will help in the creation of metaverses and more generally of virtual reality/augmented reality universes. For the time being, the software still has some limitations. First of all, the definition of the generated objects is still quite weak, but the engineers have a lot of ideas.
Of course, there are the biases that automatic generation tools can suffer from and this is part of the reason why point-e is not available to everyone on the openai website. One could very well ask this ai to generate a weapon plan for example, which should be avoided at all costs. However, point-e has been put online on github and everyone can consult the content of the project since it is open source.
More on artificial intelligence by following this FB page!