Generative AI: 3D Object Generation (OG)

3D Object Generation (OG). Generating objects like furniture, cars, boats, what have you. The output of such an AI system could be a CAD drawing or a similar 3D representation.

Glossary 📙
zero-shot learning = Solving problems like classification without having seen examples of that problem before.
WIP = Work in progress.

3D OG’s history ⛪
2014 - Goodfellow presents the first Generative Adversarial Net (GAN) [1].
2016 - Stanford presents the first 3D-GAN, generating 3D objects [2].
2018 - Improved 3D OG from 3D-GANs [3].
2019 - Pix2Vox is alive. An interactive tool where one can submit a rudimentary drawing of an object, eg. a chair, and the system generates and visualizes it in a 3D-voxel space. Check out [4].
2020 - Pix2Vox++ is out, which is the successor of Pix2Vox [5].

From here on there are many ideas spreading in different directions. For instance, OG with text only (zero-shot) [6], controllable OG [7], and multi-category OG [8]. We’ll take a closer look when the time comes. 🖤

See [3] for a transformation of a pixelated image into a 3D rendering:

Impressive!

And, what could this mean? 💡
Most obviously, you could generate objects with models like this. All kinds of objects. One of the limiting factors here is the dataset. On Google’s dataset search [9] one can find datasets that represent or are similar to the desired output. The sky is the limit!

  • All kinds of data - eg. generate fashion products: Dresses, shorts, bags, shoes, etc. This is a large shoe dataset [10].

  • Further use cases seen: furniture, cars, boats, and interior design.

There are many ways to get started and, thus, I would like to propose an easy-to-follow 5-step guide, probably called PESCI. (WIP, teaser: 1. Papers, 2. Experimentation, 3. Dataset Search, 4. Customer focus, 5. Iterate) If you would like me to prioritize it, please let me know.

GAI Gems 💎

  1. An awesome, trippy AI-generated music video [11]

  2. GAI Quizmaster on Covid restrictions (after 3 attempts my highscore is 4 😅).

  3. Generated Art in 3D, but 3D-printed.

References:
[1] Generative Adversarial Networks.
[2] 3D-GAN, watch video.
[3] High Resolution 3D Object Representation
[3] Pix2Vox paper and its web page.
[4] See Pix2Vox++ GitLab.
[5] Google Colab to try it out yourself.
[6] Zero-Shot Text-Guided Object Generation with Dream Fields.
[7] 3D-Controllable Object Generation.
[8] Multi-Category Object Generation.
[9] Google’s dataset search.
[10] Large shoe dataset.
[11] Ipython Notebook to try it out yourself.

Generative AIFrancesca Tabor