3D generated library interior generated by ProcTHOR.
An arrow pointing leftHome

ProcTHOR is a generator of AI training spaces created by AI2’s computer vision researchers

  • Mike Pearl
6/22/2022

Embodied AI systems can learn a lot from an “arbitrarily large” number of plausible interior spaces.

AI2THOR, the Allen Institute’s (which, full disclosure, is a supporting partner of PNW.ai) proprietary virtual AI training center, used to comprise 82 apartments where robots made of ones and zeroes could lumber around and knock over dining room tables in a consequence-free digital environment. Well, on June 14, with the announcement of ProcTHOR, AI2THOR finally became an infinite expanse of randomly generated spaces where robo-chaos can reign.

To clarify: Let’s say you were training a real-world robot to take bread out of the fridge and toast it, or navigate a kitchen without breaking anything, in the hopes of one day creating a dexterous and agile electronic kitchen assistant. You would want to perform those tasks not just in a lab, but preferably in a million or so different realistic kitchens. No one would let you rent their Airbnb for such research, but that would sort of be the ideal.

If your “robot” is purely digital — meaning just an embodied artificial intelligence (E-AI) — then The Allen Institute’s Perceptual Reasoning and Interaction Research (PRIOR) team would like you to check out ProcTHOR, which you can use to generate infinite digital Airbnbs for just such a purpose.

Using the ProTHOR framework, researchers can procedurally generate an “unbounded number of diverse, fully-interactive, simulated environments,” according to PRIOR’s paper on ProcTHOR. It also notes that similar generators of spaces for such research do exist, but each comes with downsides — low interactivity, mainly.

The most useful comparable technology would probably be Megaverse, a similar piece of tech released last year that produces infinite interactive spaces that look distinctly like Minecraft-built Amazon warehouses with voids instead of walls. ProTHOR, meanwhile, generates spaces using AI2THOR, meaning they look less like Minecraft, and more like places where an infinite number of Pixar characters might live or work.

For its sample experiments, the PRIOR team used “a server with 8 NVIDIA Quadro RTX 8000 GPUs,” according to their paper — meaning roughly a $40,000 GPU budget — and produced what they argue are “state-of-the-art results.”

Their experiments with pre-trained AIs involved carrying out six AI tasks. These involved digital robots navigating toward objects, and rearranging objects in 10,000 unique interior spaces. Best of all, no Airbnb hosts had to receive any panicked direct messages about their kitchen appliances being smashed by robots.