Skip to main content

Pancoholic

Have you ever wanted to flip a pancake but worried you’d make a mess? In this VR pancake simulator, you cook virtual pancakes while synchronizing with a real-world chef who manages the actual ingredients. Will you be the cook or the ingredient manager? You decide!

Pancoholic Hero Image

Goal and motivation

We have created Pancoholic with fun in mind. Our goal was to create a fun experience in VR to cook pancakes, with the innovative aspect of using a real pan as the main interaction device. However, fun is better when shared.

We were motivated to create a multiplayer experience with two cookers: one in charge of cooking the pancakes in VR, and another in charge of providing the ingredients in real life. In this way, we create a fun experience that traverses worlds.

Pancoholic Hero Image

How to play

Two cookers, one in real life and one in VR.

The cooker in VR takes the pan prop and puts on the VR glasses. Their goal is to cook the pancake orders that arrive in the kitchen.

The cooker in real life manages the 3D-printed ingredients, which are placed on top of a portal and teleported to the cooker in VR.

Pancoholic Hero Image

Obstacles

The main obstacle found in the project was getting the physics working correctly in the HTC Vive headset. At the beginning, we thought that the same physics we had in Unity would be displayed in the Vive headset, but that was not the case.

To overcome this obstacle, we found the solution of linking the VIVE Trackers to a joint in Unity, in this way forcing the update of the physics in the headset.

Challenges

The main challenge the project faced was, without doubt, achieving a proper pancake behaviour. Pancakes are soft body objects and cannot be represented as default rigid bodies in Unity. Several solutions were tested such as Third-party plugins or Bone-like behaviour with an armature set in Blender.

In the end, it was decided to create a PhysX framework running as a Unity Native plug-in. A C++ interface was export to Unity scripts, granting the C# layer the ability to manipulate the PhysX world.

Lessons Learned

The project was developed as a fully completed experience, with different types of orders, delivery confirmation, pancake cooking, and ingredient management. At the end of the day, what our users enjoyed the most was playing with the pancake physics in VR, which filled us with joy, since it was our goal from the very beginning of the project.

Related Works

Technologies

How is it possible to merge fun from the virtual and real world at the same time? In Pancoholic, this is possible thanks to the combination of VIVE Trackers, NFC detectors, and 3D printing.

VR

We used the HTC VIVE VR headset as the main interaction device.

VIVE Trackers

A real pan is tracked in VR with a VIVE Tracker. This gives the cooker the feeling of using a real pan (the pan is 3D printed for safety purposes!).

Soft body physics simulation

The soft behaviour of the pancakes was achieved with a custom soft body physics plugin for Unity, utilizing NIVIDIA PhysX.

By mirroring pancakes instance between the Unity Mesh objects and PhysX Deformable Body, the real-time physics archived by driving a Kinematic Rigid Body using the tracker's position.

3D printing

The pan and the real-life ingredients were modelled and 3D printed.

NFC detectors

Each ingredient is attached to an NFC tag that is read through a phone. This allows us to teleport ingredients to the VR world.

Orders

Orders are handled through a custom web server, with the UI displayed on a tablet. Anyone from the crowd can create their own orders for the cookers.

3D printed pan 3D printed pan

Team: Companion Cube

Helge Kvarfordt

helgekv@kth.se

Game logic, Cooking book

Emil Hansen

emhanse@kth.se

VR tracking, Sound

Hengyi Lin

hengyil@kth.se

Custom Soft-body physics

Sofia Andersson

sofiaa5@kth.se

Modelling, 3D-printing, NFC detection

Natalia Sempere

sempere@kth.se

Unity development, UI, Web Server