Pancoholic
Have you ever wanted to flip a pancake but worried you’d make a mess? In this VR pancake simulator, you cook virtual pancakes while synchronizing with a real-world chef who manages the actual ingredients. Will you be the cook or the ingredient manager? You decide!
Goal and motivation
We have created Pancoholic with fun in mind. Our goal was to create a fun experience in VR to cook pancakes, with the innovative aspect of using a real pan as the main interaction device. However, fun is better when shared.
We were motivated to create a multiplayer experience with two cookers: one in charge of cooking the pancakes in VR, and another in charge of providing the ingredients in real life. In this way, we create a fun experience that traverses worlds.
How to play
Two cookers, one in real life and one in VR.
The cooker in VR takes the pan prop and puts on the VR glasses. Their goal is to cook the pancake orders that arrive in the kitchen.
The cooker in real life manages the 3D-printed ingredients, which are placed on top of a portal and teleported to the cooker in VR.
Obstacles
The main obstacle found in the project was getting the physics working correctly in the HTC Vive headset. At the beginning, we thought that the same physics we had in Unity would be displayed in the Vive headset, but that was not the case.
To overcome this obstacle, we found the solution of linking the VIVE Trackers to a joint in Unity, in this way forcing the update of the physics in the headset.
Challenges
The main challenge the project faced was, without doubt, achieving a proper pancake behaviour. Pancakes are soft body objects and cannot be represented as default rigid bodies in Unity. Several solutions were tested such as Third-party plugins or Bone-like behaviour with an armature set in Blender.
In the end, it was decided to create a PhysX framework running as a Unity Native plug-in. A C++ interface was export to Unity scripts, granting the C# layer the ability to manipulate the PhysX world.
Lessons Learned
The project was developed as a fully completed experience, with different types of orders, delivery confirmation, pancake cooking, and ingredient management. At the end of the day, what our users enjoyed the most was playing with the pancake physics in VR, which filled us with joy, since it was our goal from the very beginning of the project.
Related Works
- Third-party soft-body asset for Unity: Yue Softbody Physics | Physics | Unity Asset Store
- Video tutorial of making soft body object in Unity using Blender: Making Unity Softbody without plugins or coding with Blender
- NVIDIA PhysX library: Github: NVIDIA-Omniverse/PhysX
- Short Pizzicato Song by DanJFilms: https://freesound.org/s/529008/
Technologies
How is it possible to merge fun from the virtual and real world at the same time? In Pancoholic, this is possible thanks to the combination of VIVE Trackers, NFC detectors, and 3D printing.
VR
We used the HTC VIVE VR headset as the main interaction device.
VIVE Trackers
A real pan is tracked in VR with a VIVE Tracker. This gives the cooker the feeling of using a real pan (the pan is 3D printed for safety purposes!).
Soft body physics simulation
The soft behaviour of the pancakes was achieved with a custom soft body physics plugin for Unity, utilizing NIVIDIA PhysX.
By mirroring pancakes instance between the Unity Mesh objects and PhysX Deformable Body, the real-time physics archived by driving a Kinematic Rigid Body using the tracker's position.
3D printing
The pan and the real-life ingredients were modelled and 3D printed.
NFC detectors
Each ingredient is attached to an NFC tag that is read through a phone. This allows us to teleport ingredients to the VR world.
Orders
Orders are handled through a custom web server, with the UI displayed on a tablet. Anyone from the crowd can create their own orders for the cookers.
Team: Companion Cube
Helge Kvarfordt
helgekv@kth.se
Game logic, Cooking book
Emil Hansen
emhanse@kth.se
VR tracking, Sound
Hengyi Lin
hengyil@kth.se
Custom Soft-body physics
Sofia Andersson
sofiaa5@kth.se
Modelling, 3D-printing, NFC detection
Natalia Sempere
sempere@kth.se
Unity development, UI, Web Server