The next step in our rope experiments was to allow the player to grab the rope directly. We first tried to do this using Meta’s hand-tracking system.

The first prototype of a grabbable rope features a dynamic attachment between the hand and the rope. In testing, you can see there’s a noticeable gap between your hand and the rope, and no way of releasing the dynamic attachment. (↓)

The most difficult part was finding the ideal place to create a dynamic attachment to the hand and the player through the array of rope particles. 

The second prototype looked much better, as the rope’s physics were behaving more realistically. However, grabbing only worked as a pinch gesture, and the movement of the rope needed to follow the hand better. The distance between the ends is also still an issue. (↓)

Finally, we succeeded in coding a realistic double-handed rope-grabbing motion that featured full fist grips. We repurposed the ObiContactGrabber asset to work with hand tracking to enable a list of contact points on the rope, which then obtains the closest one to create a dynamic attachment there. This inherently fixed the gaping issue between the rope and the player’s hands. (↓)

Additionally, here’s what the rope-grabbing motion looks like while operating the curtain pulley system. (↓)

We then wanted to experiment quickly with hand tracking and pose recognition to slice rope through a scissor gesture. Using the same framework from the interactive sword, when the space between the fingers is recognized as a gesture, it creates a plane with the two fingers to slice the rope. (↓)

Though hand gesture recognition makes creating gestures surprisingly easy, the scissor gesture itself is not always reliable, and the hand can phase through the rope. That being said, gesture recognition is something we will explore more of in the future.