Immersive headsets continue to lift off company drawing boards, in the hopes of powering a new wave of virtual reality in the consumer space — whether it’s the Facebook-owned Oculus Rift, Sony’s Project Morpheus, Microsoft’s Hololens or HTC-Valve’s Vive, to name a few VR devices in the works (some others: FOVE, Razer… the list goes on).
Big bets are undoubtedly being made that the next wave of gaming will involve wearing something on your face. It looks very much like a strategy to reboot the otherwise rather moribund console market. Immersive peripherals offer the prospect of differentiating pro gaming from all that casual gaming being conducted on ubiquitous mobile devices — much like 3D movies offer a way for cinemas to compete with the stay-at-home pull of high end flatscreen TVs.
Whether gamers will find VR gaming to their liking — or just hideously nausea inducing — remains to be seen. The first VR games are just starting to be detailed. Design challenges abound. But none of that is putting off Dutch startup Manus Machina, which has been building a data glove input mechanism for use with virtual reality headsets since July last year so VR gamers will be able to ditch their joysticks.
He grabs the banana and it feels like he’s holding a gun.
It’s aiming to build a pair of flexible, finger-sensing gloves that are designed to be worn in conjunction with a VR headset, allowing gamers to view and manipulate their hands and fingers within virtual environments. And while Manus Machina is not the first company to try its hand (pun intended) at a data glove as an input mechanism, co-founder Stephen van den Brink says other examples have been focused on niche use-cases — such as scientific research — whereas the team is aiming is to ship a mainstream consumer product with a price-tag to match.
“It’s basically a glove that accurately tracks what your hand is doing. It’s using flex sensors on each fingers, so it senses the bending of the fingers. And it uses an IMU module on the back of the hand which tracks the rotation, the yaw, the downward rotation of the hand. It has nine degrees of freedom,” he says, explaining the tech. The first prototype was hacked together with a large Arduino board. Now it’s a custom made PCB. The gloves won’t track every type of finger twitch but are designed to track a wide range of movements, from bending single digits, to making a fist and so on.
“Data gloves already exist… a lot of universities already made a glove… and there are a lot of mock-up data glove suits already around. But they cost $10k up for each glove,” he adds.
Manus was showing off its current working prototype at the E3 games conference earlier this month (as seen in the below video). One technical challenge it’s working on is getting the electronics to play nice with the stretchable fabric it’s using for the glove. van den Brink says it’s hoping to have a dev kit version of the glove ready in the next “two to three months”, which will cost $280. The consumer version won’t be ready until next year, either Q1 or Q2, he says — with the retail price-tag for that pegged at around $200. It’s building at cost initially, aiming to make a margin when/if the market for the product scales.
As well as the gloves tracking the wearer’s fingers and the orientation of their hands so gamers can use their hands as controllers, the wearable will also provide feedback via a build in tactile engine with the aim of increasing the sense of VR immersion. A battery pack will also be built into the gloves so there’s no need for them to be wired in.
Positioning of the hands in space is done in conjunction with an external camera. This could be the same camera that’s used for VR headset tracking, according to van den Brink. He says the team is aiming to support lots of different hardware set-up scenarios (in the below demo the team is using the rear camera on a Samsung smartphone, for instance). And of course wants to be device agnostic, in terms of the VR headsets the gloves can work with.
Manus went through the three month High Tech XL startupbootcamp hardware accelerator program, from November last year. And went on to raise €700,000 from two local Dutch investors affiliated with the program, post-demo day — using some of that financing to get to E3 to start trying to drum up developer interest. Getting devs to make games that make use of granular finger tracking or even just use hand gestures as a controller mechanism is of course the really big challenge here. The gloves will need tailor-made games to attract buyers. But VR games as a category are just getting started so there’s a whole lot of building to be done here. Inevitably then, Manus is groping around in the dark at this nascent stage. And sometimes groping a banana…
“This E3 Oculus showed its first games that will be released with VR, with the big titles. Like Eve, for example. But it’s all still played with an Xbox controller. And at this moment — this is the year, or the next year is going to be figuring out how do game developers want to have hand presence in games, or want to have kind of a gesture-based system, because you don’t actually have to have hand presses in a game you can also do gesture-based,” says van den Brink. “That’s something we have to figure out with game developers — how you want to implement it.
“We already have a showcase — a game we made — where you can grab a kitchen rolling pin and use it as a steering device for playing a racing game. You can hold your kitchen rolling pin at the two handles and then just rotate that in space and you can race a bike with it, for example… You can go to the kitchen, grab something, a tool from it, and if it resembles a little bit something you’re using in the game, like a gun… in one of our videos he’s playing Portal with a banana. He grabs the banana and it feels like he’s holding a gun.”
Another scenario that sensing gloves suggest — being able to pick up virtual objects in a game using real-world hand movements — will also be possible using a skeletal hand model in the Manus SDK, so developers could enable collision detection and build games where virtual objects can be manipulated with a gloved hand. (And presumably some tactile feedback could also be delivered to the gloves to create a sense of physicality associated with a virtual object.)
But they’re not there yet. Again, developer buy in will be required to create compelling VR gaming experiences where gamers can manipulate virtual objects with their gloved hands, and maybe even feel like they’re doing so (or doing something). In the meanwhile Manus has created an interface that will let gamers map keystroke controls of existing games to hand gestures as an alternative control mechanism that turns the gloves into a custom controller. “It gives a new way of playing that game. It also feels really different. It feels a little bit more immersive. But of course the best way to use this is if you see your hands in the game and to get it specially made for it,” he adds.
Over to the games developers then.