6 Uses for the Wii U Controller’s Camera

The Wii U controller’s front-facing camera is a curious feature. The inclusion of a camera in a game controller is quite novel, but how practical can it be in actual games? Several weeks ago, long before we even knew what the Wii U was called, a few users at the forums came up with several great uses for it. All credit for these ideas goes to them, but I decided to write them up for the blog in the form of an editorial.

Video chat

The most obvious use of the camera is video chat. The controller could be used as a video calling device thanks to its built-in screen, and Nintendo’s reveal trailer for the console showed a few people enjoying an experience that could be likened to Skype or Apple’s FaceTime. However, video chat within games could be a possibility, too. You and your friend could see each other’s faces in a window on the touchscreen while playing games. As you look down at the screen to equip a new item, you could catch a glimpse of your chat partner’s face.

WeeYou suggests that multiplayer games could have a video chat lobby between rounds in which everyone would be able to participate in a group video conference. It’s a cool idea, but one that probably won’t come to light given Nintendo’s track record with their online offerings. As CUDpwns pointed out, “there’s also the issue of people showing things that aren’t their faces, if you know what I mean; that’s something Nintendo will definitely be concerned about”.

Creating Miis, using your face as a character

An idea that has already been realized on the 3DS, but would be nifty to see on the Wii U as well is to be able to create Miis and paste your face onto in-game characters using the camera. Epic Kirby hopes that the camera will be of a high enough quality to be able to create an entire Mii based on a single picture, without the need to input the colour of your eyes, skin, and hair first. It’s not exactly a feature that will set the world on fire, but it can make for fun times if implemented well – Face Raiders comes to mind.

Multiplayer Face Raiders

Speaking of Face Raiders, Epic Kirby also suggested an expanded multiplayer version of it designed for the living room space. Of course, this idea relies on the ability to use multiple tablet controllers at a time; but assuming that’s possible, every player’s face could be thrown into a virtual world displayed on the TV, and they’d try to shoot each other without hitting themselves.

The controller’s motion sensors could also come into play to let the game world expand beyond the TV. The Wii U tech demo Shield Pose has already shown that the sensors are accurate enough for you to be able to look around the world as if the controller was a window into it. The omission of a rear-facing camera means that this world won’t be your living room, but it’s still an entertaining idea to expand the hilarity of Face Raiders to a multiplayer environment.

Analyzing facial expressions

BioWare’s Mass Effect games have gotten lots of publicity over their emphasis on making choices. When Commander Shepard is asked a question by an NPC, the player can choose to reply with a specific emotional reaction. This could be taken a step further by having the controller’s camera read your actual expression right off of your face, and making in-game characters respond accordingly.

Another interesting way to make use of faces is for games to remember them, and react to seeing familiar ones (such as yours). Nintendogs + Cats already does this, in that your pets remember what you look like and react differently to seeing you and someone else trying to play with them. I’m envisioning that such a system could also potentially be used to automagically log you into an account-based online system the moment you pick up the controller (setting aside for a moment the fact that we have no idea how the Wii U’s online system will work).

Head tracking

Head tracking is an interesting technique in which your head’s movements are tracked and the game’s visuals adjusted accordingly to create a very convincing illusion of depth. If that still confuses you, check out this video demonstration for one implementation of the technique. It’s not quite true 3D, but it’s the closest you can get without having a 3D TV.

I don’t think there’s a chance of this one actually happening, because Iwata has said that they’re not going to be focusing on 3D experiences, and I also doubt that head tracking would work well, if at all, if both your head and the controller are moving. Still, it doesn’t hurt to dream a little, especially at this stage of the Wii U’s development. Imagine if you could shift the on-TV viewpoint in a racing game by simply turning your head to the side.

Bringing the real world into the game

Ruthie, my assistant writer, pitched the idea of using the camera as a method of interaction between real life and the game world. One could provide items needed in the game by holding them up to the camera. Uncharted on the PlayStation Vita has been shown to do this, in that you can hold the Vita up to a real-world light source to bring light into the game. Going back further, Microsoft showed off the stunning Milo tech demo a couple of E3’s ago, in which a demonstrator drew a picture of a fish on a piece of paper and held it up to the Kinect camera (then known as Project Natal) to give it to Milo. Milo then “grabbed” the drawing from the top of the screen and commented on it.

I see the most potential in this idea, and feel that the controller’s (relative) portability would really help in realizing it. You could easily take the controller around the house to scan in different objects, and the built-in screen means you won’t lose touch with the game while you’re away from the TV. There are no limits to what developers could achieve with this functionality, and I would love to see some of the talented minds in the industry take a go at it.

So, these are several concepts our community came up with for the Wii U. Some of them are more likely to happen than others, but they’re all great ideas. Do you have any other potential uses for the camera you’d like to add to the list?