6 Uses for the Wii U Controller’s Camera

Wii U controller camera

The Wii U controller’s front-facing camera is a curious feature. The inclusion of a camera in a game controller is quite novel, but how practical can it be in actual games? Several weeks ago, long before we even knew what the Wii U was called, a few users at the forums came up with several great uses for it. All credit for these ideas goes to them, but I decided to write them up for the blog in the form of an editorial.

Video chat

The most obvious use of the camera is video chat. The controller could be used as a video calling device thanks to its built-in screen, and Nintendo’s reveal trailer for the console showed a few people enjoying an experience that could be likened to Skype or Apple’s FaceTime. However, video chat within games could be a possibility, too. You and your friend could see each other’s faces in a window on the touchscreen while playing games. As you look down at the screen to equip a new item, you could catch a glimpse of your chat partner’s face.

Wii U Video Call

WeeYou suggests that multiplayer games could have a video chat lobby between rounds in which everyone would be able to participate in a group video conference. It’s a cool idea, but one that probably won’t come to light given Nintendo’s track record with their online offerings. As CUDpwns pointed out, “there’s also the issue of people showing things that aren’t their faces, if you know what I mean; that’s something Nintendo will definitely be concerned about”.

Creating Miis, using your face as a character

Nintendo 3DS Mii Maker photo

An idea that has already been realized on the 3DS, but would be nifty to see on the Wii U as well is to be able to create Miis and paste your face onto in-game characters using the camera. Epic Kirby hopes that the camera will be of a high enough quality to be able to create an entire Mii based on a single picture, without the need to input the colour of your eyes, skin, and hair first. It’s not exactly a feature that will set the world on fire, but it can make for fun times if implemented well – Face Raiders comes to mind.

Multiplayer Face Raiders

Speaking of Face Raiders, Epic Kirby also suggested an expanded multiplayer version of it designed for the living room space. Of course, this idea relies on the ability to use multiple tablet controllers at a time; but assuming that’s possible, every player’s face could be thrown into a virtual world displayed on the TV, and they’d try to shoot each other without hitting themselves.

Face Raiders

The controller’s motion sensors could also come into play to let the game world expand beyond the TV. The Wii U tech demo Shield Pose has already shown that the sensors are accurate enough for you to be able to look around the world as if the controller was a window into it. The omission of a rear-facing camera means that this world won’t be your living room, but it’s still an entertaining idea to expand the hilarity of Face Raiders to a multiplayer environment.

Analyzing facial expressions

BioWare’s Mass Effect games have gotten lots of publicity over their emphasis on making choices. When Commander Shepard is asked a question by an NPC, the player can choose to reply with a specific emotional reaction. This could be taken a step further by having the controller’s camera read your actual expression right off of your face, and making in-game characters respond accordingly.

Mass Effect

Another interesting way to make use of faces is for games to remember them, and react to seeing familiar ones (such as yours). Nintendogs + Cats already does this, in that your pets remember what you look like and react differently to seeing you and someone else trying to play with them. I’m envisioning that such a system could also potentially be used to automagically log you into an account-based online system the moment you pick up the controller (setting aside for a moment the fact that we have no idea how the Wii U’s online system will work).

Head tracking

Head tracking is an interesting technique in which your head’s movements are tracked and the game’s visuals adjusted accordingly to create a very convincing illusion of depth. If that still confuses you, check out this video demonstration for one implementation of the technique. It’s not quite true 3D, but it’s the closest you can get without having a 3D TV.

I don’t think there’s a chance of this one actually happening, because Iwata has said that they’re not going to be focusing on 3D experiences, and I also doubt that head tracking would work well, if at all, if both your head and the controller are moving. Still, it doesn’t hurt to dream a little, especially at this stage of the Wii U’s development. Imagine if you could shift the on-TV viewpoint in a racing game by simply turning your head to the side.

Bringing the real world into the game

Ruthie, my assistant writer, pitched the idea of using the camera as a method of interaction between real life and the game world. One could provide items needed in the game by holding them up to the camera. Uncharted on the PlayStation Vita has been shown to do this, in that you can hold the Vita up to a real-world light source to bring light into the game. Going back further, Microsoft showed off the stunning Milo tech demo a couple of E3’s ago, in which a demonstrator drew a picture of a fish on a piece of paper and held it up to the Kinect camera (then known as Project Natal) to give it to Milo. Milo then “grabbed” the drawing from the top of the screen and commented on it.

Milo

I see the most potential in this idea, and feel that the controller’s (relative) portability would really help in realizing it. You could easily take the controller around the house to scan in different objects, and the built-in screen means you won’t lose touch with the game while you’re away from the TV. There are no limits to what developers could achieve with this functionality, and I would love to see some of the talented minds in the industry take a go at it.

So, these are several concepts our community came up with for the Wii U. Some of them are more likely to happen than others, but they’re all great ideas. Do you have any other potential uses for the camera you’d like to add to the list?

We can deliver all the latest Wii U news straight to your inbox every morning. Want in?

Comments, Reactions, and General Hooliganism

  1. The final idea is pretty awesome; I’ve been wanting something like that to come out for the Kinect before I buy it (for the PC) and so far only a few, small ideas have come up. I hope the WiiU is more successful.

  2. nintendo3DS

    I like the concept about “Bringing the world into the game,” I could see that functionality benefit from quite a lot of games. The “Analyzing of facial expressions” would be pretty neat to see on the Wii U too.

  3. Wiiwouldlike2playwithU

    This is something that I woud enjoy the camera being used for. I have no idea if you though of this, but using the camera in conjuction with the mic for a game like civ 5

  4. Mariomaster1

    Though this may be rather unrelated to this article, has anyone else thought about whether there will be Miis on the Wii U…..or was it quite obvious? I just began to think about this when I was playing a Wii game that allowed the Miis to be used…..since Wii games will be playable on Wii U.

    • bob

      Of course, they’re on the 3DS too. Besides, New Super Mario Bros Mii needs Mii’s.

  5. James

    It the wrong tech for head tracking. Head tracking uses the infrared lights and infrared cameras in the Wii remote and sensor bar. It doesn’t use a regular camera.

    It also requires the infrared lights to be mounted on your head.

  6. boywinkle

    head tracking does not need infrared – head tracking and tiltong exists on DSi that 3D hiden game on DSiware..

    the camera could track head moves

  7. Jmaster720

    isn’t all of this stuff already on kinect 😛

    • Rafael

      i didn’y know that kinect had mis and face raiders. if it does please show me footage

  8. jj

    well at e3 they showed a starfox demo usin wiiu pad and the players faces were shown above each players craft in actual realtime like a webcam, instead of the usuall starfox characters which would have been simple images just. cant see miis being a part of this system as they are going all out with open network and allot of 3rd party support a total U turn from the original wii

  9. Good news and bad news

    I like the head-tracking for 3D, I think it could really bring a new dimension (sorry) to gaming. The second screen has already been shown to allow 360% viewing outside of the T.V. screen view, so extending that to show a 3D view of the world too would be cool.

    I’m not sure how the interaction with the real world would work or why, perhaps I’m not very imaginative, but I can’t see what it would bring? Also, augmented reality use would seem to be limited by a front-facing camera? No doubt Nintendo will be showing some amazing ideas in the near future, let’s hope third party devs take run with them.

  10. Heather Metzler

    Is it possible to remotely view the room, via the controller? Ty