Sony Patents A Motion Control System That Uses Ordinary Objects As Controllers

By Spencer . July 2, 2009 . 3:24pm

ps31

We saw Sony unveil a LED wand during their E3 press conference. The device in conjunction with an EyeToy camera brought motion control to the PlayStation 3. In addition to that Sony developed another motion control system that uses everyday objects.

 

A recently published patent from Sony details a system where a camera can dynamically map an object — any real world object — for use in a video game. The illustration has a U shape block, but the patent outlines other example objects “include items such as coffee mugs, drinking glasses, books, bottles, etc.” While these are given as examples the object mapping system is not limited to those objects, it can identify any three dimensional object.

 

3dobject flowchart

Before using an object like the U shaped block the camera captures has to capture it. Figure 2 explains the system where players show the object, rotate it, and save a profile for it in a file. The system then analyzes movements and translates them into in game actions.

 

Here are some examples of how Sony could use this technology. The “virtual world light sword” in figure 3B is particularly interesting. When a player holds the U-shaped block up the sword is “on” and the blade will be extended in the game. When the U-shaped block is upside down the sword is “off”.

ps32 ps34 ps35 ps36 ps37

 

The system looks like Microsoft’s Project Natal, but instead of driving with an imaginary steering wheel players can use an everyday item like a plate.

 

Good? Bad? Is Sony’s LED ring controller better?


Read more stories about & & & on Siliconera.

  • MisterNiwa

    Well, i like the idea of using objects as remote controlers.

    Imagine you play a Swordplay game and you get a special plastic sword with it.

    Would be pretty cool. :)

  • http://www.nakedsushi.net/ Louise

    Finally! I now have a use for the giant plastic letter U I have lying round my living room.

  • Peanut-

    Alrdy better than natal.
    I just dont like the look of natal no object in hand, no buttens.
    I still prefere sonys Wand tho. But this is still better than Natal.

    Wand > This > Natal.

    In essence all 3 Should be brilliant if done correctly.

  • TurkeyPotPie

    Interesting tech. But what incentive does Sony have to allow you to use any object when they could sell you a plastic shell for a baseball bat, fishing pole, tennis racket, etc. (see all the Wii add-ons available).

    Kind of crazy to think that Sony has had all this tech percolating in the labs for years (going back to EyeToy), and yet they never brought it to market successfully in the way Nintendo did with the Wii (for the record, I find the EyeToy and the various other motion control things Sony has done and Project Natal a lot more interesting than the Wii remote). The Wii’s success could have been Sony’s if they had thought about the market differently.

    • tzaylor

      I agree the wii’s success could have been Sony’s. Another example of being ahead of their time. They had motion control with the eye toy but the market wasn’t ready. Now they have blu-ray too early. I mean, PS3 over Xbox is a no brainer if you like Blu Ray movies, but not many people have them yet.

  • [email protected]

    There were some sony demos of mapping of real world objectsfor a game. It was a simple tank game but they can use the eye to scan in objects to become their tanks. Eyepet can also do User drawings and become 3d objects that the pet can play with.

    Sony needs to promote the tech they have, cause they have had basic motion control since ps2 with the eyetoy, but now everyone atributes motion control to the wii. Now Natal is taking ideas from the eyetoy and branding it as a wholly original idea by Microsoft.

    • tzaylor

      and that is so frustrating! The Japanese company needs to take an American risk and actually do something with their amazing technologies instead of letting everyone else rip them off!

      • tzaylor

        they need to act instead of react to everyone else

  • UFO

    Why Sony, why you cancel this!!!!!???

    First thing I was gonna do is scan a dildo.

    • http://www.siliconera.com Spencer

      I don’t think this has been canceled. The patent was filed in December and published just last week so there’s a chance Sony is toying with it now.

  • planb

    I don’t like the idea of using any object as an imaginery controller: we already saw, what some wii controllers could do to your plasma or LCD. Now imagine if some ppl are using a real baseball bat when playing a game and that thing slips out of their hand while swinging it…
    No company can guarantee safety with such a device.

  • http://www.facebook.com/profile.php?id=689940976 facebook-689940976

    its going to be used in conjuction with the wands its going to be an update, and i can see the comparisons to natal but i think it is more advanced and its definatly going to be easier to use, ie stearing wheel, guns stuff you can actualy hold will help with emersion in the game instead of just holding your hands out.

  • Natal Rules

    Ok for one thing Natal you can do the same thing. Remember the skateboard, the kid was able to scan it? Plus there is NO saying that you have to use nothing in your hand, to play swords or a racing game. If Natal can see your movement with nothing in your hand you can surely use a sword or a plate if you like as well. Natal is better because it uses sensors to scan the body so it can see the movement of 35 joints in the body, plus it has voice commands. The sony one ist just a camera and no voice commands..woo hoo

    • cutter.T

      it does have Voice control, and if sony patented it microsoft can’t use it.

    • MisterNiwa

      Lol, Xbox fanboy…

      Natal doesnt want you to have something in your hands, it wants you to be bare handed!!

      Too baaaaad that you cant play bare handed Tennis or Baseball or fight with a sword…

      Oh well Natal is so great it wants you to imagine that you have something in your hand you dont actually have!

      Sony and Nintendo want you to still have something to grip on, because thats the way it needs to be. You need to wield something in your hand to imagine to wield something in a game.

      There is no way this controller-less-eyetoy-fake-which-is-so-better-because-its-from-microsoft is going to beat Sony/Nintendo that way.

      Natal is going to fail, because they take the Gamers most important device, the controller.

      Its like taking a christ his bible.

      • Sychotic

        like taking a christ his bible? OMG u need to come up with something better than that.

    • Mark

      LOL, you lap up everything Microsoft tell you like a lap dog?

      Next thing you will be telling us, is that RROD is not 40% because Microsoft said it’s not….

    • tzaylor

      eye toy already has voice recog. Microsoft just ripped it off and marketed it better

  • http://www.facebook.com/people/Mark-Carney/600744266 Mark Carney

    Microsoft is behind the 8Ball on this technology. Sony has been on this since PSEye on the PS2. There are games in the PSEye library that already do what M$ has in mind for the X-box 360. If Sony gets this patent then M$ may have to change some things to not infringe on this patent… Or it could make Natal obsolete since it would actually infringe on this patent… I can see a patent war in the future…

    • Natal Rules

      Again the eyetoy is just a camera and the software senses your basic movment…you can do that with the right software and your camera on your computer http://www.camspace.com/.

      Natal uses the camera mostly to scan and the sensors track the motion of the body. 2 different technologies and Natal is more sophisticated.

      Now since the article mentions the Sony wand I’ll also inform you of this. Sony and Nintendo stole each others idea and heres how.

      1) first Nintendo came out with the Wiimote which has a camera thats in the Wiimote and as you move it tracks the bar thats below (or on top) of your TV. So thats why you can take candles and put them by your TV and the Wiimote will still work.

      2) Heres where it gets interesting. The Plus unit is just a movment sensor (like the 6 axis on the Sony controller).

      3) Now Sony and their Wand has you put the camera on (or below) your TV and it tracks the ball thats on the controller. So I’m guessing if you take a candle you might be able to get some movement without using the wand.

      So while Sony claims that their technology is better than Nintendos because it does 1 to 1 movement, the Nintendo technology with the Plus added does the same 1 to 1 movement as the Sony. The one thing the Sony has that the Nintendo doesn’t is since the camera is on the TV it can scan objects which the Nintendo can’t.

      • st

        you know what there commercial for the Natal is stupid if there is a kid with skateboard then why dosnt he go out side and skate. i mean come on whats next an invisible tv and console lol i can see it now m$ e-xbox invisible drr the e means imaginary lol $100 additions needed $1000 worth

  • cutter.T

    They’ve already made a game with this for the ps2, EyeToy Play: Hero, just search on youtube, if hey had better advertising schemes everyone would already know.

    • http://www.siliconera.com Spencer

      Yeah, we know about EyeToy Play: Hero, but this is different since it can use any object, not just the sword bundled with the game.

  • http://n4g.com/ oldSCHOOLgamer

    why patent something microsoft already can do with natal…lol
    i mean natal is already proven tech with dev kits already out and sony just has…. nothing but their fans to hype up.
    sounds like trouble for sony and does that mean they’re gonna scrap the dildo?…i hope so since they never did admit to copying or borrowing from nintendo at all and no it doesn’t sound better than natal since natal is all in one and it already works. so if microsoft wanted to say they can do that they can ’cause they already have…remember the guy with the skateboard on the tech demo anyone…i do. to me sony just wants to get sued with claims like that.

    • Natal Rules

      I agree, Natal already has devs working on games for it. I don’t think Nintendo would sue Sony because they both stole each others tech and Nintendo would be afraid that Sony would make a counter suit. Sounds like a standoff to me. Microsoft couldn’t sue Sony because Microsoft’s technology is different in the fact it doesn’t just use a camera but sensors. Sony can’t sue Microsoft because Natal uses the sensors to track the movement and not the camera.

      If Microsoft comes out with Natal before or around the same time as Sony’s controller its game over for Sony. The only way Sony has a chance is to bring out their controller a year or more ahead of Natal…but as soon as Natal comes out (and if it lives up to even half of the hype), Sony’s marketshare will plummet because Natal is WAY ahead of the Sony technology.

  • kurtis

    so for driving games including sonys new motion thingy, we could use a wii wheel possibly while 360 owners look like idiots “imagining” a wheel is in their hands

    • Natal Rules

      I’ll say this one more time. With Natal you can hold anything in your hand, it won’t matter. You don’t have to pretend to hold a steering wheel, you can rip out the steering wheel from your car and use it. Natal will still see your hands and be able to track them. Thats the good thing about Natal, you can pick up a sword and use it in a sword fighting game. It tracks your hands (or body parts).

      • Skeptic

        Is Natal 1:1 movement for left,right,up,down,forward,backward?

        How much does Natal depend on proper lighting conditions to work?

        • http://n4g.com/ oldSCHOOLgamer

          i didn’t know you needed proper lighting for motion sensors to track you or track motion…if it was like that the wii would be in trouble and then you gotta ask yourself why would sony step into this market knowing so. that should be the least of your worries since microsoft has dev kits (as i said before) already for game development and sony just has word of mouth so i’d be more skeptical towards sonys claims than anybody elseses.

          • Still Skeptic

            If you where in a pitch black room, would natal still track you perfectly?

            If your only light is your monitor’s warm glow would natal still work right?

            I’ve used the eyetoy on PS2, lighting and backdrop made all the difference.

          • http://n4g.com/ oldSCHOOLgamer

            those are good questions but i look at it this way..

            -If you where in a pitch black room, would natal still track you perfectly?
            yup you don’t need lighting for motion sensors to track you but you do need light for a camera to track you like the eye toy which i have for my ps3 and still have for my ps2 which is not that responsive with games that i played so i think sony might just shoot themselves on the foot by patenting something without a motion tracker since the camera those some of the work.

            -If your only light is your monitor’s warm glow would natal still work right?
            for things like scanning maybe not, the motion sensors and voice recognition take care of the rest.

            and yes the eye toy is more sensitive to lighting and backdrops that’s why its just a camera and i also have the camera for my xbox 360 which was compatible with some games and it also does some similar stuff like the eye toy with limited motion sensing (Totemball which requires Vision Camera) since there’s nothing to track forward movement, hand movements and such.

          • Natal Rules

            I would have to agree with oldSCHOOLgamer. Think of it this way. Motion sensors are used in houses or businesses and they don’t keep their lights on at night. For the most part motion sensors sense inferred (heat) that the body emits so you can be in a totally black room and it’ll see you. I’m guessing though the camera may enhance the sensors, but since they’re using sensors, they could use a camera that sees inferred as well so the cam could see you in the dark also.

            I’m not what technology MS is using but to answer your question yes the sensors will see you.

    • http://n4g.com/ oldSCHOOLgamer

      i know people who do air guitar at rock concerts, shadow box at the gym and do fade aways pretending to be kobe bryant to fool around are they idiots too?… ’cause i see a market here specially for games like guitar hero or rock band, sports games and yes you can use peripherals with natal it’s obvious some people can’t register a lot of the stuff from E3 and the microsofts tech demos.

      so why would sony want to patent the use of “ordinary objects” things when they already showed a wii like controller that’s supposed to do all the controlled movements through button layouts and capturing via camera or eye toy?…unless they’re planning on scraping the wand like controller or they’re planning on coming out with add-ons in the future for it like the wii with the motion plus sold separately. now there’s something to be a little more skeptical to be about. so if mocrosoft wanted to patent something they might go in the direction of patenting the “scan your own gear” into a video game thing something sony can’t do with the eye toy.

    • http://n4g.com/ oldSCHOOLgamer

      i know people who do air guitar at rock concerts, shadow box at the gym and do fade aways pretending to be kobe bryant to fool around are they idiots too?… ’cause i see a market here specially for games like guitar hero or rock band, sports games and yes you can use peripherals with natal it’s obvious some people can’t register a lot of the stuff from E3 and the microsofts tech demos.

      so why would sony want to patent the use of “ordinary objects” things when they already showed a wii like controller that’s supposed to do all the controlled movements through button layouts and capturing via camera or eye toy?…unless they’re planning on scraping the wand like controller or they’re planning on coming out with add-ons in the future for it like the wii with the motion plus sold separately. now there’s something to be a little more skeptical to be about. so if mocrosoft wanted to patent something they might go in the direction of patenting the “scan your own gear” into a video game thing something sony can’t do with the eye toy.

      • http://n4g.com/ oldSCHOOLgamer

        oops sorry double post

  • Mark

    Anyone know when this patent was filed? I heard it was 6 months ago, which predates anything Microsoft have recently annoucned…

    • http://www.siliconera.com Spencer

      The patent was filed in the US and it was in December 2008, around six months ago.

    • http://n4g.com/ oldSCHOOLgamer

      predates?..nope more like negates their waggle controller. it doesn’t phase natal at all since its a camera with voice recognition, facial recognition and motion trackers and doesn’t need added peripherals unless preferred or for customizing purposes. plus natal was in the works way before this patent claim from sony.

      • Urwrstnghtmr

        plus natal was in the works way before this patent claim from sony” Yea, right. Natal was in the works by some company that microsoft “bought”. When will you oldSCHOOLgamers realize that microscam is a software manufacture, not hardware. 33% sure that Natal will have a 33% failure rate just like 33% of you so called oldSCHOOLgamers HExboxes.

        • http://n4g.com/ oldSCHOOLgamer

          @Urwrstnghtmr

          you answered your own question if you had one there…lol
          i know microsoft is a software comp[any and i know microsoft aquired 3DV the company that is in charge of natal mostly see i did my research and i’m a gamer first fanboy never since i own a ps3 as well. so to tell you the truth i’m more excited for natal than any other motion sensing controller that’s it not sony telling me they aquired a patent.

          peace out fanboy next time you do your research before coming out the mouth and telling someone they don’t know about what they own…lol

  • xilex

    anyone have a link to the patent?

  • Teamtendo

    Natal is an eyetoy plus that http://en.wikipedia.org/wiki/Time-of-flight_camera

    Speaking of pattents, anyone remember the dual-shock??

  • http://twitter.com/nathanpprince Nathan Paul Prince

    You`ve finally found a way into homes! Congratulations! Humans are now identified visually inside their own domiciles, their actual public shared quarters are identified in 3 dimension, so they know what kind of lamp you ACTUALLY own (as opposed to falsifying those online surveys you fake your way through) and on top of that, by clicking ” Accept” and not bothering to read the fine print, you become a more focused market research target because by not having a particular product in your living room, ads will be more specific to your email when you log on to the net.
    Ingenious. I thought about this about 12 years ago and still noone believed me… now this is proof. About darn time.
    Now BUY IT! run!!

  • BpN

    How is this better or new compared to Microsoft’s Project Natal? Microsoft showed in their demo the capability to scan objects as well for in game use, so how is this a new feature?

  • Sychotic

    natal 360 is better on so many levels! U don’t need to use any objects. just ur own movements with no other objects are enough. check out natal xbox 360.

  • Sychotic

    why would u want to have something to hold on to when u can actually be hands free and the fact that you can scan in your own objects like a skateboard or even your own tennis racket is pretty cool and Sony has definitely not invented that. We are talking about possibly not having remote controls ever again. Everything can be motion sensored or voice activated. Farkhyea! I reckon Sony is always one step behind.

  • Karsten

    Historically – sony has actually often been on the leading edge – and in many ways created superior products that the market was not ready for. We all know how the VHS vs. Betamax battle ended – but the Betamax was a superior product which could not cope – actually due to the porn-industy selecting the VHS format.

    A couple of years ago, i saw a Sony picture-frame, the first one i ever laid eyes on, but very few were sold – now, a few years later, the competition has hit the market, and now they are being sold everywhere.

    Sony this time played the Blue-Ray card, and against all prior history – they actually won over the HD-dvd – ‘great’ one should think – but for some reason, the market is very slow to adopt the format.

    I once read through some of the Sony company history – and the theme of being on the cutting edge and still not getting the sale was found many places.

    Personally – i am getting sick of hearing ‘fanboys’ always claiming the product they love is the best – Neither Natal nor the Sony cam controller has been released yet – only a somewhat simple demo of natal under their own control has been given. In stead of discussing which is better (lets face it – none of you actually know anything about any of the finished products other than what Microsoft and Sony want you to know) – lets all await the future, and hope they can live up to the hype!

    Personally i hope both Sony and Microsoft stay in the market, and will continously keep compeeding – because that will keep prices down and will make sure the companies are always pushing the envelope, and thinking outside the box.

    Well – that was my thought.. flame away if you prefer..

    PS: I own the xbox (first one) and PS3 (and psp) and will at all times buy the console i personally think is best instead of being a fan of any-one console.

    • http://n4g.com/ oldSCHOOLgamer

      i agree for the most part since i love both microsoft (software) and sony (hardware) equally but only microsoft has shipped out dev kits to all main game developers out there which means natal is ready and they actually had some of the press at E3 try it and even the movie stars came in skeptical and left wanting to buy it. even though these people only got to play natals tech demo they also got to try burnout paradise and there are vids to prove microsofts tech is ready for games on youtube and late night with jimmy fallon. so to me if it’s that easy to configure burnout paradise to natal at a so called “early stage” than i’m impressed since it’s prolly beyond that and a good move microsoft aquiring hardware companies to expand in different areas.

      mind you i own a ps3 too (PSN ID:OLDschoolGAMER-1) but they’re a little late on software tweaks like home still in beta and firmware updates. at least the ones people are looking forward to and request but end up just being rumors. so as far as sony i’m just more into their games not their claims on how they plan on entering the casual market.

Video game stories from other sites on the web. These links leave Siliconera.

Siliconera Tests
Siliconera Videos

Popular