Page 1 of 3 123 LastLast
Results 1 to 25 of 63

Thread: Binaural sound technique - extreme sound possibility

  1. #1
    Join Date
    Sep 2009
    Posts
    134

    Default Binaural sound technique - extreme sound possibility

    I hate doing this, but it feels like your advices get lost under the weight of 51 page long "your ideas" post. I am adding it here too, I believe it will be merged with it anyway in some time.

    So here it goes:

    No matter what we believe, it can't be denied that sound is very important in a thief game. The perfect use of sound had a huge part on the impact of the first game. Both in-game and in cutscenes. How can we forget the persuasive voice of Constantine? The music? The moaning of a zombie?

    The technology has changed, however. I don't know if it was discussed before, but please consider the possible appliance of binaural sound recording technique.

    For an example, please watch this, using your headset or earphones.This sound technique is way beyond stereo technology, and can be a real surprise the first time you hear it. If you don't know what this is, I guarantee that you will... be thrilled... especially when you think of it in a thief game.

    http://www.youtube.com/watch?v=IUDTlvagjJA

    When we use our real ears to hear sounds, our head and auricles bend and change the sounds. So when something drops to the ground on the left, the sound waves that come to your left ear is different than the sound waves that come to your right ear. This small difference in sound level and wave differences are interpreted by our brain and this gives us an estimation of the distance and place of the sound source.

    Binaural technique is a special recording method. Basically, two high-tech microphones are placed in a sculptured human head, inside the ears - which are shaped exactly like ears. Then it is placed in the middle of the recording room. Every sound has a different wave on each microphone, and a special algorithm records the sound. As a result, when you use your headset to listen this sound, it feels extremely real. You can understand if the voice is coming from above or below your head level, close as 1 meter or far as 4 meters or whispered right into your ear.

    Now consider this effect in thief 4.

    Exciting, right?

    However, it is very very diffıcult to do it in game. Becase the player is free to go anywhere, and the sound sources can change during the game according to the flow of that particular gameplay progression. There is no way of placing the mannequin head in everywhere possible in every combination. The only way to solve this would be to create a program that calculates all the variables in the change of placement of the soundsource, and modify the waves according to the players positioning in real-time game.

    I believe we will have this technology someday, but it will be a long wait.

    However, this technique can still be used in Thief4

    -Possible -in game, game engine cutscenes where player loses control and is watching the game, probably during the beginning or ending of a mission. We all know how strong the story can get in a thief game, and this sound tech can enhance the experience of receiving the story.

    -Introduction, between the mission cutscenes, ending movie : if we watch these from the perspective of Garret, this technique can be used. Imagine a cutscene where Garret is walking in a dark corrdior and suddenly a portcullis is shut behind him with a loud crank.

    Maybe we see the full usage of this technique in thief 5

  2. #2
    Join Date
    May 2009
    Location
    reading comp 101
    Posts
    892

    Default

    Thanks windwalker, really interesting

    And for me, sound is Thief.

    I'm not so fussed about the graphics, as long as they are good enough. (Which obviously is a subjective point!)
    For the first time ever I've got a rig that I can turn everything up and on in Thief but I never noticed the lack when I couldn't.
    Ground breaking graphics get touted as the usp for so many games but only a small proportion of people have powerful enough PC's to see them in their full glory. And it seems as if often, dev time gets wasted on pretty, pretty rather than story and gameplay.

    But I want sound in Thief to push the boundaries as far as possible.
    It creates immersion more than anything else imo.

  3. #3
    Join Date
    Jun 2009
    Location
    The City
    Posts
    1,004

    Default

    WOW

    That was some very impressive use of sounds, when listening to the fragment with my eyes closed I actually felt scared when he put the paper bag over my head while I knew it was all just sound. also the snipping of the scissor was very uncomfortable.

    I fully support this idea and I do want to see this in thief 4, but I can understand if it isn't possible. If they do however it is possible to play the game with your eyes closes because you constantly know the exact position of a guard through sounds.

    A very good find wind walker, one of the best improvements submitted so far.
    Hello EM!
    *akward silence*

  4. #4
    Join Date
    Feb 2004
    Location
    On Sabbatical
    Posts
    1,701

    Default

    but he missed my fringe

  5. #5
    Join Date
    Sep 2009
    Posts
    134

    Default

    Quote Originally Posted by Pieter888 View Post
    WOW
    . If they do however it is possible to play the game with your eyes closes because you constantly know the exact position of a guard through sounds.

    A very good find wind walker, one of the best improvements submitted so far.
    Thank you, and that sentence of you clearly says it all. The place of the sound source is so accurately felt by the senses that you actually do not have to see it to feel it's place. If it was possible to do it in-game, you could really invent a new playstyle like "the blind rogue" or something like that. Also, in total darkness you could still navigate away from your enemies or obstacles.

    However I believe we will have to make due with the cutscenes and demos for now. Yet still, I would like to see the Thief 4 as the first game that uses binaural technique in gaming history. That would also be a boost for Eidos Montreal. I hope they can find a way to implement this into the game.

  6. #6
    Join Date
    Jan 2008
    Location
    The City
    Posts
    7,636

    Default

    It would make headphones superior to 5.1 or 6.1 surround sound for the needlessness of several speakers to create the perception. For those without surround sound, or the inability to tote all the speakers around with a laptop, this would be beautiful. I heard other demos of this years ago, and years before that, a guy had made a fake human head of materials approximating flesh and bone, and with microphones in the ear canals, to study how the ears receive sound that the mind can interpret directionally. It would be groundbreaking if it could be implemented, and if any game should break ground with awesome sound design, it should be Thief again. I don't know if it's possible, yet, in a virtual world, where it isn't about the recorded sound, but their simulated location in relation to the player character.

  7. #7
    Join Date
    Sep 2009
    Posts
    134

    Default

    Dear Jtr,

    I believe it would be possible someday, to mathematically modify a basic sound sample as if it was being directed towards a virtual human head with two microphones inside the ears; however I believe it would take well over a year of dedicated work to achieve this, and it would also be very demanding on the hardware. Imagine an ordinary thief scene; two guards patrolling, one singing, three torches on several locations, lightning sounds from the bad weather, garrets own footsteps... all will have to be modified by each mouse action, ie. where the player turns his face to. Now add walking, leaning, all the moves that a thief player can do... it would take a serious processing power to apply this in real-time game.

    Now I am sure this will one day come true, however it will take at least one full year to achieve this.

    And thinking on it.. we will probably wait more than one year for thief 4, so it is... maybe possible.

    Currently, though, I believe it can be done in game pre-scripted cutscenes, videos between game levels etc. at the moment... And I really, really want to see - actually "hear"- it in a Thief game.

  8. #8
    Join Date
    Jun 2009
    Location
    The City
    Posts
    1,004

    Default

    hehe, this also means there's even less reason to put in third person view because you don't have to look around the corners any more, you can simply hear whether or not a guard is there, and if the guard is walking away or towards you.

    Indeed it's pretty high tech stuff, but I'd love to see thief do it. There are simply no cons to this idea.
    Hello EM!
    *akward silence*

  9. #9
    Join Date
    May 2009
    Location
    The Spiraling Sea
    Posts
    2,639

    Default

    "Binaural Recording" is what you are talking about here and is a recording technique used for "3D positional sound".

    Binaural recording needs to be recorded with a microphone at each ear in the real world. Whatever is recorded in that natural acoustic environment is set in stone on a specific time-line (like a movie). There is no way interact with or change anything once recorded.

    Fear not, for there are ways to simulate an acoustic environment in a virtual world. Creative Technologies EAX has already implemented 3D positional sound using Head Related Transfer Functions (HRTF's) and reverberation. EAX uses complex algorithms to simulate a natural acoustic environment in real-time with as many as 128 different sounds heard simultaneously. It is similar to "ray tracing" when computing light in visual 3D modeling...just with the audible spectrum instead of the visual.

    T1/T2 utilized EAX 2 and TDS used EAX 4. With EAX 5, close to everything that you had heard on that demo video can be achieved, if designed properly.

    Read more about EAX here.

    Thief 4 should have EAX 5, but if they use consoles as the lowest common denomenator, then we may not get it, as consoles do not support EAX...or there might be no time spent on design....
    Last edited by Vae; 11-11-2009 at 06:15 AM.

  10. #10
    Join Date
    Jun 2009
    Location
    The City
    Posts
    1,004

    Default

    Thanks Vae fot the information.

    So if we have somewhat similar technique existing already It will be an interesting option for EM to use EAX 5.
    Hello EM!
    *akward silence*

  11. #11
    Join Date
    May 2009
    Location
    Magyarország
    Posts
    542

    Default

    Oh. My. God.

    That was... indescribeable.
    I've never heard of this before, but I WANT Thief IV done that way.

  12. #12
    Join Date
    Jul 2008
    Posts
    446

    Default

    Still there are no monaural cues to locate the sound precisely in the vertical plane (elevation-back n forth). It is difficult to reproduce accurate sound elevation localization trough microphones because the pinna (and later the brain) of the ear is responsible for that and it is covered with the head speakers.
    It is a nice attempt though.

  13. #13
    Join Date
    Mar 2007
    Location
    Here and There
    Posts
    557

    Default This is really fascinating...

    Quote Originally Posted by Vae View Post
    "Binaural Recording" is what you are talking about here and is a recording technique used for "3D positional sound".

    Binaural recording needs to be recorded with a microphone at each ear in the real world. Whatever is recorded in that natural acoustic environment is set in stone on a specific time-line (like a movie). There is no way interact with or change anything once recorded.

    Fear not, for there are ways to simulate an acoustic environment in a virtual world. Creative Technologies EAX has already implemented 3D positional sound using Head Related Transfer Functions (HRTF's) and reverberation. EAX uses complex algorithms to simulate a natural acoustic environment in real-time with as many as 128 different sounds heard simultaneously. It is similar to "ray tracing" when computing light in visual 3D modeling...just with the audible spectrum instead of the visual.

    T1/T2 utilized EAX 2 and TDS used EAX 4. With EAX 5, close to everything that you had heard on that demo video can be achieved, if designed properly.

    Read more about EAX here.

    Thief 4 should have EAX 5, but if they use consoles as the lowest common denomenator, then we may not get it, as consoles do not support EAX...or there might be no time spent on design....
    Let's get rid of the consoles then! No, no I'm joking...hm...to a certain extent, but what matters here is the huge amount of importance recognised to sound in Thief as it can be truly used as a tool to make the game more engaging and certainly to trick the player into false expectations, which, in the end, generates more tension - uncertainty - difficulty.

    I'm gonna read the content of your link, now. Thanks Vae!
    "The Essence of Balance is Detachment"
    Glyph07

  14. #14
    Join Date
    May 2009
    Location
    Magyarország
    Posts
    542

    Default

    The problem is, as far as I can see, this only works with pre-recorded sounds. But should an AI (as in Thief) move about in a non-pre-recorded way, how would that... work?

  15. #15
    Join Date
    Jun 2009
    Posts
    118

    Default

    Hell yeah this is a good idea. Way better for immersion which I think we can all agree is a very important part of thief.
    Last edited by 13LACK13ISHOP; 11-11-2009 at 01:15 PM.

  16. #16
    Join Date
    May 2009
    Posts
    4,098

    Default

    This was great. Would be very nice if they could do this in Thief IV somehow.

    In lieu of having an ultimate binaural solution, I wonder if placing the foley artist's microphone in just one ear shaped (and textured) contraption would make the sounds more natural to us; as maybe the acoustic waves would get shaped by the contraption's outer and inner ear to record a more pleasant, natural -- less sterile -- sound. (Of course, if they just stick the microphone in the middle of a room without an ear-shaped contraption and record the sound that way, our ears would probably shape the acoustic wave somewhat when we go to play the game. But in this case, the sterile sound files simply get blasted in our ear and it's not as environmentally accurate since we're not actually sitting in the recording room letting the acoustics take shape in our ear there. So it seems like having the wave shaping done in the studio first would help make the sounds more natural when we put on our headsets and play the game. I could be wrong. Sorry I'm unclear here. It makes sense in my mind, I just don't have the time at this moment to make it more clear in text; sorry...

  17. #17
    Join Date
    Sep 2009
    Posts
    134

    Default

    Quote Originally Posted by Herr_Garrett View Post
    The problem is, as far as I can see, this only works with pre-recorded sounds. But should an AI (as in Thief) move about in a non-pre-recorded way, how would that... work?
    In fact, it is possible to write a program, that creates a virtual human head in location of players head physically, then simulate the sound waves in the room the player is located in, then bend and modify these simulated sounds as a real human head would do, then apply this sound as if recorded by the left and right microphone. This would require a very good sound processing power. Contrary to some of the ideas, I believe this solution does not require a real-sound recording. A powerfull software supported with a powerfull processing power can handle all the sound modifications virtually. Basically, virtual sound waves are going to be sent into a 3-D room, and all the sound waves will be expanded mathematically, then they will be modified in the simulation according to where they bent from or what they hit, including the players virtual head and ears, and will be repacked and relayed to the player. This would give, more or less the same results, with the binaural technique. However, I am not an expert, neither in sound techniques nor programming; thus I might be missing some important points. Maybe it is harder then it sounds, maybe it is easier.

    But no matter what, this would really, really rock in a thief game, thats for sure.

  18. #18
    Join Date
    May 2009
    Location
    The Spiraling Sea
    Posts
    2,639

    Default

    Quote Originally Posted by Herr_Garrett View Post
    The problem is, as far as I can see, this only works with pre-recorded sounds. But should an AI (as in Thief) move about in a non-pre-recorded way, how would that... work?
    It wouldn't. This needs to be simulated through a 3D positional sound processor like EAX.

    Quote Originally Posted by DarknessFalls View Post
    In lieu of having an ultimate binaural solution, I wonder if placing the foley artist's microphone in just one ear shaped (and textured) contraption would make the sounds more natural to us; as maybe the acoustic waves would get shaped by the contraption's outer and inner ear to record a more pleasant, natural -- less sterile -- sound. (Of course, if they just stick the microphone in the middle of a room without an ear-shaped contraption and record the sound that way, our ears would probably shape the acoustic wave somewhat when we go to play the game. But in this case, the sterile sound files simply get blasted in our ear and it's not as environmentally accurate since we're not actually sitting in the recording room letting the acoustics take shape in our ear there.
    There are eight different ear shapes, so you would not want to bias the sound this way, as it would serve only one ear type while skewing the others. Binaural recording can not be used for real time gaming, but could be used for any pre-recorded sequence such as a cut-scene.

    So it seems like having the wave shaping done in the studio first would help make the sounds more natural when we put on our headsets and play the game. I could be wrong. Sorry I'm unclear here. It makes sense in my mind, I just don't have the time at this moment to make it more clear in text; sorry...
    Any wave shaping in the studio would only be done to create or modify a sound object for the game. EAX would then process the virtual space using Head Related Transfer Functions (HRTF's) and reverberation to simulate natural acoustics as best as possible.
    Last edited by Vae; 11-11-2009 at 04:03 PM.

  19. #19
    Join Date
    Jan 2008
    Location
    The City
    Posts
    7,636

    Default

    Quote Originally Posted by windwalker View Post
    Dear Jtr,

    I believe it would be possible someday, to mathematically modify a basic sound sample as if it was being directed towards a virtual human head with two microphones inside the ears; however I believe it would take well over a year of dedicated work to achieve this, and it would also be very demanding on the hardware. Imagine an ordinary thief scene; two guards patrolling, one singing, three torches on several locations, lightning sounds from the bad weather, garrets own footsteps... all will have to be modified by each mouse action, ie. where the player turns his face to. Now add walking, leaning, all the moves that a thief player can do... it would take a serious processing power to apply this in real-time game.

    Now I am sure this will one day come true, however it will take at least one full year to achieve this.

    And thinking on it.. we will probably wait more than one year for thief 4, so it is... maybe possible.

    Currently, though, I believe it can be done in game pre-scripted cutscenes, videos between game levels etc. at the moment... And I really, really want to see - actually "hear"- it in a Thief game.
    Heh, I said it's not possible YET. As in TODAY and in the short term. Of COURSE it will happen someday. But this is on the context of Thief 4, is it not?
    Now consider this effect in thief 4.

    Exciting, right?
    You didn't say "Now consider this effect in a Thief game." See how I framed my positive and hopeful response above within the confines of T4?


    Also, it's dang hard to get a processor to simulate a billion soundwaves in motion, absorbing and reflecting, canceling and combining, when there is no air, water, or solids or sound energy in a virtual world. Just as there are no photons doing the same. The killer is that a big processor that spends an hour or four calculating all of it to fake it for a spit-second, and another hour or four for another split-second. To get a processor that can do it seamlessly in real time in a common commercial soundcard is a long time coming. As it is, room sound is general, and location is simple, and the only complexity is sound traveling generally around solids through air (virtual water is just air with different gravity/mass variables, and sound isn't properly muted nor are the higher frequencies quashed. We're talking needing to exponentially increase propagation and occlusion calculations, including virtual head and ear-shapes, not to mention shoulders, which would not be so necessary but eventually important to make the sound even more natural.

    Once again, we need to support science in the effort to create quantum and DNA computers, and then wait for them to become commercially affordable.
    Last edited by jtr7; 11-11-2009 at 09:21 PM.

  20. #20
    Join Date
    May 2009
    Posts
    4,098

    Default

    Quote Originally Posted by Vae View Post
    It wouldn't. This needs to be simulated through a 3D positional sound processor like EAX.
    There are eight different ear shapes, so you would not want to bias the sound this way, as it would serve only one ear type while skewing the others. Binaural recording can not be used for real time gaming, but could be used for any pre-recorded sequence such as a cut-scene.
    Well, the youtube video (...err, audio) apparently used this for the binaural technique: "Basically, two high-tech microphones are placed in a sculptured human head, inside the ears - which are shaped exactly like ears. Then it is placed in the middle of the recording room."

    They didn't have to record it eight different ways to make it sound good to everyone, so I think we'd be safe using one ear shape. I have a feeling using one ear shape, even if it's not ideal for the 7 other ear types, would be beneficial to all ear types ... and would be better (more natural to all) than using no ear shape.

  21. #21
    Join Date
    May 2009
    Location
    The Spiraling Sea
    Posts
    2,639

    Default

    Engineers have used both ear and no ear dummy heads for binaural recording. Some insist on one or the other...some also insist on having shoulders and others even add hair. Most pros using ears do so with an unusually small outer ear. All of these choices are relatively minor in effect, but then some are pickier than others when it comes to sound.

    Anyway, one of the reasons why binaural recording isn't more popular is because you need to use headphones in order to appreciate the full effect...which I like a lot...

  22. #22
    Join Date
    Jan 2008
    Location
    The City
    Posts
    7,636

    Default

    Yep. And with two-speakers in front of you, the sound has to compensate or it's unusual, and with 5.1 or 6.1 surround sound, it would conflict unless the other speakers were assigned non-directional game-mechanic sounds like music, general ambients that are meant to seem to come from everywhere, and Garrett's inner monologue.

  23. #23
    Join Date
    Sep 2009
    Posts
    134

    Default

    It would be possible to chose it from the options menu;

    Binaural - stereo etc.
    About the sound waves being virtually created;

    Yes. It would require tremendous amounts of processing power, and a very good software. Possible the game engine should be calculating the sound waves in-real-time during gameplay to achieve the effect...which means it is close to impossible for T4.

    It should be possible for cutscenes, though.

    And for the garrets inner voice; it could be relayed from the mouth. One can hear his own sound after all Not really an inner sound, but still

    Yet it would be enough to use it in cutscenes and pre-scripted scenes during gameplay, which would be nice story drivers. Then it can switch back to EAX or whatever sound technique the game actually uses.

    Yet I am waiting impatiently for the day that we will be able to hear this full-game. I gave it a one-year but seems it would require more. Maybe like ten.

  24. #24
    Join Date
    Aug 2009
    Location
    Unexpected Places
    Posts
    271

    Default

    HEY MY HAIR! ...oh nm

  25. #25
    Join Date
    Jun 2009
    Posts
    136

    Default

    I am pretty sure I don't understand all of the workings of sound in games especially this new amazing sound technique so don't be too harsh in your anger.. Say they did recordings of sounds at the essentially 26 different sound locations (four directions, four corners and both of these above and below as well), though they wouldn't need to do every location because they could mirror/reverse the sound outputs in the headsets for the other directions. So they record to the right of the "person" and reverse that if the sound is on the left instead and do the same thing with directly in front, in the front-right corner, in front and a little above, in the corner and a little above, to the right and a little above, and directly above the microphones that would be enough to map sound in 360 around a person. That is not my actual point thats just the gears in me brain attempting to understand the concept. Couldn't the sound be an actual part of the A.I.? Like you have a guard walking and whistling. Both sounds project from him in 360 degrees. Imagine the sound as like a mist so its really thick around and near him like a cloud and it spreads further out getting thinner or quieter all the way out until you cant hear it anymore. So couldn't they build a sound system like that? Where the audio like the A.I. is responding to proximity, and direction and the hundred other variables they have successfully used to make Thief work? If anyone could understand that rant please tell me what you think.. just be gentle.
    "This city could do with more red." "That's our lives your talking about!" "I'll probably be horrified once I'm sober."

Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •