XNA Creators Club Online
Page 1 of 1 (7 items)
Sort Posts: Previous Next

Syncing gameplay with audio

Last post 04-22-2008 10:09 PM by Baltico X. 6 replies.
  • 01-15-2008 6:57 PM

    Syncing gameplay with audio

    I see a lot of tutorials on how to sync audio clips with gameplay events but how would one go about doing the reverse, that is having gameplay syncronize with audio. E.g. in Guitar Hero it's integral that the note track is properly synced with the music file that plays. I've tried setting up timers using gameTime.TotalRealTime and gameTime.ElapsedRealTime but they don't even stay syncronized with one another let alone the real time. There must be a better way to do this.

    Thanks a lot,

    Giles.
  • 01-16-2008 1:18 AM In reply to

    Re: Syncing gameplay with audio

    Check out this thread I started: http://forums.xna.com/thread/40281.aspx

    Read up on that and you should get an idea on how to set up a varible for timing. I was trying to get a variable for the scale. Every beat I would get a value of 1. So, you can set an event to trigger when that value is 1.

    Or, if you are simply wanting something like do "X" at "Y" time you could do something like

    if(gametime.TotalGameTime.Minutes == 1 && gametime.TotalGameTime.Seconds == 34 && gametime.TotalGameTime.Milliseconds == 27)
    {
    //do this
    }


    So that would do something at 1min, 34 seconds, 27 miliseconds.



    Things to remember:
    ***TotalSeconds and TotalMilliseconds constantly reset. For example; TotalSeconds will count like this: 57, 58, 59, 0, 1, 2.

    ***If you set something to be valid at any given second, it will be valid that WHOLE second. This is why I always try to use milliseconds. For example, if you say

    if (gametime.TotalGameTime.Seconds == 10)
    {
    //spawn enemy
    }

    then you will spawn like 50 enemies in that one second, because the update loop runs a lot of times each second.


    Check out the pong source code in my blog. I have a class in there called MusicSequencer. It may help you out.
    Try Not! Do, or Do Not. There is no try.
  • 01-16-2008 1:34 PM In reply to

    Re: Syncing gameplay with audio

    Thanks, I've managed to set up a timer and I'm starting the music file at the same time (as soon as the game runs at the moment). Is there a way of loading the music into memory since I am getting an inconsistent lag at the start (somewhere around 0.1s) where I am assuming the music is being loaded?

    As for your code snippet, is it likely that update is called exactly once on that millisecond? I'd have thought you would have to set a flag for when you get past that point. Also syncing with music surely it's better to use the RealTime rather than GameTime returns.

    Is there a possibility that on another system the audio might desync with the beats? Would it be better using markers? I've managed to add markers in the right places in XACT but I can't figure out how to use them inside XNA.
  • 01-17-2008 1:11 PM In reply to

    Re: Syncing gameplay with audio

    I can't be sure but I think your problem lies with how much you're loading up at one time.  Try doing break between loading the game or level and the song, maybe like a "ready" message for 2 or 3 seconds, then call a method for loading the song and setting the current time.

    http://www.freewebs.com/campelmxna/ - C# and XNA tutorials
    The only stupid mistake is the one you make twice
  • 01-18-2008 12:02 PM In reply to

    Re: Syncing gameplay with audio

    Thanks, that seems to work. Now if only XACT supported mp3s....
  • 03-21-2008 12:46 PM In reply to

    Re: Syncing gameplay with audio

    What about getting the audio to play at the correct rate the whole time - is this possible?  My game uses a standard timer and I have events that are set to occur at precise times.  I'd like to set the time based on the music, but if the music gets off sync with the actual time, then things get screwed up. 

    I don't like the idea of the timer being based off of the music because that causes problems.  For example, in Guitar Hero, you might have the scrolling notes be created based off of some timer.  Sure, you could create the notes based off of the song's current time, but then how would they scroll down to the point where the player has to hit them?  If you use a system time-based velocity, then you can get a little off sync during that 1 - 2 seconds that it takes for the note to travel down.  If you base the timing you use for the velocity on the music, then you can get small jumps or jitters if the music has a small jump.  Yeah, it's minor, but all of these little things scream "HACK" to me.  I'd like to just guarantee that my music stays aligned with the time, then do everything according to my accurate system timer.

    Thoughts?
    --Vic--
  • 04-22-2008 10:09 PM In reply to

    Re: Syncing gameplay with audio

    Vic, this is the way I achieved it on my DBP entry.

    I created loopable pieces of music that could be seamessly interchanged and played continuosly if desired. These little loops were several bars long, some were 2, others were 4 bars. Each bar was recorded using a known beat per minute using a metronome of course, for example 130 BPM or 90 BPM.

    It is good game programming practice to make your game go with the rythm of the computer time, so all movement will take place according to the elapsed game time. On Music based games, you go with the rythm of the music, and if you think of it, it is still a time based programming approach.

    To avoid the problem that you described, what I did was to manually play the cues for the next music loop, move everything according to the elapsed time and the estimated duration of a single loop sample, then when the time to play a new loop arrived, sync everything again, so each time a loop is played everything else gets in sync.

    Here is part of a class I called Virtual Orchestra in my game. It updates a public property called isNewMeasure . Also note the method PlayedPercent, which I use to calculate the position of the visual elements according to the estimated music time.

    //.NET Framework 2.0
    using System;
    using System.Collections.Generic;
    //XNA
    using Microsoft.Xna.Framework;
    using Microsoft.Xna.Framework.Audio;
    using Microsoft.Xna.Framework.Content;

    namespace BabyGamer_Musical_Rain
    {
    public class VirtualOrchestra
    {
    private MyGame game;

    public int BPM = 90; //Beats per second
    public int numMeasures = 8; //Number of measures per wave
    public TimeSpan musicTimeSpan = TimeSpan.Zero; //Hold the elapsed time since last Play
    public TimeSpan lastMusicTimeSpan = TimeSpan.Zero; //Hold the elapsed time since of the last loop, for debugging
    public bool isNewMeasure = false;

    public VirtualOrchestra(MyGame game)
    {
    this.game = game;
    }

    public void Update(GameTime gameTime)
    {
    musicTimeSpan += gameTime.ElapsedGameTime;
    //Check if we are in a new music loop
    if (musicTimeSpan.Add(TimeSpan.FromMilliseconds(StimatedPlayCueDelay())) > TimeSpan.FromMilliseconds(60000 * numMeasures / BPM))
    //if (musicTimeSpan > TimeSpan.FromMilliseconds(60000 * numMeasures / BPM))
    {
    isNewMeasure = true;
    //Reset TimeSpan
    lastMusicTimeSpan = musicTimeSpan;
    musicTimeSpan = TimeSpan.Zero;
    }
    else
    {
    isNewMeasure = false;
    }
    }

    public void PlayNextMeasure()
    {
    //Play cue
    game.soundBank.PlayCue("YourLoop");
    }


       public float PlayedPercent
       {
    get { return (float)(musicTimeSpan.TotalMilliseconds / (60000 * numMeasures / BPM)); }
       }

    }
    }

    Put something similir to this in your main update method. Try to do it as early as you can, and leave all the other logic take place after you have played the loop.

                    virtualOrchestra.Update(gameTime);
    if (virtualOrchestra.isNewMeasure)
    {
    //Play Next measure. Must be first instruction of the update functions beacause speed is needed
    virtualOrchestra.PlayNextMeasure();
    }

    Finally a sample of how to do calculations based on the music progress.

    fi.Y = (float)(fallingHeight - fallingHeight * virtualOrchestra.PlayedPercent);

    fi.rotation = MathHelper.TwoPi * virtualOrchestra.PlayedPercent;
    As I stated, this worked for me nicely. In my game musical instruments were falling from the sky and the music was playing accourdingly to the instruments falling and instruments previously caught; you may think of an analogy to Guitar Hero falling notes and previously caught notes. Each instrument took exactly one loop to go from sky to floor, and they rotated exactly 360 degrees (TwoPi) per loop.
Page 1 of 1 (7 items) Previous Next
var gDomain='m.webtrends.com'; var gDcsId='dcschd84w10000w4lw9hcqmsz_8n3x'; var gTrackEvents=1; var gFpc='WT_FPC'; /*<\/scr"+"ipt>");} /*]]>*/
DCSIMG