-
|
|
|
Hi!
Is it possible to sync audio samples to the BPM of a song in XNA ?
I want to make a music player, so the sync should be sample-accurate to the BPM.
Looking through the XNA 3.1 API this seems impossible, but maybe I am missing something?
Thanks so much for any reply.
- Lama
|
|
-
-
- (4374)
-
premium membership
-
Posts
1,648
|
|
You probably don't need to post this 3 times in 3 hours. Have a look at the MediaPlayer.GetVisualizationData method - I haven't used it but I would hope it does something along the lines of what you want to do.
|
|
-
|
|
|
Thanks for the reply, and so sorry for any double posts(EDIT: I accidentally posted this first in the DirectX Audio forum)...
Looks to me like GetVisualizationData returns something similar to the FFT of the audio output.
I don't see how that would help me play a sample in sync to a song?
|
|
-
-
- (8307)
-
premium membership
MVP
-
Posts
6,143
|
|
There is no built-in BPM sync. You have to roll your own. The FFT data would be a great place to start. For example, you can look for energy fluctuation in the 80 - 130 Hz range, and when it goes up, that's a "beat" and when it goes down, it's not a beat. Tweaking the tolerances for what means "up" is a black art, and usually has to be somewhat adaptive/learning, to be robust across a large corpus of music.
Jon Watte, Direct3D MVP Tweets, occasionallykW X-port 3ds Max .X exporter kW Animation source code
|
|
-
|
|
|
Thanks, but I'm sorry, I'm not talking about BPM detection (although that is another interesting topic)...
What I'm looking for is really simple:
I want to be able to trigger a sample (say a kick drum for instance) in a constant fixed tempo (say 120 BPM).
EDIT: Sorry that my original question was unclear.
|
|
-
-
- (2363)
-
premium membership
-
Posts
268
|
|
It's possible to more or less trigger a sample at a constant fixed tempo, but it won't be sample-accurate. Just set up a timer for the music and use that to determine when to play audio cues. If you stick to short cues, the timing may be accurate enough for what you want. Trying to do it with long cues (entire songs etc) will probably end up with messed up timing.
My game 'Jammer' works that way, launching cues at the beginning of musical bars as needed. The results aren't sample accurate, but to my ears it's musically workable. After all, normal music played live is not sample accurate. .
|
|
-
-
- (8307)
-
premium membership
MVP
-
Posts
6,143
|
|
You'll probably get the best precision if you spawn a second thread, and put it on hardware thread 4 or 5.
Then, in a tight loop, just call Stopwatch.Now to get the current time, figure out whether it's time to trigger or not, if it is, start the sound effect instance (or XACT cue).
Finally, inside the tight loop, also call AudioEngine.Update() (or whatever that call is, I forget).
You might also want to turn off vsync, and turn off fixed time step, and do very little rendering in your Render(); that will allow you to get called back more often than 60 times a second (which would give you a 17 milliseconds slop, which is too big).
Jon Watte, Direct3D MVP Tweets, occasionallykW X-port 3ds Max .X exporter kW Animation source code
|
|
-
-
- (1388)
-
premium membership
-
Posts
1,118
|
|
I'm also working on a Musical Puzzle game. We tried lots of different permutations of the game's GameTime functions and they all seemed to accumulate time at different rates. In the end I also chose StopWatch and I wrote a Metronome class which fires an event that other classes can subscribe to when the beat has fired.
Here's a cleaned up version of the class I use:
| Metronome metronome; |
| |
| private void InitializeMetronome() |
| { |
| metronome = new Metronome(); |
| metronome.Tempo = 120; |
| metronome.Beat += OnBeat; |
| } |
| |
| private void OnBeat(object sender, EventArgs e) |
| { |
| // Do stuff on beat |
| } |
| public class Metronome |
| { |
| public event EventHandler Beat; |
|
| #region [ Fields and Properties ] |
| |
| public static Stopwatch Watch = new Stopwatch(); |
| private TimeSpan beatTime; |
| private double timeout = 1; |
| |
| /// <summary> |
| /// Number of seconds to pass before |
| /// reading the next level component |
| /// </summary> |
| public Double SecondsPerBeat |
| { |
| get { return timeout; } |
| set { timeout = value; } |
| } |
| |
| /// <summary> |
| /// Tempo in Beats Per Minute |
| /// </summary> |
| public Double Tempo |
| { |
| get { return 60.0 / SecondsPerBeat; } |
| set { timeout = 60.0 / value; } |
| } |
|
| #endregion |
| |
| #region [ Constructor ] |
| |
| /// <summary> |
| /// Main constructor for the Metronome, a counter class |
| /// for spawning track components in time with the music. |
| /// </summary> |
| /// <param name="secondsPerBeat">Number of seconds that pass between beats</param> |
| public Metronome() |
| { |
| Watch.Reset(); |
| } |
|
| #endregion |
| |
| #region [ Game Methods ] |
| |
| /// <summary> |
| /// Keep track of the beat |
| /// </summary> |
| /// <param name="gt">GameTime</param> |
| public void Update(GameTime gt) |
| { |
| if (Watch.Elapsed > beatTime) |
| { |
| beatTime += new TimeSpan(0, 0, 0, 0, (int)(SecondsPerBeat * 1000)); |
| if (Beat != null) Beat(this, null); |
| } |
| } |
|
| #endregion |
| } |
| |
|
|
-
|
|
|
Thanks so much for the replies! I've tried your code MrLeebo.
I can see that this would work for, e.g., a very simple rythm game or as a rough beat estimate, but for sample triggering in a music player it is too inaccurate imo.
By recording the output and comparing it to a sample-accurate example, running at 138 BPM, I've registered beats
that where off by up to 1285 samples or 29ms (44kHz) which of course is audiable.
I also tried simply changing the Update call to sync to the beat, like:
| TargetElapsedTime = new TimeSpan(0, 0, 0, 0, (int)(60.0 * 1000 / |
| metronome.Tempo)); |
But the results where pretty much the same (up to 979 samples ~ 22ms difference).
Now, this is not surprising in any way, since the sound api isn't given any information as to when the sample is to be played, except that it has to be played sometime within the next processed block of audio (EDIT: I'm guessing this is what SoundEffect Play(..) does).
Any other ideas?
|
|
-
-
- (8307)
-
premium membership
MVP
-
Posts
6,143
|
|
|
|
-
|
|
|
Well, to be honest it is not clear to me how xna handles audio internally. But I know XAudio2,
and it seems to me that the problem isn't calling Play(...) at an exact time of the beat:
I made a small test where I update every 20th millisecond ( TargetElapsedTime = new TimeSpan(0, 0, 0, 0, 20) )
And call Play() using a 1 sample impulse sample. If Play() was sample-accurate then the result would be a series of impulses in 20ms intervals.
But instead the result is a series of impulses where some of them have 20ms, some of them 10ms and some of them 30ms spacing.
And at no point does it go into an IsRunningSlowly state.
So I believe that the problem is the Play() function itself. It likely works on blocks of audio (as in XAudio2), and hence
the sample triggering isn't sample accurate to the actual time of the call.
I don't see how introducing a separate thread would solve that problem, but since I'm new to xna I might be mistaken ?
Have you tried it?
|
|
-
-
- (8307)
-
premium membership
MVP
-
Posts
6,143
|
|
When you set TargetElapsedTime, you set a parameter for how often Update() is called in the main thread, on average. Update() will not be called on a real-time schedule for you. That's not what you want for real-time sound.
My suggestion was to move all audio-related work in your application to another thread, to get away from the main thread. It seems you haven't tried that at all. That's the first thing you need to do to be able to get closer to real time.
I'm assuming you're using XACT with Cues. If you're using SoundEffect, then that's not intended to be as precise and well controlled as XACT, AFAICT. That was an assumption on my part -- maybe you didn't know that. You also should use the advanced constructor for the AudioEngine, rather than the default. That allows you to set the "look-ahead" amount, which may or may not in turn affect the amount of buffering.
So, try those things, keeping audio entirely off the main thread, in a tight loop entirely dedicated to pumping and playing audio, and let us know how it goes. It may still be that you can only get 10-millisecond quantization no matter what, which while not great, also isn't entirely terrible (some vintage samplers from the late '80s had similar problems, and were still used frequently).
Jon Watte, Direct3D MVP Tweets, occasionallykW X-port 3ds Max .X exporter kW Animation source code
|
|
-
|
|
|
Thanks for your reply. Your approach just might work. Although I'm still not sure that the actual sample _triggering_ is sample-accurate?!
And I'm an xna noob, so you're right, I didn't know those audio/XACT tweaks you mention.
But getting a mediocre(10ms) beat sync by running a busy loop on a separate CPU/thread is hardly a best case scenario.
I hope you agree that it would be better to utilize the cpu resources for something more meaningful?!
I've only played a little but with XACT...I'm wondering if it is possible to use it to sequence simple music
that is in sync with a sample-accurate BPM ? Ideally I would like to mix several channels of audio, but I'm unsure of
how one should setup and trigger cues so they won't run out of sync?
Thanks again.
|
|
-
|
|
|
Well, after playing with XACT I'm about to give up...
It turns out you CAN trigger multitrack cues and let it start a new sound immediatly when the current sound/loop ends by using transitions and "stop as authored" in XNA/XACT, thus keeping in sync. But the problem is that it starts a "sound" and not another "cue", so afaik you cannot have several cues play after one another while keeping a perfect beat sync?!
I really wish XNA had raw audio access like XAudio2 and Silverlight3. In that case beat sync would be so simple...
|
|
-
-
- (585)
-
premium membership
-
Posts
385
|
|
Yes, you can have several cues play one after another while keeping a perfect beat sync (at least, within 1/60 of a second, since that's how frequently game logic updates). Here's how I did it in a prototype for a game I made a while back.
First I set up an integer called drumTime. Then in the Update method I called this:
| drumTime += gameTime.ElapsedGameTime.Milliseconds; |
Then when I wanted to check if the relevant file should start playing, I called this code:
| if (drumTime > ((60000 / 120 - 22) * 4)) |
| { |
| drumTime = 0; |
| |
| cleandrums1 = soundBank.GetCue("cleandrums1"); |
| cleandrums1.Play(); |
| } |
60 000 represents the number of milliseconds in a minute, 120 represents the BPM I was using, the 4 was because I was using a sound file 1 bar (ie 4 beats) long, and the 22 is because I found I needed a slight delay to get things to line up properly. My code was a bit more complicated than that because I had multiple drum, guitar, etc. files being called at different times, but that's the basics of it. I had no trouble keeping multiple instruments in time using that code.
~ Adam ~ Time Flows, But Does Not Return - a game about the feeling that your life is escaping you Too Big To Fail - a prototype created for September's Experimental Gameplay Project on the theme of "Failure". My Gamasutra blog about game design
|
|
-
-
- (8307)
-
premium membership
MVP
-
Posts
6,143
|
|
Turns out that 1/60th of a second is about 17 milliseconds -- and then you have to add the jitter of the mix-ahead of the software mixer in XACT -- so the "beat" you get out of that will be quite sloppy, compared to what you get out of a sequencer.
The best solution would be to allow pushing/streaming samples from the program. You can then just count samples to generate a steady beat. We've been asking for this for almost three years now; sooner or later we might get it :-)
Jon Watte, Direct3D MVP Tweets, occasionallykW X-port 3ds Max .X exporter kW Animation source code
|
|
-
-
- (585)
-
premium membership
-
Posts
385
|
|
I suppose it depends on what exactly you're syncing. In my case, I was generating dynamic music that responded to the player's actions. None of the musical parts were longer than one bar long, and at the beginning of every bar I reset all my timers back to zero, so nothing would ever go more than a few milliseconds off. It takes a fair bit of effort to record and program dynamic music in segments one bar or less (especially since I was mixing each instrument seperately), but it worked quite well for me.
~ Adam ~ Time Flows, But Does Not Return - a game about the feeling that your life is escaping you Too Big To Fail - a prototype created for September's Experimental Gameplay Project on the theme of "Failure". My Gamasutra blog about game design
|
|
-
|
|
|
Well, to be fair, turns out you CAN sequence a series of wav files in XACT.
The idea is to add several wave files to a simple "play wave" in a track and setting the cue as "ordered".
Then enable "infinite" loop and "new wave on loop" on that play wave.
Technically this means that the cue sounds will play one after another, while keeping in sync, and it works fine inside XACT.
However, I have not been able to get it to work when loading the project inside XNA. The sound doesn't loop. I presume this is a bug(?)...
But even if I could get it to work, I would still need some way to get a loop count from XACT in order for the game to sync correctly, and I don't think this is currently possible? Any clues as to what they are working on in XNA 3.2? ;-)
EDIT: It works inside XNA if you remember to call audioengine.Update()..doh.
But the last problem still holds...is it possible to somehow get the current loop count of a cue?
|
|
-
|
|
|
But the timing is terrible... :(
If I run a looped cue with a single sound (sound 1) and then run another cue with two ordered sounds (of the exact same size as sound 1)...
Then playing both cues will start in sync and sound okay initially, but then the ordered cue will slowly drift out of sync....
Another idea was to set instance limiting to 1 on a cue and then in software play two instances of this cue.
Esentially this means that cue1 is played and then cue2 is played once cue1 finishes.
Technically I could then get a loop count, but I tried it and again XACT fails and inserts gaps between the cues, making the timing useless.
|
|
|