hi, Greg and Pau<br><br>Thank you for your updates. <br>Greg's code was being extremely valuable to me. <br>I did more tests and just made an oboe version for "ContinuousExcitationSynthesizer" by modifying Greg's code.<br>
However, my program crashes unpredictably (for some loops) and the sound is jitterring with my oboe sound samples in SDIF.<br>Here I have a number of questions about Greg's "ContinuousExcitationSynthesizer" to ask you(mostly for Greg).<br>
<br>(1) The format of the score<br><br>I believed that the score for test is given in the format of<br>"frequency amplitude" frame by frame.<br>The test scores given by Greg work fine with his TestScore.<br>e.g.<br>
TestScore::LoadScore. line: 452.596 0.333 <br>TestScore::LoadScore. number of tokens: 2 // only 2 tokens - freq, amp which is correct<br>scoreVetor [0][0] = floatToken =452.596<br>scoreVetor [1][0] = floatToken =0.333<br>
<br>But anything I edited by hand results in segmentation fault<br>because the program mistook multiple lines for one frame.<br><br>e.g. <br>TestScore::LoadScore. line: <br>456.313 0.444<br>433.111 0.431<br>413.131 0.421<br>
436.121 0.421<br>TestScore::LoadScore. number of tokens: 5 // this should not be 5 !<br>scoreVetor [0][4] = floatToken =456.313<br>scoreVetor [1][4] = floatToken =0.444<br>scoreVetor [2][4] = floatToken =0.431<br>Segmentation fault<br>
<br>This problem is really odd to me since both the test score was essentially the same<br>except for that I typed in the second file.<br><br>(2) the synthesized frames<br><br>In the output of the synthesis, I can see *every other frame* in the test score was present in the synthesized audio:<br>
e.g. <br>counter: 0, pitch: 452.596, amplitude: 0.333 // this is the 1st line of longnote.txt<br><br>counter: 1, pitch: 452.783, amplitude: 0.342 // this is the 3rd line of longnote.txt<br>Oraison: creating new voiceId 1212439648<br>
<br>counter: 2, pitch: 432.353, amplitude: 0.39 // this is the 5th line of longnote.txt<br>Oraison: creating new voiceId 1212439648<br><br>counter: 3, pitch: 468.269, amplitude: 0.379 // this is the 7th line of longnote.txt<br>
......<br><br>Why is the even lines in the text score are skipped in the synthesis?<br>The assumption for "one frame" in the score is SR = 44100Hz and HOPSIZE = 256 Samples.<br>Is that correct?<br><br>Also, I noticed there was about 944 frames of non-zero frames ( about 5.5sec). <br>
And there are 945 frames of "pitch 0 amplitue 0" appended to the synthesized audio.<br>The score actually contains 1888 lines (frames).<br>Can you explain why I got only half of the frames synthesized?<br><br>(3) ( to both Pau and Greg )<br>
<br>Besides the problems above, the sound is not good, which perhaps is because the loop points are not that good.<br>I don't know whether the jitterring effect was due to the zero-cross points in the loop or the crossfading frame by frame.<br>
Now I need to figure out this by finding some "perfect" loops.<br><br>Also, I did not use all 3 dynamics in my samples ( only mf now).<br>I am hoping to be able to use the loop in a note of different dynamics, when the amplitude in the score is changed while the frequency does not jump for more than a semitone. (Of course there should not be any new attack.) <br>
Do you think this is the right way to go in order to incorporate the change in dynamics? I am open to your suggestions.<br><br>I don't know if Greg has any official affiliation/obligation in this CLAM project this year,<br>
But I appreciate his help a lot!<br><br>Best regards,<br>Han, Yushen<br><br><br><div class="gmail_quote">On Fri, May 30, 2008 at 4:47 AM, Greg Kellum <<a href="mailto:greg.kellum@gmail.com" target="_blank">greg.kellum@gmail.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>Hi Yushen,<div><br></div><div>I was looking at the problem the wrong way. I was looking at it as a choice between looping at the zero crossings or cross fading. But of course you can do both, and I guess it will sound better if you do do both...</div>
<div><br></div><div>Best,</div><div>Greg</div><div><div></div><div><div><br></div><div><br></div><div><br><div>On May 29, 2008, at 7:04 AM, Han, Yushen wrote:<br><blockquote type="cite">Greg,<br><br>I see. That makes sense.<br>
I would not expect to have good loops without crossfading.<br>It seems that one can't say much about the quality of the loop until he could hear the loop with crossfading.<br>I need to look into LoopingSDIFFileReader before I can tell more about the loops that the loopMaker found.<br>
<br>I did some test with my oboe samples and the loop (precise to sample) sound very good with zero crossings/no crossfading.<br>Thanks for your reminder.<br><br>Best regards,<br>Han, Yushen<br><br><div class="gmail_quote">
On Thu, May 29, 2008 at 12:41 AM, Greg Kellum <<a href="mailto:greg.kellum@gmail.com" target="_blank">greg.kellum@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi Yushen,<br> <br> The loop making algorithm is not trying to match up the start and end<br> points exactly so that you could just loop there; it's intended to be<br> used together with crossfading. In my own experience trying to match<br>
up the zero crossings rather than cross fading doesn't work well;<br> Trying that in Kontakt always sounded like crap, but I suppose if<br> you're trying to loop very harmonic material with a stable period it<br>
could work... In any case the looping algorithm is looking for<br> spectrally similar frames, and then in my synth, they are crossfaded.<br> This happens in the class LoopingSDIFFileReader... Before you<br> re-implement the algorithm you might want to try looping your material<br>
in Kontakt or some other sampler to see if you can get a good loop<br> using only zero crossings and no crossfading...<br> <br> Best,<br> <font color="#888888">Greg<br> </font><div><div></div><div><br> <br> On Wed, May 28, 2008 at 10:10 PM, Han, Yushen <<a href="mailto:yushen.han@gmail.com" target="_blank">yushen.han@gmail.com</a>> wrote:<br>
> Hi,<br> ><br> > I forget to mention that the most of my samples are 1 - 1.5 sec long. (oboe)<br> > The stable state is as long as 1 sec usually, which is way too short for<br> > Greg's algorithm.<br>
><br> > I could get some good loops if I specify the startSample and endSample.<br> > But startTime endTime (in ms) did not generate good loops.<br> ><br> > Best regards,<br> > Han, Yushen<br> ><br>
> On Wed, May 28, 2008 at 4:07 PM, Han, Yushen <<a href="mailto:yushen.han@gmail.com" target="_blank">yushen.han@gmail.com</a>> wrote:<br>
>><br> >> Hi, Pau and Greg<br> >><br> >> Thanks to Greg's reply, I am able to covert my audio samples to SDIF and<br> >> identify loops ( with startTime/endTime in ms specified in a .XML file).<br>
>> But If I extract the loop with startTime/endTime from the SDIF file, the<br> >> loops contain "clicks" and hence sound bad.<br> >><br> >> Here I suspect hat a ms may not be the best unit to specify a loop.<br>
>> e.g. suppose SR = 44100 Hz, HOP = 256 samples (as used in Greg's code),<br> >> for a note C4 (f = 261.6Hz), we will have 3.8ms per period i.e 168.8<br> >> samples per period.<br> >> The fraction will lead to round error if we specify the loop by ms.<br>
>><br> >> In the hope that a sample-by-sample approach will avoid the click,<br> >> I am trying to develop my algorithm to identify an optimal loop based on<br> >> the best f0 given by<br> >> segment.GetFrame(counter).GetFundamental();<br>
>><br> >> However, I check out the reference for segment, frame, and audio but did<br> >> not know how to specify the unit by sample.<br> >><br> >><br> >> <a href="http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Segment.html" target="_blank">http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Segment.html</a><br>
>> <a href="http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Audio.html" target="_blank">http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Audio.html</a><br> >><br> >> Do you agree that it is the reason why the loop sounds like "clicking"?<br>
>> Do you know if there is a way to refer to a chunk of audio by sample?<br> >> Thanks!<br> >><br> >> Best regards,<br> >> Han, Yushen<br> >><br> >><br> >><br> >><br>
>><br> >><br> >><br> >><br> >> On Thu, May 22, 2008 at 6:51 AM, Pau Arumí <<a href="mailto:parumi@iua.upf.edu" target="_blank">parumi@iua.upf.edu</a>> wrote:<br> >>><br> >>> On dj, 2008-05-22 at 03:11 -0400, Han, Yushen wrote:<br>
>>> > Hi, Greg and Pau<br> >>> ><br> >>> > I did some tests on SDIF files out of wave files and so far it works<br> >>> > fine.<br> >>> > Now I am trying to find out the loop of a single note and store the<br>
>>> > loop in .xml file.<br> >>> > Here I have questions for each of you separately:<br> >>> ><br> >>> > (1) for Greg<br> >>> > I compiled your makeLoop (from the examples folder) successfully.<br>
>>> > But if I specify the start time and the end time of the steady state,<br> >>> > makeLoopExe always "calculate 0 EarthMover's distance".<br> >>> > If I do not specify the time, makeLoopExe would never finish. ( it<br>
>>> > gives dots ".................." )<br> >>> ><br> >>> > Did you notice this problem before? Maybe it only happened to me.<br> >>> ><br> >>> > Also, in your previous message you said you "turned off the database<br>
>>> > lookup function".<br> >>> > Can you please explain more about what "the lookup function" is<br> >>> > supposed to do and why you turned it off?<br> >>> ><br>
>>> > ( I could see that you disabled the residual part in the SDIFInConfig.<br> >>> > But I am not sure if this is what you were talking about... )<br> >>> ><br> >>> > (2) for Pau:<br>
>>> > So far we have not decide which woodwind instrument we should work<br> >>> > with.<br> >>> > Should I work on Saxophone or Oboe?<br> >>> > Do we have annotated saxophone notes from SALTO with all loop<br>
>>> > identified?<br> >>> ><br> >>> > I cut and labeled 72 oboe notes recorded in an anechoic chamber by<br> >>> > hand (24 pitches C4-B5 * 3 different dynamics).<br> >>> > It will take a while to find the loop for those notes. Before I<br>
>>> > proceed, I would like to make sure I am working on the right<br> >>> > instrument :-)<br> >>><br> >>> Hi Yushen,<br> >>><br> >>> Yes please, proceed with your oboe notes.<br>
>>> I'll also search for the old-salto saxophoe database, but there are<br> >>> chances they are not available.<br> >>> It would be interesting to work with a couple (or more) instruments,<br>
>>> though we can concentrate on the sound quality for only one of them.<br> >>> I think that working with multiple instruments will force us to improve<br> >>> the tools for creating instruments, which should be one of the goals.<br>
>>> Another goal i'd like to propose is offering transformations not typical<br> >>> in time-domain based samples, taking advantage of the underlying<br> >>> spectral model. Timber morphing from one instrument to another is the<br>
>>> first that comes to my mind.<br> >>> Of course, the first milestone is having a usable instrument.<br> >>><br> >>> Pau<br> >>><br> >>><br> >>> ><br> >>> > Best regards,<br>
>>> > Han, Yushen<br> >>> ><br> >>> > On Sun, May 18, 2008 at 10:35 AM, Greg Kellum <<a href="mailto:greg.kellum@gmail.com" target="_blank">greg.kellum@gmail.com</a>><br> >>> > wrote:<br>
>>> > Hi,<br> >>> ><br> >>> > In response to point 3, I turned off the database lookup<br> >>> > function at<br> >>> > some point, because of an exception that kept occurring... I<br>
>>> > can't<br> >>> > remember whether I eventually fixed this or not. I'll have to<br> >>> > check...<br> >>> ><br> >>> > Greg<br>
>>> ><br> >>> ><br> >>> > On Sat, May 17, 2008 at 6:04 PM, Han, Yushen<br> >>> > <<a href="mailto:yushen.han@gmail.com" target="_blank">yushen.han@gmail.com</a>> wrote:<br>
>>> > > Hello, Pau<br> >>> > ><br> >>> > > Thanks for your concerns. Here is some update from me.<br> >>> > ><br> >>> > > 1. Based on Greg's last update, I am able to compile the<br>
>>> > command-line<br> >>> > > version of continuousExcitationSynthesizer.<br> >>> > > Also, I can read the XML file where the looping information<br> >>> > is stored with<br>
>>> > > his metadata reader.<br> >>> > > As for SDIF, I am able to use CLAM/examples/SDIF2Wav and<br> >>> > Wav2SDIF to convert<br> >>> > > between SDIF and WAV.<br>
>>> > ><br> >>> > > 2. In the NetworkEditor, I still could not have<br> >>> > "SDIFDatabaseProcessing".<br> >>> > > (Greg had reported that he fixed this problem.)<br>
>>> > > Right now I don't really use the network editor but the<br> >>> > command-line<br> >>> > > executable version.<br> >>> > ><br>
>>> > > To resume our remote debugging, last time you concluded<br> >>> > > "It seems your OS is not loading some of the processings. To<br> >>> > confirm<br>
>>> > > this: how many processings do you have in the 3D Audio<br> >>> > category?<br> >>> > > You should see 14."<br> >>> > > However, I only had 1 processing - VectorBasedArrayPanning<br>
>>> > available after<br> >>> > > compiling that 3D plugin.<br> >>> > ><br> >>> > > 3. I did some experiment with 12 oboe notes in one octave<br>
>>> > and concluded that<br> >>> > > if there is only one source sounding,<br> >>> > > the first 4 harmonics are quite in phase ( I only observe<br> >>> > the first 4<br>
>>> > > harmonics). Therefore I was thinking that we may disregard<br> >>> > the instantaneous<br> >>> > > phase in the SMS. However, this is not true for an<br>
>>> > orchestra.<br> >>> > ><br> >>> > > "- [2+ days] To investigate the effect of slowly-changing<br> >>> > phase changes<br> >>> > > between different partials<br>
>>> > > on the perception of the harmonic sound. If the<br> >>> > slowly-change phase did<br> >>> > > not affect the sound<br> >>> > > very much ( suppose all harmonics are all in phase ),<br>
>>> > further<br> >>> > > simplification in synthesis<br> >>> > > could be done. This experiment is open to suggestion<br> >>> > from the mentor."<br>
>>> > > I will write to you separately about that and more raw<br> >>> > ideas.<br> >>> > ><br> >>> > > 4. Right now I am trying to modify Greg's code and Wav2SDIF<br>
>>> > example for:<br> >>> > > "- [1-2 days] To synthesize with SMS with the analysis<br> >>> > result (in a simple<br> >>> > > setting).<br>
>>> > > To produce some audible result to evaluate the performance<br> >>> > of SMS<br> >>> > > analysis/transformation."<br> >>> > ><br>
>>> > > It seemed that the Wav2SDIF example had done the majority of<br> >>> > the work.<br> >>> > > I am studying the code but I don't know if I could improve<br>
>>> > this part.<br> >>> > > Do you recommend me to read some updated documents on SDIF?<br> >>> > > <a href="http://www.cnmat.berkeley.edu/SDIF/" target="_blank">http://www.cnmat.berkeley.edu/SDIF/</a> does not seem to be very<br>
>>> > recent.<br> >>> > ><br> >>> > > 5. I apologize for my late updating.<br> >>> > > I am thinking of keeping a blog on this project...<br>
>>> > ><br> >>> > > Best regards,<br> >>> > > Han, Yushen<br> >>> > ><br> >>> > ><br> >>> > ><br>
>>> > ><br> >>> > > On Sat, May 17, 2008 at 7:40 AM, Pau Arumí<br> >>> > <<a href="mailto:parumi@iua.upf.edu" target="_blank">parumi@iua.upf.edu</a>> wrote:<br>
>>> > >><br> >>> > >> Hello Yushen,<br> >>> > >><br> >>> > >> how is all doing? I'd like to hear in what you are up<br>
>>> > regarding your<br> >>> > >> project.<br> >>> > >><br> >>> > >> are you still blocked with the plugin installation? in that<br>
>>> > case is it<br> >>> > >> possible for you have also a linux box?<br> >>> > >><br> >>> > >> i'd like you to write some raw ideas about what to be<br>
>>> > implemented and<br> >>> > >> some planning, and then i'll comment and keep refining the<br> >>> > planning.<br> >>> > >> is that ok?<br>
>>> > >><br> >>> > >> cheers!<br> >>> > >> pau<br> >>> > >><br> >>> > >><br> >>> > ><br>
>>> > ><br> >>> ><br> >>> ><br> >>><br> >><br> ><br> ><br> </div></div></blockquote></div><br></blockquote></div><br></div></div></div></div></blockquote>
</div><br>