[Clam-devel] Re: gsoc project - real time synthesizer for a woodwind instrument
Greg Kellum
greg.kellum at gmail.com
Fri May 30 01:47:00 PDT 2008
Hi Yushen,
I was looking at the problem the wrong way. I was looking at it as a
choice between looping at the zero crossings or cross fading. But of
course you can do both, and I guess it will sound better if you do do
both...
Best,
Greg
On May 29, 2008, at 7:04 AM, Han, Yushen wrote:
> Greg,
>
> I see. That makes sense.
> I would not expect to have good loops without crossfading.
> It seems that one can't say much about the quality of the loop until
> he could hear the loop with crossfading.
> I need to look into LoopingSDIFFileReader before I can tell more
> about the loops that the loopMaker found.
>
> I did some test with my oboe samples and the loop (precise to
> sample) sound very good with zero crossings/no crossfading.
> Thanks for your reminder.
>
> Best regards,
> Han, Yushen
>
> On Thu, May 29, 2008 at 12:41 AM, Greg Kellum
> <greg.kellum at gmail.com> wrote:
> Hi Yushen,
>
> The loop making algorithm is not trying to match up the start and end
> points exactly so that you could just loop there; it's intended to be
> used together with crossfading. In my own experience trying to match
> up the zero crossings rather than cross fading doesn't work well;
> Trying that in Kontakt always sounded like crap, but I suppose if
> you're trying to loop very harmonic material with a stable period it
> could work... In any case the looping algorithm is looking for
> spectrally similar frames, and then in my synth, they are crossfaded.
> This happens in the class LoopingSDIFFileReader... Before you
> re-implement the algorithm you might want to try looping your material
> in Kontakt or some other sampler to see if you can get a good loop
> using only zero crossings and no crossfading...
>
> Best,
> Greg
>
>
> On Wed, May 28, 2008 at 10:10 PM, Han, Yushen <yushen.han at gmail.com>
> wrote:
> > Hi,
> >
> > I forget to mention that the most of my samples are 1 - 1.5 sec
> long. (oboe)
> > The stable state is as long as 1 sec usually, which is way too
> short for
> > Greg's algorithm.
> >
> > I could get some good loops if I specify the startSample and
> endSample.
> > But startTime endTime (in ms) did not generate good loops.
> >
> > Best regards,
> > Han, Yushen
> >
> > On Wed, May 28, 2008 at 4:07 PM, Han, Yushen
> <yushen.han at gmail.com> wrote:
> >>
> >> Hi, Pau and Greg
> >>
> >> Thanks to Greg's reply, I am able to covert my audio samples to
> SDIF and
> >> identify loops ( with startTime/endTime in ms specified in a .XML
> file).
> >> But If I extract the loop with startTime/endTime from the SDIF
> file, the
> >> loops contain "clicks" and hence sound bad.
> >>
> >> Here I suspect hat a ms may not be the best unit to specify a loop.
> >> e.g. suppose SR = 44100 Hz, HOP = 256 samples (as used in Greg's
> code),
> >> for a note C4 (f = 261.6Hz), we will have 3.8ms per period i.e
> 168.8
> >> samples per period.
> >> The fraction will lead to round error if we specify the loop by ms.
> >>
> >> In the hope that a sample-by-sample approach will avoid the click,
> >> I am trying to develop my algorithm to identify an optimal loop
> based on
> >> the best f0 given by
> >> segment.GetFrame(counter).GetFundamental();
> >>
> >> However, I check out the reference for segment, frame, and audio
> but did
> >> not know how to specify the unit by sample.
> >>
> >>
> >> http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Segment.html
> >> http://www.clam.iua.upf.edu/doc/CLAM-devel-doxygen/classCLAM_1_1Audio.html
> >>
> >> Do you agree that it is the reason why the loop sounds like
> "clicking"?
> >> Do you know if there is a way to refer to a chunk of audio by
> sample?
> >> Thanks!
> >>
> >> Best regards,
> >> Han, Yushen
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Thu, May 22, 2008 at 6:51 AM, Pau Arumí <parumi at iua.upf.edu>
> wrote:
> >>>
> >>> On dj, 2008-05-22 at 03:11 -0400, Han, Yushen wrote:
> >>> > Hi, Greg and Pau
> >>> >
> >>> > I did some tests on SDIF files out of wave files and so far it
> works
> >>> > fine.
> >>> > Now I am trying to find out the loop of a single note and
> store the
> >>> > loop in .xml file.
> >>> > Here I have questions for each of you separately:
> >>> >
> >>> > (1) for Greg
> >>> > I compiled your makeLoop (from the examples folder)
> successfully.
> >>> > But if I specify the start time and the end time of the steady
> state,
> >>> > makeLoopExe always "calculate 0 EarthMover's distance".
> >>> > If I do not specify the time, makeLoopExe would never finish.
> ( it
> >>> > gives dots ".................." )
> >>> >
> >>> > Did you notice this problem before? Maybe it only happened to
> me.
> >>> >
> >>> > Also, in your previous message you said you "turned off the
> database
> >>> > lookup function".
> >>> > Can you please explain more about what "the lookup function" is
> >>> > supposed to do and why you turned it off?
> >>> >
> >>> > ( I could see that you disabled the residual part in the
> SDIFInConfig.
> >>> > But I am not sure if this is what you were talking about... )
> >>> >
> >>> > (2) for Pau:
> >>> > So far we have not decide which woodwind instrument we should
> work
> >>> > with.
> >>> > Should I work on Saxophone or Oboe?
> >>> > Do we have annotated saxophone notes from SALTO with all loop
> >>> > identified?
> >>> >
> >>> > I cut and labeled 72 oboe notes recorded in an anechoic
> chamber by
> >>> > hand (24 pitches C4-B5 * 3 different dynamics).
> >>> > It will take a while to find the loop for those notes. Before I
> >>> > proceed, I would like to make sure I am working on the right
> >>> > instrument :-)
> >>>
> >>> Hi Yushen,
> >>>
> >>> Yes please, proceed with your oboe notes.
> >>> I'll also search for the old-salto saxophoe database, but there
> are
> >>> chances they are not available.
> >>> It would be interesting to work with a couple (or more)
> instruments,
> >>> though we can concentrate on the sound quality for only one of
> them.
> >>> I think that working with multiple instruments will force us to
> improve
> >>> the tools for creating instruments, which should be one of the
> goals.
> >>> Another goal i'd like to propose is offering transformations not
> typical
> >>> in time-domain based samples, taking advantage of the underlying
> >>> spectral model. Timber morphing from one instrument to another
> is the
> >>> first that comes to my mind.
> >>> Of course, the first milestone is having a usable instrument.
> >>>
> >>> Pau
> >>>
> >>>
> >>> >
> >>> > Best regards,
> >>> > Han, Yushen
> >>> >
> >>> > On Sun, May 18, 2008 at 10:35 AM, Greg Kellum <greg.kellum at gmail.com
> >
> >>> > wrote:
> >>> > Hi,
> >>> >
> >>> > In response to point 3, I turned off the database lookup
> >>> > function at
> >>> > some point, because of an exception that kept
> occurring... I
> >>> > can't
> >>> > remember whether I eventually fixed this or not. I'll
> have to
> >>> > check...
> >>> >
> >>> > Greg
> >>> >
> >>> >
> >>> > On Sat, May 17, 2008 at 6:04 PM, Han, Yushen
> >>> > <yushen.han at gmail.com> wrote:
> >>> > > Hello, Pau
> >>> > >
> >>> > > Thanks for your concerns. Here is some update from me.
> >>> > >
> >>> > > 1. Based on Greg's last update, I am able to compile
> the
> >>> > command-line
> >>> > > version of continuousExcitationSynthesizer.
> >>> > > Also, I can read the XML file where the looping
> information
> >>> > is stored with
> >>> > > his metadata reader.
> >>> > > As for SDIF, I am able to use CLAM/examples/SDIF2Wav
> and
> >>> > Wav2SDIF to convert
> >>> > > between SDIF and WAV.
> >>> > >
> >>> > > 2. In the NetworkEditor, I still could not have
> >>> > "SDIFDatabaseProcessing".
> >>> > > (Greg had reported that he fixed this problem.)
> >>> > > Right now I don't really use the network editor but
> the
> >>> > command-line
> >>> > > executable version.
> >>> > >
> >>> > > To resume our remote debugging, last time you
> concluded
> >>> > > "It seems your OS is not loading some of the
> processings. To
> >>> > confirm
> >>> > > this: how many processings do you have in the 3D Audio
> >>> > category?
> >>> > > You should see 14."
> >>> > > However, I only had 1 processing -
> VectorBasedArrayPanning
> >>> > available after
> >>> > > compiling that 3D plugin.
> >>> > >
> >>> > > 3. I did some experiment with 12 oboe notes in one
> octave
> >>> > and concluded that
> >>> > > if there is only one source sounding,
> >>> > > the first 4 harmonics are quite in phase ( I only
> observe
> >>> > the first 4
> >>> > > harmonics). Therefore I was thinking that we may
> disregard
> >>> > the instantaneous
> >>> > > phase in the SMS. However, this is not true for an
> >>> > orchestra.
> >>> > >
> >>> > > "- [2+ days] To investigate the effect of slowly-
> changing
> >>> > phase changes
> >>> > > between different partials
> >>> > > on the perception of the harmonic sound. If the
> >>> > slowly-change phase did
> >>> > > not affect the sound
> >>> > > very much ( suppose all harmonics are all in
> phase ),
> >>> > further
> >>> > > simplification in synthesis
> >>> > > could be done. This experiment is open to
> suggestion
> >>> > from the mentor."
> >>> > > I will write to you separately about that and more raw
> >>> > ideas.
> >>> > >
> >>> > > 4. Right now I am trying to modify Greg's code and
> Wav2SDIF
> >>> > example for:
> >>> > > "- [1-2 days] To synthesize with SMS with the analysis
> >>> > result (in a simple
> >>> > > setting).
> >>> > > To produce some audible result to evaluate the
> performance
> >>> > of SMS
> >>> > > analysis/transformation."
> >>> > >
> >>> > > It seemed that the Wav2SDIF example had done the
> majority of
> >>> > the work.
> >>> > > I am studying the code but I don't know if I could
> improve
> >>> > this part.
> >>> > > Do you recommend me to read some updated documents
> on SDIF?
> >>> > > http://www.cnmat.berkeley.edu/SDIF/ does not seem to
> be very
> >>> > recent.
> >>> > >
> >>> > > 5. I apologize for my late updating.
> >>> > > I am thinking of keeping a blog on this project...
> >>> > >
> >>> > > Best regards,
> >>> > > Han, Yushen
> >>> > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > On Sat, May 17, 2008 at 7:40 AM, Pau Arumí
> >>> > <parumi at iua.upf.edu> wrote:
> >>> > >>
> >>> > >> Hello Yushen,
> >>> > >>
> >>> > >> how is all doing? I'd like to hear in what you are up
> >>> > regarding your
> >>> > >> project.
> >>> > >>
> >>> > >> are you still blocked with the plugin installation?
> in that
> >>> > case is it
> >>> > >> possible for you have also a linux box?
> >>> > >>
> >>> > >> i'd like you to write some raw ideas about what to be
> >>> > implemented and
> >>> > >> some planning, and then i'll comment and keep
> refining the
> >>> > planning.
> >>> > >> is that ok?
> >>> > >>
> >>> > >> cheers!
> >>> > >> pau
> >>> > >>
> >>> > >>
> >>> > >
> >>> > >
> >>> >
> >>> >
> >>>
> >>
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.clam-project.org/pipermail/clam-devel-clam-project.org/attachments/20080530/c4f0ceea/attachment-0003.htm>
More information about the clam-devel
mailing list