[Clam-devel] Re: GSoC proposal for CLAM: Real-time synthesizer using SMS models

Han, Yushen yushen.han at gmail.com
Fri Apr 4 08:36:01 PDT 2008


Thanks for your information again, David and Greg.

Now I know that I am expected to make a working interface for a
synthesizer for whatever algorithm.
Probably I will be stuck with SMS this summer since I've seen a few
processing already available for it.

The project turns to be more like a design problem.
In that case, I wonder what level of interaction I should expect for
the user (musicians).
I would like to ask some more questions about this.

What kind of (visual) feedback should I give while synthesizing the sound?

Should I make a complete GUI or a MIDI interface? Or both?

For either a simple MIDI keyboard or a breath controller, ( should we
specify a MIDI device?)
I suppose that I will deal with simple parameters such as pitch,
volume, attach velocity.
But how about pitch modulation, different types of attack behavior?
I am interested in these problems with SMS. But I am not sure if this
is the focus.

-----

To be able to see how Greg's synthesis looks like, now I am having a
problem with
the Processing "SDIFDatabaseProcessing"
(Greg's example network "synth_with_sliders.clamnetwork" needs
SDIFDatabaseProcessing)

I was able to compile SDIFDatabaseProcessing.hxx and .cxx and got .o
and .ox file.
Greg's example network "synth_with_sliders.clamnetwork" needs
SDIFDatabaseProcessing

However, I could not add the processing "SDIFDatabaseProcessing" in
the Network Editor.
It simply complained "The processing type 'SDIFDatabaseProcessing' is
not supported,
even though I manually put SDIFDatabaseProcessing.hxx to my
/usr/local/include along with other working .hxx files.
(I noticed that I can only add a processing if it is listed on the
left of NetworkEditor,
apparently SDIFDatabaseProcessing is not there.)

Greg, somehow I cannot see your state-of-the-art interface, (but could
read your code :-)
It seems that you can have an eBow instrument as the Midi input in a
network, then synthesize the sound.
Is that enough for a synthesizer?
I know you are a digital artist in addition to a developer.
What would you expect me to improve in the interface, from a
musician's perspective?

Does the following link show the right quality of the sound?
http://www.gregkellum.com/temp/a4_multipleloops.mp3

Again, thank you for all your time to support a potential developer for CLAM!

Best regards,
Han, Yushen

On Fri, Apr 4, 2008 at 6:59 AM, David García Garzón <dgarcia at iua.upf.edu> wrote:
> On Divendres 04 Abril 2008, Han, Yushen wrote:
>  > Hi,
>  > Thank you for clarifying things and walking me into CLAM, Greg.
>  > Now I get a better idea about the scope of the project.
>  >
>  > While estimating the number of hours for my project,
>  > I still expect for suggestions from my potential mentor  for the
>  > following things:
>  >
>  > (1) I think I need to choose between SPP and SMS in this situation.
>  > Now I slightly favor SPP because it is very similar to what I am doing
>  > right now.
>  >
>  > In the light of that the 2004 paper
>  > "Real-time Spectral Synthesis for Wind Instruments" By Mayor et al.
>  > concluded "Spectral Peak Processing (SPP) achieved better sound quality"
>  > I really want to know if you want me to focus on SMS for GSoC.
>
>  SPP in CLAM would be great but, as currently there are no support for such
>  algorithms. But as Greg said, you have limited time in GSoC to have something
>  fully finnished. If you choose SPP the key aspect of your project would be
>  having the SPP working and then the synthesizer as optional or post-GSoC task
>  just in case SPP comes to be a short task.
>
>  In any case, new algorithms are not our priorities this year, but having final
>  user interface, improving existing apps and mature CLAM infrastructure. This
>  proposal with the SPP algorithm is interesting for us, but just if you commit
>  to the project to have some app (synth) working even after GSoC. By our
>  experience, althouth this is one of the goals of GSoC, few students keep
>  contributing and this would be a weak point on your proposal (unless you have
>  a strong need of having such synth because you integrate it as your thesis or
>  something like that)
>
>  In any case, having the final user application for the synth is more likely to
>  get accepted. See the notes at the GSoC 2008 wiki page.
>
>  http://clam.iua.upf.edu/wikis/clam/index.php/GSoC_2008#Maximizing_elegibility
>
>
>  > (2) It is still a question whether I could use the "original
>  > recording" for SALTO?
>  > If not, I would like use the pitched notes (with 3 velocities)
>  > recorded in anechoic chamber
>  > http://theremin.music.uiowa.edu/MIS.html
>  > as I am using for my current research project.
>  > (Please ask about this if you can, Greg. Thank you in advance!)
>  > I did not try making database on my own, Greg,
>  > but I think allowing the user to integrate their own samples into SDIF
>  > database is a good idea.)
>
>  I think that original salto database was published on CLAM web page so i think
>  that we have the right to publish them. But i don't know if they need some
>  conversion to Greg metadata. Salto data is not currently available on the web
>  but this should be a bug, not a license concern. Am i wrong, Pau?
>
>  A new database would be great but i suppose you would need studio quality for
>  the recording. (i migth be wrong, Greg or Pau could tell you better)
>
>
>  > (3) As for the instrument in SALTO, am I supposed to pick up
>  > saxophone? (soprano or alto?)
>  > Or I can choose any woodwind instrument ( I prefer oboe in that case).
>
>  I think so, but think twice on the problem of recording a new database.
>
>
>
>  > I am going to submit my proposal on Friday anyway.
>  > Before that, I really appreciate if you can give clear answer for things
>  > above.
>  > Best regards,
>  > Han, Yushen
>  >
>  > On Thu, Apr 3, 2008 at 6:06 AM, Greg Kellum <greg.kellum at gmail.com> wrote:
>  > > Hi,
>  > >
>  > >  Well, I personally think it would be great if you reimplemented SALTO
>  > >  with Spectral Peak Processing, but there is of course no code for SPP
>  > >  in CLAM.  So, you would have to implement this yourself, and you would
>  > >  likely have to make some changes to the SDIF code so that you could
>  > >  store information about the location of spectral peaks.  I spoke with
>  > >  Oscar about SPP before starting my own project, and it was something
>  > >  that I would have liked to have done in addition to SMS, but there
>  > >  just wasn't enough time.  If you found time to do this though, it
>  > >  would add something new of general and lasting value to CLAM's
>  > >  codebase...  But you need to be careful about realistically planning
>  > >  your time and setting priorities.  At the end ideally you should have
>  > >  a working saxophone synthesizer that can be controlled via MIDI, and
>  > >  this is going to be a lot of work in and of itself...
>  > >
>  > >  The original saxophone recordings from the SALTO project are still
>  > >  around by the way on CD.  I'm not sure if the MTG would be willing to
>  > >  make these publicly, but you could write Oscar Mayor to ask...  (Or if
>  > >  you'd like, I could ask him for you...)
>  > >
>  > >  With regards to the database format, I don't quite understand what you
>  > >  mean by whether you should aim for implementing a full-sized,
>  > >  pre-caculated database or a light, customizable one.  In the end a
>  > >  database is a database, and what I worked on last summer was just
>  > >  providing some flexibility in the way that the database was created so
>  > >  that one wouldn't need to hard code links to specific files with
>  > >  specific properties.  I did this by creating a metadata file for each
>  > >  SDIF file.  As users might want to replace or refine the sound files
>  > >  being used by your synth, I would advise you to take a similar
>  > >  approach...
>  > >
>  > >  With regards to getting to know CLAM better, there are a number of
>  > >  examples in CLAM's example directory.  Some of them show how to use
>  > >  CLAM as a framework by connecting different CLAM processings, i.e.
>  > >  modules, together using CLAM networks.  Others show how to use CLAM
>  > >  rather as a class library by connecting together processings by
>  > >  passing the outputs of one processing to the inputs of another.  I
>  > >  spent a lot of time last summer just writing examples myself trying to
>  > >  figure out how everything worked...
>  > >
>  > >  I haven't had to time to look at the compilation problems yet, but I
>  > >  will later this evening...
>  > >
>  > >  Best,
>  > >  Greg
>  > >
>  > >  On Wed, Apr 2, 2008 at 8:30 AM, Han, Yushen <yushen.han at gmail.com> wrote:
>  > >  > Hi, GSoC mentors.
>  > >  >
>  > >  >  (Hope I am sending email about GSoC to the right place.)
>  > >  >  I have 3 questions about the idea -  Real-time synthesizer using SMS
>  > >  > models
>  > >  >
>  > >  >  (1) I was reading Greg Kellum's thesis "Violin Driven Synthesis from
>  > >  >  Spectral Models"
>  > >  >  which described his project for GSoC 2007:
>  > >  > continuousExcitationSynthesizer. His system emphasizes the
>  > >  > customizability of the SDIF database, allows users to work with their
>  > >  > "potentially insufficient data set" by pitch and brightness
>  > >  > interpolation.
>  > >  >
>  > >  >  In Salto project for GSoC 2008, I am supposed to build a database for
>  > >  > saxophone. I can obtain the (anechoic) sampled sound of every single
>  > >  > possible pitches with 3 velocities,
>  > >  >  I wonder if a full-sized, pre-caculated database or a light,
>  > >  >  customizable one should be the focus in this project.
>  > >  >  Please advise.
>  > >  >
>  > >  >  (2)
>  > >  >  SALTO: A Spectral Domain Saxophone Synthesizer By Hass
>  > >  >  documented the SALTO project updated to 2001.
>  > >  >  Is this paper updated enough for a whole picture of the SALTO
>  > >  > project? I would like to know more about what we do have in SALTO to
>  > >  > choose a start point.
>  > >  >
>  > >  >  While Real-time Spectral Synthesis for Wind Instruments By Mayor et
>  > >  > al. in 2004 concluded that Spectral Peak Processing (SPP) achieved
>  > >  > better sound quality
>  > >  >  than the implementation by SMS.
>  > >  >  If this is true, should I be stuck at SMS for this real-time
>  > >  > synthesis project?
>  > >  >
>  > >  >  (3) Can you please give me some clue about the detail of real-time
>  > >  >  processing in CLAM?
>  > >  >  I use audioQueueService for Mac but was not familiar with CLAM.
>  > >  >  Some reference will be helpful.
>  > >  >
>  > >  >  Your suggestions will be valuable for me to fit the right work in my
>  > >  > proposal! (Thanks to Greg for sending me his thesis.)
>  > >  >
>  > >  >  Best regards,
>  > >  >  Han, Yushen
>  > >  >
>  > >  >
>  > >  >  ---
>  > >  >
>  > >  >  Salto is a CLAM legacy application to synthesize Saxo.
>  > >  >   This project consist on revamp a very simple Salto like synthesizer.
>  > >  >  It consists on:
>  > >  >     * Building a db of sms samples
>  > >  >     * Building processings to
>  > >  >           o process control signals
>  > >  >           o choose samples
>  > >  >           o interpolate
>  > >  >           o create stream flow
>  > >  >           o etc.
>
>
>
>  --
>  David García Garzón
>  (Work) dgarcia at iua dot upf anotherdot es
>  http://www.iua.upf.edu/~dgarcia
>




More information about the clam-devel mailing list