[Clam-devel] Re: GSoC proposal for CLAM: Real-time synthesizer using SMS models
Han, Yushen
yushen.han at gmail.com
Fri Apr 4 08:36:01 PDT 2008
Thanks for your information again, David and Greg.
Now I know that I am expected to make a working interface for a
synthesizer for whatever algorithm.
Probably I will be stuck with SMS this summer since I've seen a few
processing already available for it.
The project turns to be more like a design problem.
In that case, I wonder what level of interaction I should expect for
the user (musicians).
I would like to ask some more questions about this.
What kind of (visual) feedback should I give while synthesizing the sound?
Should I make a complete GUI or a MIDI interface? Or both?
For either a simple MIDI keyboard or a breath controller, ( should we
specify a MIDI device?)
I suppose that I will deal with simple parameters such as pitch,
volume, attach velocity.
But how about pitch modulation, different types of attack behavior?
I am interested in these problems with SMS. But I am not sure if this
is the focus.
-----
To be able to see how Greg's synthesis looks like, now I am having a
problem with
the Processing "SDIFDatabaseProcessing"
(Greg's example network "synth_with_sliders.clamnetwork" needs
SDIFDatabaseProcessing)
I was able to compile SDIFDatabaseProcessing.hxx and .cxx and got .o
and .ox file.
Greg's example network "synth_with_sliders.clamnetwork" needs
SDIFDatabaseProcessing
However, I could not add the processing "SDIFDatabaseProcessing" in
the Network Editor.
It simply complained "The processing type 'SDIFDatabaseProcessing' is
not supported,
even though I manually put SDIFDatabaseProcessing.hxx to my
/usr/local/include along with other working .hxx files.
(I noticed that I can only add a processing if it is listed on the
left of NetworkEditor,
apparently SDIFDatabaseProcessing is not there.)
Greg, somehow I cannot see your state-of-the-art interface, (but could
read your code :-)
It seems that you can have an eBow instrument as the Midi input in a
network, then synthesize the sound.
Is that enough for a synthesizer?
I know you are a digital artist in addition to a developer.
What would you expect me to improve in the interface, from a
musician's perspective?
Does the following link show the right quality of the sound?
http://www.gregkellum.com/temp/a4_multipleloops.mp3
Again, thank you for all your time to support a potential developer for CLAM!
Best regards,
Han, Yushen
On Fri, Apr 4, 2008 at 6:59 AM, David García Garzón <dgarcia at iua.upf.edu> wrote:
> On Divendres 04 Abril 2008, Han, Yushen wrote:
> > Hi,
> > Thank you for clarifying things and walking me into CLAM, Greg.
> > Now I get a better idea about the scope of the project.
> >
> > While estimating the number of hours for my project,
> > I still expect for suggestions from my potential mentor for the
> > following things:
> >
> > (1) I think I need to choose between SPP and SMS in this situation.
> > Now I slightly favor SPP because it is very similar to what I am doing
> > right now.
> >
> > In the light of that the 2004 paper
> > "Real-time Spectral Synthesis for Wind Instruments" By Mayor et al.
> > concluded "Spectral Peak Processing (SPP) achieved better sound quality"
> > I really want to know if you want me to focus on SMS for GSoC.
>
> SPP in CLAM would be great but, as currently there are no support for such
> algorithms. But as Greg said, you have limited time in GSoC to have something
> fully finnished. If you choose SPP the key aspect of your project would be
> having the SPP working and then the synthesizer as optional or post-GSoC task
> just in case SPP comes to be a short task.
>
> In any case, new algorithms are not our priorities this year, but having final
> user interface, improving existing apps and mature CLAM infrastructure. This
> proposal with the SPP algorithm is interesting for us, but just if you commit
> to the project to have some app (synth) working even after GSoC. By our
> experience, althouth this is one of the goals of GSoC, few students keep
> contributing and this would be a weak point on your proposal (unless you have
> a strong need of having such synth because you integrate it as your thesis or
> something like that)
>
> In any case, having the final user application for the synth is more likely to
> get accepted. See the notes at the GSoC 2008 wiki page.
>
> http://clam.iua.upf.edu/wikis/clam/index.php/GSoC_2008#Maximizing_elegibility
>
>
> > (2) It is still a question whether I could use the "original
> > recording" for SALTO?
> > If not, I would like use the pitched notes (with 3 velocities)
> > recorded in anechoic chamber
> > http://theremin.music.uiowa.edu/MIS.html
> > as I am using for my current research project.
> > (Please ask about this if you can, Greg. Thank you in advance!)
> > I did not try making database on my own, Greg,
> > but I think allowing the user to integrate their own samples into SDIF
> > database is a good idea.)
>
> I think that original salto database was published on CLAM web page so i think
> that we have the right to publish them. But i don't know if they need some
> conversion to Greg metadata. Salto data is not currently available on the web
> but this should be a bug, not a license concern. Am i wrong, Pau?
>
> A new database would be great but i suppose you would need studio quality for
> the recording. (i migth be wrong, Greg or Pau could tell you better)
>
>
> > (3) As for the instrument in SALTO, am I supposed to pick up
> > saxophone? (soprano or alto?)
> > Or I can choose any woodwind instrument ( I prefer oboe in that case).
>
> I think so, but think twice on the problem of recording a new database.
>
>
>
> > I am going to submit my proposal on Friday anyway.
> > Before that, I really appreciate if you can give clear answer for things
> > above.
> > Best regards,
> > Han, Yushen
> >
> > On Thu, Apr 3, 2008 at 6:06 AM, Greg Kellum <greg.kellum at gmail.com> wrote:
> > > Hi,
> > >
> > > Well, I personally think it would be great if you reimplemented SALTO
> > > with Spectral Peak Processing, but there is of course no code for SPP
> > > in CLAM. So, you would have to implement this yourself, and you would
> > > likely have to make some changes to the SDIF code so that you could
> > > store information about the location of spectral peaks. I spoke with
> > > Oscar about SPP before starting my own project, and it was something
> > > that I would have liked to have done in addition to SMS, but there
> > > just wasn't enough time. If you found time to do this though, it
> > > would add something new of general and lasting value to CLAM's
> > > codebase... But you need to be careful about realistically planning
> > > your time and setting priorities. At the end ideally you should have
> > > a working saxophone synthesizer that can be controlled via MIDI, and
> > > this is going to be a lot of work in and of itself...
> > >
> > > The original saxophone recordings from the SALTO project are still
> > > around by the way on CD. I'm not sure if the MTG would be willing to
> > > make these publicly, but you could write Oscar Mayor to ask... (Or if
> > > you'd like, I could ask him for you...)
> > >
> > > With regards to the database format, I don't quite understand what you
> > > mean by whether you should aim for implementing a full-sized,
> > > pre-caculated database or a light, customizable one. In the end a
> > > database is a database, and what I worked on last summer was just
> > > providing some flexibility in the way that the database was created so
> > > that one wouldn't need to hard code links to specific files with
> > > specific properties. I did this by creating a metadata file for each
> > > SDIF file. As users might want to replace or refine the sound files
> > > being used by your synth, I would advise you to take a similar
> > > approach...
> > >
> > > With regards to getting to know CLAM better, there are a number of
> > > examples in CLAM's example directory. Some of them show how to use
> > > CLAM as a framework by connecting different CLAM processings, i.e.
> > > modules, together using CLAM networks. Others show how to use CLAM
> > > rather as a class library by connecting together processings by
> > > passing the outputs of one processing to the inputs of another. I
> > > spent a lot of time last summer just writing examples myself trying to
> > > figure out how everything worked...
> > >
> > > I haven't had to time to look at the compilation problems yet, but I
> > > will later this evening...
> > >
> > > Best,
> > > Greg
> > >
> > > On Wed, Apr 2, 2008 at 8:30 AM, Han, Yushen <yushen.han at gmail.com> wrote:
> > > > Hi, GSoC mentors.
> > > >
> > > > (Hope I am sending email about GSoC to the right place.)
> > > > I have 3 questions about the idea - Real-time synthesizer using SMS
> > > > models
> > > >
> > > > (1) I was reading Greg Kellum's thesis "Violin Driven Synthesis from
> > > > Spectral Models"
> > > > which described his project for GSoC 2007:
> > > > continuousExcitationSynthesizer. His system emphasizes the
> > > > customizability of the SDIF database, allows users to work with their
> > > > "potentially insufficient data set" by pitch and brightness
> > > > interpolation.
> > > >
> > > > In Salto project for GSoC 2008, I am supposed to build a database for
> > > > saxophone. I can obtain the (anechoic) sampled sound of every single
> > > > possible pitches with 3 velocities,
> > > > I wonder if a full-sized, pre-caculated database or a light,
> > > > customizable one should be the focus in this project.
> > > > Please advise.
> > > >
> > > > (2)
> > > > SALTO: A Spectral Domain Saxophone Synthesizer By Hass
> > > > documented the SALTO project updated to 2001.
> > > > Is this paper updated enough for a whole picture of the SALTO
> > > > project? I would like to know more about what we do have in SALTO to
> > > > choose a start point.
> > > >
> > > > While Real-time Spectral Synthesis for Wind Instruments By Mayor et
> > > > al. in 2004 concluded that Spectral Peak Processing (SPP) achieved
> > > > better sound quality
> > > > than the implementation by SMS.
> > > > If this is true, should I be stuck at SMS for this real-time
> > > > synthesis project?
> > > >
> > > > (3) Can you please give me some clue about the detail of real-time
> > > > processing in CLAM?
> > > > I use audioQueueService for Mac but was not familiar with CLAM.
> > > > Some reference will be helpful.
> > > >
> > > > Your suggestions will be valuable for me to fit the right work in my
> > > > proposal! (Thanks to Greg for sending me his thesis.)
> > > >
> > > > Best regards,
> > > > Han, Yushen
> > > >
> > > >
> > > > ---
> > > >
> > > > Salto is a CLAM legacy application to synthesize Saxo.
> > > > This project consist on revamp a very simple Salto like synthesizer.
> > > > It consists on:
> > > > * Building a db of sms samples
> > > > * Building processings to
> > > > o process control signals
> > > > o choose samples
> > > > o interpolate
> > > > o create stream flow
> > > > o etc.
>
>
>
> --
> David García Garzón
> (Work) dgarcia at iua dot upf anotherdot es
> http://www.iua.upf.edu/~dgarcia
>
More information about the clam-devel
mailing list