Fwd: [Clam-devel] CLAM and NASPRO

[Stefano D'Angelo]

sorry again from cross-posting.. I just don't know why gmail behaves this way...

---------- Forwarded message ----------
From: Stefano D'Angelo <zanga.mail at gmail.com>
Date: 2-mar-2007 19.00
Subject: Re: [Clam-devel] CLAM and NASPRO
To: David García Garzón <dgarcia at iua.upf.edu>


2007/3/2, David García Garzón <dgarcia at iua.upf.edu>:
> On Thursday 01 March 2007 20:09:07 Stefano D'Angelo wrote:
> > 2007/3/1, Pau Arumi <parumi at iua.upf.edu>:
> > > En/na Stefano D'Angelo ha escrit:
> > > >> however, i think its wiser to start simple and split modules only
> > > >> when the advantage is apparent (yes, enabling parallel
> > > >> development *is* an advantage).
> > > >> but given the ladspa experience, and the fact that the number of
> > > >> wrappers is not meant to grow very much (like plugins do) or have
> > > >> other dependencies (like audio backends do) i don't really see
> > > >> the need right now.
> > > >> but maybe you'll give me reasons to convince me of the contrary.
> > > >
> > > > Yes I'm going to do that (repeating some basic stuff since it could
> > > > make you understand better what I mean) :-)
> > > > All plugin APIs have one thing in common: a function that takes some
> > > > input and throws out some output (as well as inputs and outputs lists,
> > > > etc.).
> > > > Then they build around such "functional core" all the bells and
> > > > whistles (logical metaphor, GUI handling, session storing, etc.) which
> > > > really makes them incompatible between each other (you can always
> > > > build bridges but the result usually does not have 100% functionality
> > > > of "the original implementation").
> > > > The way which all such APIs (I know of) work is by an exclusively
> > > > algorithmic core, which the host application doesn't know about. It
> > > > just feeds input and takes output (plugins are a black box).
> > > > As I and other people pointed out, there's a big margin of improvement
> > > > that can be made by letting the host know about the real plugin
> > > > functionality (once again: SFTF/z-transform/zeros and poles etc.),
> > > > especially if they are arranged in a network. For example all
> > > > subsequent LTI filters can be "merged" in only one filter knowing
> > > > their transfer function (a practical case: an equalizer and a delay in
> > > > series).
> > >
> > > i'm not interested at all with that kind of efficiency improvements in a
> > > plugin host. all hosts see plugins as black-boxes and that why they are
> > > successful, imo.
> > > i don't even think that many typical plugins are really LTI filters. but
> > > anyway, why don't you see this as a toolkit for compiling plugins,
> > > managing a set of white-box algorithms. this would make sense to me. but
> > > a host is another story.
> >
> > Optimization is not always related to LTI filters... the concept I'm
> > talking about is a bit wider, that could be one thing.
> > Take it this way: what if plugins know how to "merge" them with other
> > plugins (which is equivalent to saying that they give some functional
> > representation of what they do to the host - see below)? As someone
> > said on the LAD, it would be like overriding the + operator in C++.
> > Then... I'm not willing to write a host or a new standard! I'm going
> > to basically develop a library!
> > So when I was saying "host" before, I was referring to the library,
> > which does that for the host.
>
> So, then i don't think you are so far from what we are doing. We provide tools
> to build algorithms. Those algorithms could become a plugin, and, so far, the
> algorithms could use existing plugins.

Absolutely not, it's much like the same thing.
The only difference I might see is that optimization when you
"generate" a processing object by networks or other stuff is quite
straightforward, but no host application I know does optimizations
when launched, and I think that it cannot be done these days.
That's a pity, since the only thing that a host (or a library the host
uses) is a functional representation of what processing objects do. If
such could be provided things would get better.
It's not just an idea of mine, also Steve Harris said on the LAD ml
that he's gonna implement an extension to LV2 going in such direction
using the STFT.

> > > > No plugin arch is currently capable of taking advantage of such
> > > > things, apart from LV2 via extensions (see below).
> > > > In this prospective, there is also the possibility to replace the
> > > > current "plugin fashion" with other kind of files which could be more
> > > > manageable and easy to develop (text, XML, etc.)
> > > > Getting back to different plugin arch features, there's also the
> > > > possibility to supply externally features which a given standard
> > > > misses (for example session storing for DSSI/LADSPA).
> > > >
> > > > How to do all of that?
> > > > Two ways:
> > > >
> > > > 1. build a new extensible API. And that's LV2.
> > > > What's wrong with it? In my opinion the extension system since it does
> > > > not solve compatibility problems, it creates new ones!
> > > > What if there will be two extensions that do the same thing with some
> > > > little difference that makes them incompatible? Either one "wins" or
> > > > it becomes a new issue! The same old story.
> > > >
> > > > 2. implement an extensible wrapper. And that's NASPRO.
> > > > Why should it be superior?
> > > > a. It's not going to be a "floating standard", but a solid
> > > > implementation. Developers do not need to reinvent the wheel, and
> > > > people who want to experiment with a new API or to bring new features
> > > > to an existing API just have to write (or modify) a module, and that
> > > > will work on any NASPRO host. The same goes for people who are
> > > > creating new formats which are not based on dynamic modules (this
> > > > could have a great meaning in the research field).
> > > > b. modularity often means consistency, since module developers are
> > > > forced to use a standard API to accomplish their tasks (of course I'm
> > > > not saying that hard-coding implies writing a mess, but sometimes
> > > > there's such tendency)
> > > > c. again, knowing about the network can bring to optimizations, and in
> > > > this case these could be done by NASPRO itself, maybe together with
> > > > its modules. (I still haven't found out how to do this in a "standard"
> > > > way).
> > > >
> > > > So, coming back to the original question: modularity brings the
> > > > features of letting non-hardcore developers easily extend your work,
> > > > let them experiment and play with different ways of doing the "same
> > > > old task", and, last but not least, raise interest to your project.
> > >
> > > standards are all about having people to use them. now that the LV2 is
> > > in its definition stage, i don't think that proposing a new standard will
> > > help you to have more users. why not try to convince the LV2 proposers
> > > that you have a better idea? and *specially* providing working examples
> > > and extensions.
> >
> > Again... I'm not proposing a new standard, I'm offering a way to use
> > existing ones!
> > The "two ways" are two ways of solving the problem, since LV2 authors
> > have addressed the same issue, but I do not agree with them that LV2
> > extensions are valuable as a solution.
> > Personally, I do not even agree that plugins should implement
> > functionality which has nothing to do with their natural task of
> > processing sound (GUIs being an example), but anyway that's how it
> > goes right now.
> > Here you see the diffence: in LV2 plugin authors are encouraged (or
> > helped) to put new stuff into plugins via extensions, in NASPRO
> > they're encouraged to work exclusively on the DSP part, and if they
> > miss a feature, they can implement directly in the library itself, via
> > modules, so that it becomes available to everyone.
>
> Processing without GUI... all right, that's what Ladspa 1 was about. But of
> course this is not enough and that's because they are thinking on LV2.

I thought that they're concentrating on expanding capabilites, not
just relating to a GUI....
Anyway it's not that I think GUIs are not useful, on the contrary I'm
saying that as it was for LADSPA, a better approach would be that a
plugin just contains processing stuff and the host/library properly
generates the GUI (LADSPA was a bit limited on this, expecially with
labels and default values, but LV2 seems a lot stronger).

> > > >>  > 3. embedding: will it be (or is it) possible (or to say better,
> > > >>  > convenient and efficient) to use CLAM in embedded applications?
> > > >>
> > > >> not sure what you mean:
> > > >
> > > > I meant the second one :-P
> > > >
> > > >> - "embed clam in applications": sure it's really straightforward.
> > > >>   you just need to instantiate a networkplayer, choose the backend,
> > > >>   load the clamnetwork file, and run the networkplayer.
> > > >
> > > > What about not forcing application developers to use clamnetwork files
> > > > and the network paradigm? Maybe they just want a robust plugin
> > > > wrapper/implementation (I'm honestly thinking about NASPRO :-P)
> > >
> > > well, you can do a network as simple as you whish.
> > > i think this is quite good:
> > >
> > > net.AddProcessing("AudioSink");
> > > net.AddProcessing("AudioSource");
> > > net.AddProcessing("VST/PhatCompressor");
> > > net.ConnectPorts("AudioSource.out", "VST/PhatCompressor.in");
> > > net.ConnectPorts("VST/PhatCompressor.out", "AudioSink.in");
> >
> > Just an idea, however that's quite nice.
>
> Not just an idea. Add on the top
>         Network net;
> and It's fuzzing working CLAM code. ;-)

What a bad English I must "speak"... the "idea" I was referring was
splitting network representation from objects. Anyway it seems pretty
easy with CLAM.
I hope that it doesn't slow down processing.

> > > >> - "clam in embedded systems" well, clam use floating point.
> > > >>   (float or double is configurable) this is a limitation in some
> > > >>   embedded systems.
> > > >
> > > > Not a big problem, I think.
> > > >
> > > >>   on the other hand clam_core do not (or not mandatory should)
> > > >>   depend on many libs if any at all. and you can add other
> > > >>   components as needed.
> > > >
> > > > That's fine. But audio backends are not part of clam_core?
> > > >
> > > >> actually, as i said, there is some work to do to make clam more
> > > >> extensible. this includes moving processings in core to other
> > > >> libs/plugins.
> > > >
> > > > That's fine too.
> > > > Here I make a proposal: I have the impression that we're going towards
> > > > the same direction but from different point of views (which does not
> > > > imply that the approach should be different).
> > > > It seems to me that what NASPRO is about is at a lower level than
> > > > CLAM, and that it could bring great enhancements to CLAM itself (maybe
> > > > I'm a bit pretentious).
> > > > So we're going to do basically the same work, and it really makes no
> > > > sense to work each on its own. CLAM is a great and successful product
> > > > and its development is very ahead of my tiny NASPRO, but with the
> > > > "wrapping" task we're almost at the same point (I've just completed
> > > > the module loading part, you have experimentally implemented LADSPA
> > > > support).
> > >
> > > i disagree here. what clam is most ahead is in the wrapping thing.
> > > the wrapper is a processing object that gets configured with a given
> > > plugin. mapping plugins properties to the PO and have a running system
> > > with a PO network is the hard thing.
> >
> > So I would say: how much is CLAM ahead in "mapping plugins properties
> > to the PO" with different APIs, since that's the fundamental part of
> > what I'm working on?
>
> Modules/Plugins: Jack, AudioUnits (through PortAudio), Ladspa, VST
> Hosts: Ladspa

AudioUnits through PortAudio? Does PortAudio do this?
But you mean at "using" level or at "generation" level?

> > > > In my opinion we could go this way: make NASPRO become part of the
> > > > CLAM project, but keep its code separated from CLAM itself (I leave to
> > > > you answers to technical issues like using  CLAM data types in NASPRO,
> > > > etc.), maintain C support in NASPRO and build C++ bindings for it to
> > > > be used in CLAM, maybe develop a C interface for CLAM, and let it
> > > > contain all network-related stuff. In other words, NASPRO could become
> > > > the "wrapping part" of CLAM, and could be used on its own when network
> > > > is not requested by the host application developer.
> > > > What do you think about this?
> > >
> > > as said in my previous mail, all "host" related code is very C++. it
> > > could be
> > > seen as a (separate) layer but it have contact with:
> > > - processing-tree and factory: to organize all instantiable processing
> > > objects
> > >   and plugins
> > > - processing-object that wraps the actual plugin.
> > >
> > > separate this in a independent module is not obvious at all for me at
> > > this stage. if there's a singe think we have learned during the 6 years
> > > of clam development is that big-up-front-design is evil. of course you
> > > are welcomed to
> > > make an api proposal for such modules, and/or send patches to go into
> > > that direction (we always pay a lot of attention to patches!).
> > > but my plan is to go the safe way of iterating "add small functionality
> > > - refactor"
> > > over and over.  and maybe at some point the code call for a "extract
> > > modules" refactoring.
> >
> > ???
>
> He comes to say that a big cute design considering all the cases always falls
> when it has to address the detail. That's something we did a lot and we
> realized that it was not the best approach. At last you end up having a big
> empty skeleton that is never filled and hinders you when you want to adapt
> new situations. Designing just for the next small feature and having the
> minimal baggage, just in case you have to move, is always a better option.
> This is why we are not wondering about APIs we are not addressing right now.

I'm going to work the same way too.
I'm not going to implement all features since the first release: my
plan is wrapping LADSPA, then moving to the "functional parts" of LV2,
DSSI and VST in order.
Anyway some abstraction must be made at the beginning, but it's not as
much as it might seem.

> Anyway, let's start codding. As we code we'll find the real problems to face.
> Now that seems much like philosophy. I confess i just have a very fuzzy idea
> of what are proposing. Code a first shot of your idea and show us the code.
> We can help you when you find an specific issue you may think we have already
> addressed. And we can eventually get profit on the steps you make on
> different paths than the one we took.

I must confess that it might sound a bit confusing, but the idea in my
mind is quite clear (that's not enough I know, but it's a start).
When the first release will be ready maybe things will get more clear.
After that I wish our two projects could have a strong
cooperation/collaboration.

Talking about Google Summer of Code, I might consider to work for CLAM
during summer, so in case you apply, just tell me! I'm not 100% sure,
but it's a big opportunity for me (apart from the 4500$ - I would earn
more then my parents :-)

Sincerely,

Stefano