audio signal processing

audio signal

audio object

See also

audio signal

audio      adjust      average
displ      dim      dynamic
envelope      end
max      min      module
normal      NP
period      play      precision      puls
read      ring
sort      speed      status      STOP
time      traj


Causes audio playback buffers, an error is returned if ini(n)audio was not done.
1) In interaction mode is not necessary to call this command (which is automatically processed before each image if the audio mode is active, ie if ini(n)audio and yes audio have been made).
2) In interpreter mode audio must be called before other commands containing audio as audio var or displ audio.

audio average

Returns the average of audio signal amplitude.
envelope: signal envelope.
normal: returned value between 0.0 and 1.0.
sort(s): fft (s=1: real, s=2: imaginare).

displ audio

See displ audio".

audio dim

Returns the number of samples of the audio signal can be modified ini(n)audio, it takes the value 2^n.

audio dynamic

audio dynamic(n)

Returns (f_min, a_min, f_max, a_max) = coordinates of the extrema of the fft
If n>0: return vectors (a1, f1, a2, f2, a3, f3, ...) obtained by the division in the order recursive n of segments (a_min, a0, a0, a_max) with a0 = (a_min a_max +) / 2 (n = 0 by default).
normal: standardized values between 0.0 and 1.0.
dynamic(n): returne 2*(n+1) vectors (n=0 by default).
envelope: for the enveloppe.
sort: for fft.
var: for the signal.

audio end

audio end

Close audio.

audio envelope

audio var envelope(p)

Returns the envelope of the audio signal with precision p (6 by default).
audio(1): returns the derivative of the signal (speed).
generate: generates the envelope.
sort: processes the fft of the signal.
yes envelope must be enabled for the envelope is calculated.

audio limit

audio limit

Returns the low pass filters and high-pass of the amplitude of the audio signal (initialized 0,1 ie without filter).

audio limit(d1,d2)

Change those filters.

audio read

num=audio read name("nn.wav")
        Read file nn.wav in the buffer number num (between 0 and 255).
directory("dir"): from directory dir.
format("fff"): extension fff.
number(n): file number n of directory dir.

See also
alphabet audio directory("dir").
name audio directory("dir").
number audio directory("dir").

num=read audio("ima/0.wav ima/1.wav ima/2.wav");
returns num = 0,1,2.
audio play(num);
Plays together sounds ima/0.wav ima/1.wav ima/2.wav.
num=audio read name("0 1 2")directory("ima")format("wav")

num=audio read directory("ima")format("wav");
is equivalent to:
num=audio read name("*.wav")directory("ima");

num=audio read directory("ima")number(11,1);
files number 11 and 1 of directory ima.

do not confuse audio read with read audio.

audio play(num)

to play the buffer num.

audio play(num) period

to play the loop buffer num.
Can play multiple sounds simultaneously.

audio STOP

To stop .

audio ini abs

To kill buffers.

audio min

audio min

Returns the minimum amplitude of the audio calculated on NP audio images.

audio module

audio module

Returns the audio amplitude average calculated on NP audio images.

audio normal

audio normal

Returns 1 if the audio signal is normalized (between 0.0 and 1.0), otherwise returns 0. It will be necessary, to show the standard curves (audio, fate and envelope), to dilate eg 100 (option coe(100) of displ audio).

audio normal(n)

< -----------------> Change this flag.

audio NP

audio NP

Returns the number of images to calculate the extrema and the module (default is 1).

audio NP(n)

Calculation of extrema and the module on n images (1 by default).

audio period

audio period(p,m)

Returns the standard period (between 0.0 and 1.0) of the audio signal p samples (default 128, p <= 16384), with an approximation m (0 by default).

audio period(p) var

Returns the list of p values accumulated by audio precision (128 default, p<=16384). audio precision (default 128, p <= 16384).
Note: You must have done audio precision before.

audio play

audio play(n)

Plays buffer number n.

audio play(n)period(p)

Plays the buffer number n on p images periodically (indefinitely default)
You must call audio play period each image to decrement p.

audio play period

Times decrements and returns the number of buffers stopped.

audio play("name.wav""))

Plays file name.wav.

audio precision

audio precision

Returns the precision of the envelope of the audio signal.

audio precision=p

Change this precision.

audio sort precision

Returns the precision of the envelope of the fft of the audio signal.

audio sort precision=p

Changes this precision.
        1) p<=1: the envelope is identic to the signal.
        2) more p is large, more the envelope is flexible.
        3) The precisions are set to 6 by default.

audio ring

audio ring("name.wav")

Plays the file name.wav

audio ring("name.wav")period

Play the file name.wav periodically.

audio ring(f,t)

Plays the sound frequency sine f during t ms synchronously.Audible frequencies: f = 37 (severe) f = 9000 (high).
To play multiple sounds simultaneously see audio read.

audio ring law ext

Returns the extension e,c,f p.

audio ring law law

Returns the law L.

audio ring law period

Returns the period p.

audio ring law rem


audio ring law ring

Returns the coefficient freq.

audio ring law validate(v)

        v>0: enables (default).         v<=0: disables.

audio ring law(L)

Plays the sound frequency=law(x,L) where L=0,y1,0,x2,y2,0,...,1,yn,0) and x varies between 0 and 1 during 25 images.
ext(e,c,f): with extension e,c,f.
period(p): during p images (25 default).
ring(freq): freq * frequency (1000 default).
audio ring(1000)law(0,.1,0, .5,1.5,0, 1,.1,0); plays a sound of frequency from 100 Hz, to 1500 Hz and then to 1OO Hz.
Note: requeries the interaction mode.

audio sort generate

Causes calculating the fft audio signal, must be done before each call to a command containing audio sort (as audio sort var).
audio(1): returns the derivative of the signal (speed).
var sort: returns fft.
var sort(1): return the real part.
var sort(2): return the imaginary part.
envelope(p): return the envelope with precision p) of the fft of the audio signal.
limit: returns the low pass frequency and high-pass frequency filters (initialized 0,1 ie without filter).
limit (d1, d2): change these filters.
1) yes sort must be enabled for the fft is calculated.
2) in interaction mode it is not needed to call this command (which is called automatically before each image if the audio mode is active ie if ini(n)audio has been done.

audio speed

When present processes with the speed of the signal.

audio speed var

Returns the speed of the signal.

audio speed sort var

Returns the speed of the fft.

audio speed time

Returns the computation delay of the speed.

audio speed time(t)

Changes this delay.
The speed is computed one frame on t frames.

audio status

Returns 1 if audio ON (see audio ini, else returns 0.

audio STOP

audio STOP

Stops all buffers

audio STOP(n)

Stops buffer n.

audio time

audio time(v,dv,n)

Analysis n times the audio buffer by detecting the number of extrema above v (dv = approximation) default dv = 0 and n = 1.

audio traj

audio traj t vol(id)

Returns the property audio of trajectory type t of volume id.

audio(c)traj t vol(id)

Changes this property.
When audio has been initialized by audio ini, the trajectory type t of volume id is modulated by c * (audio max).

audio traj

audio traj t vol(id)

Returns audio signal (between -127 and 128)
above(m): returns the signal reduced to its component s > m.
adjust(a): adjusts the signal s according to the law s=s+a*s*sin(i*dan), where n is its size and i in [0,n-1].
audio(1): returns the derivative of the signal (speed).
exp(e): returns signal^e.
max: returns the signal reduced to its maximal component.
sort: processes the fft of the signal.
speed: processes the speed of the signal (according to audio speed time).
        It must have been audio ini(n). The returned vector is of dimension 2^n. vol

audio vol

Associates a wav file to the transformations of a volume.
first build volumes with properties audio transf vol(id)dim(n).
Then associate trajectories: traj(ni)transf(...)vol(id)....
Finally launch the interactive read time interaction func (5,"F5"))
with function F5 containing exec audio vol;
exec audio is executed every image.
time audio: allows controling this execution.
write audio vol: saves the resulting audios.

vol audio
vol func
vol time
vol transf
vol transf acc
vol transf speed
write audio vol

audio audio vol(id)dim(np,dimp)

Creates property audio audio of volume id with np size dimp values audio.

audio(num)audio vol(id)

Returns vector number num (between 1 and np).

audio(num)audio vol(id)=x1,x2,...,xdimp

Change this vector.
x1=frequency, x1,x2,...,xdimp=a1,a2,...=amplitude defining a musical note.
When using write audio vol those notes are mixed with others sounds.

audio func("ff")vol(id)dim(n)

Creates property audio func of volume id with n values, function ff executed each image. In such a function the number of the volume currently processed may be retrivied by num=number vol displ;, the index of the current value may be retrivied by t=time audio vol;, and thus changed by audio(t)func vol(num)=v;. See func func_MEN_VOL_FUNC() of file demo1_audio.func.

audio time(t)func vol(id)

Returns the value index t (1 <= ind <= n).

audio time(t)vol(id)=v

Changes this value.

audio time vol(id)dim(n)

Creates property audio of volume id with n values.

audio time(ind)vol(id)

Returns the value index ind (1 <= ind <= n).

audio time(ind)vol(id)=v

Changes this value.
when using write audio vol the audio property of volume id acts as any audio transf. Allows producing specific sounds at given times.
t=time audio vol;audio time(t,t+6,t+25)vol(pelvis)=0,1,0;
produces a guitar sound.

audio transf vol(id)dim(n)

Creates property audio transf of volume id with n values transf= rota tran, audio.
exec audio vol must be done every image.
dim(n,3): np vectors size 3 = (value,speed,acc), interpreted as frequency, module and phase.

audio time(t)transf vol(id)

Returns the value index t, with t= time audio vol(id)
audio(t)transf vol(id)=...; allows compute this value each image.

audio time(t)transf vol(id)=v

Changes this value.

audio acc transf vol(id)dim(n)

Creates property audio acc transf of volume id with n values acc transf.

audio speed transf vol(id)dim(n)

Creates property audio speed transf of volume id with n values speed transf.
audio time(t)transf acc vol(id)
audio time(t)transf speed vol(id)
exec audio vol must be done every image.

See also

coe module vol(id)
coe transf vol(id)
meta transf vol(id)

exec audio vol

write audio vol

For writing the result.

time audio vol


Can be written at one time:
audio vol(id)acc speed rota tran coe(c)dim(np);
audio vol(id)dim(250);coe audio vol(1)=1.5;
can be written:
audio vol(id)coe(1.5)dim(250);
Other example:
audio vol(id)acc tran coe(.75)smooth(12)dim(125);

audio object

audio audio       audio audio add       audio audio time       audio audio audio time
depth       dim audio(id)       dynamic
memory       move(id)       music
period       puls
sin       sin interpol       sin law       sin sin       smooth       stock
time       time module       time sin       time sin add       time sin abs      transf
var      vol


Command audio(id)time(t) allows build an object type audio duration t (seconds). Options allow specify some proprties.

audio audio


Does: audio(id2) = copy of audio(id1).
acc: does: audio(id2) = acceleration of audio(id1).
add: does audio(id2),=audio(id1).
coe(c): amplitude multiplier.
speed: does: audio(id2) = speed of audio(id1).


Mixes audios id1,id2,... in audio id0: id0=sum(audio(idi)).
average: id0=sum(audio(idi))/n: avoids frequencies overflow.
coe(c1,c2,...): id0=sum(ci*audio(idi)).
interpol: interpolating.


Concatenes audios id1,id2,...,idn in audio id0.
add(s): mixing (idi,idi+1) on s seconds (for i in [1,n-1[).

audio audio time


Adds segment[t1,t2,...,tn] from audio id1 at address t0 of audio id2.
coe(c1,c2,...,cn): modulating the add with coiefficients (c1,c2,...,cn).
ext(e,c,f): with extension (e,c,f).
write("nnn.wav"): writes the result in file nnn.wav.
audio(1)time(1,1.5,2)audio(2)time(2)coe(0,1,0)ext(e,c,0); write audio(2)name("A.wav");system("A.wav"); screen;displ audio(2)poi(-400,0,400,100)dila(1);

audio audio audio time


Builds audio id3 = audio(id1)[0,t[ + mix(audio(1)[t1,t1[,audio(id2[0,t1-t[) + audio(id2[end1-t,t2]): crossfade betweed the 2 audios id1 and id2, with options.
t1=time(id1), t2=time(id2).

audio dynamic

As any others objects of anyflo an audio id can be provided with dynamic properties:
mass(m): m = mass
spring(stif,visc): stif = stiffness, visc = viscosity.

audio image

audio image
image transf
image win
image speed win
image acc win
image win network
write audio image

audio image(id)

        Returns the property audio of image id.

audio image(id)=v

        Assignes v to the property audio of the image id.
v=a1,a2,...: the buffers numbers will be played according to the speed of the correspondant pixel of the image id
v=-freq: a ring of frequence freq * spe will be played (where spe is the module of the average speed of the image id).
The duration of the sound is specified by period image(id).
It is necessary to make yes dynamic.

audio image transf

As audio vol associates a wav file to the transformations of an image.
coe audio image(id)=c: multiplier of the frequency.

audio image(id)transf dim(ni)

audio win(x1,y1,x2,y2)image(id)dim(n)

Creates property audio win of image id with n values.
Each time the image id is changed the correspondant value is set to the average luminance of window(x1,y1,x2,y2) of the image id (x1,y1,x2,y2 in [0,1]).

audio win(x1,1,y1,1,x1,2,y1,2, x2,1,y2,1,x2,2,y2,2, ..., xnw,1,ynw,1,xnw,2,ynw,2) image(id)dim(n)

Creates nw properties audio win of image id with n values.
Each time the image id is changed the correspondant value is set to the average luminance of window (xi,1,yi,1,xi,2,yi,2) of the image id.

audio win(nb)image(id)dim(n)

Creats an audio win(xi,j,yi,j,xi,j,yi,j) image(id)dim(n) with nb*nb sub windows (i,j) in [1,nb]*[1,nb] and n values creating a specific audio. When write audio is invoked the sum of audios [1,nb*nb] is computed in audio nb+1 which is saved.
speed: using speed transformation of the image.


Returns the value n1 of property win n2 of image id.

write audio image saves the resulting audios.

when write audio image(id)name(quot;...")speed is invoked (with option speed), the speed variation of the image is used as coefficient of the wave form.

audio speed win(x1,y1,x2,y2)image(id)dim(n)

Creates property audio speed win of image id with n values.
Each time the image id is changed the correspondant value is set to the average speed of luminance of window(x1,y1,x2,y2) of the image id (x1,y1,x2,y2 in [0,1]) with same syntax as image win.

audio acc win(x1,y1,x2,y2)image(id)dim(n)

Creates property audio speed win of image id with n values.
Each time the image id is changed the correspondant value is set to the average acceleration of luminance of window(x1,y1,x2,y2) of the image id (x1,y1,x2,y2 in [0,1]) with same syntax as image win.

audio win network(num)image(id)

Creats an audio win(xi,j,yi,j,xi,j,yi,j) network(num)image(id)dim(n) with nb*nb sub windows (i,j) in [1,nb]*[1,nb] and n values associated with network number num.
Each time the image id is changed the correspondant value is set to the validation of neural num for the motif which is the luminances of sub windows.


Returns the value number n of window w of image id associated at network n.


Changes this value to v.

audio memory


builds signal from memory id2
dim(n): resized to n.
format: only x coordinates of memory points.

audio smooth

audio smooth transf vol(id)

Returns the audio smooth transf property of volume id, with transf = rot, tran.

audio smooth transf vol(id)=n

Changes this property.

audio smooth derivative transf vol(id)

Returns the audio smooth dedrivate transf property of vol id, with derivate = acc, speed.

audio smooth derivative transf vol(id)=n

Changes this property.

audio stock


Returns the vector ind of property stock of audio id.


generates and writes the property stock of audio id in file nnn.stock (i.e. the averages of frequencies and amplitudes each 1/25 second).
See alos stock audio(id), displ audio(id) and write stock audio.


Music synthesis from images

How to generate a musical note

First generate a sinusoidal waveform, for example:
Then change the waveform, for example:

Music synthesis from images

First transform an interactive process capturing sensors (such as audio signal, motion captures, ...) in a tmp.mem file using SENSOR_WAV function of
sensor.func tool, then:

First method: dynaphone (from a dynamic model):
build a dynamic scene (objects equipped with dynamic properties) associated with SIMUL sensor (reading the tmp.mem file) by commands audio transf vol.
activate yes dynamic
start interaction func(5,"F5")
each image do exec audio vol(id) and, at the same time, record images using interaction write image.
At the end of the processus write the result with write audio vol in a file tmp.wav.

Second method: imaphone (from an animated image):
Create an image(id) with property audio image(id)transf
activate yes dynamic
start interaction func(5,"F5")
each image read a recorded image namexxx.eee, do exec audio image(id)
At the end of this processus write the result with write audio image in a file tmp.wav.

It is enough to mount tmp.wav with the generated images.

audio period


Repeats the signal between times [t1,t2] on [t3,t4] (0,1 default) (normalized) of audio object id.

audio rand


signal randomly initialized with coefficient c (0.1 default) on t seconds (1 default).

audio puls


signal initialized with a sine frequency df: sin(f);f+=df;
puls(f1,df1,f2,df2,...): df1=sin(f2);f2+=df2;...
coe(c): c*f; (default 0.1).

audio read


signal initialized with the audio file nn.wav.

audio spe

audio(num)set module

Returns audio volume of directx audio buffer number num.

audio(num)set module(v)

Changes this volume.

audio sin


f1=a*sin(f*t+p) with t=duration.
dist(d): d = time increment (1 default).
rand(r1,r2,dn): random in (r1,r2)dn = amplitude of variation in (0,1).
sin(fi,pi,ai)module(mi): interpolating modules.


inserts, on an existing audio(id), signal: a*sin(f*tt+p) with tt in [t1,t2].
coe(c): wave form coefficient c normalized between 0.0 and 1.0 (attack).

audio(id)time(t)sin(f1,p1,a1, f2,p2,a2, ...,fn,pn,an)

signal=interpolation(fi,ai,pi) on t seconds.
adjust(n)ext(e,c,f): extending the 3D curve (fi,ai,pi) to n points.

audio(id)time(t)sin(f1,p1,a1, f2,p2,a2, ...,fn,pn,an)add

signal=sum(a1*sin(fi*tt+pi)) on t seconds.

audio(id)time(ti,)sin(fi, ai,pi)abs

inserts, on an existing audio(id), signal: a*sin(f*tt+p) with tt in [t1,t2].
coe(c1,c2): signal = c1 * (old signal) + c2 * signal (0,1 default).
ext(e,c,f): extension.

audio sin law

audio(id)time(t)sin(f0,a0,p0) law(y1,y2,...,yn)

Frequency fluctuates around f0 as law(0,y1,0, 1/n,y2,0, ...,1,yn,0).
dist(d1,d2,...): time increments (0.01 default).
module(m1,m2,...): modules multiplied by mi (1 default).


Same with an accelerated law.


Same with a decelerated law.


Same with a poendulum law.

audio sin sin

audio(id)time(t)sin(f0,a0,p0) sin(f1,a1,p1, f2,a2,p2,...)

Frequency fluctuates around f0 as:
dist(t0,t1,...): increments of times (1,.1,.01,... default).
exp(e): sin ^ e.

audio sin interpol

audio(id)time(t)sin(f1,p1,a1, f2,p2,a2, ...)interpol(0,t1,t2,...,1)

       frequencies between fi and fi+1
       phases between pi and pi+1
       amplitudes between ai and ai+1
at time ti (normalized between 0.0 and 1.0).
limit(L): if L is a 3D line (x=frequency).
max(m): frequency maximum (20000 default).

audio time


Builds an empty signal duration s seconds.
       c = number of channels (1 default).
       s = datas size: 8 (char), 16 (short) (default).

audio time module


Changes the waveform of an existing audio object id by seting the amplitude to mi at times ti (in [0,1]).
acc: accelerated variation.
coe(c): module ^= c, increases (c<1) or decreases (c>1) the variations.
dec: decelerated variation.
ext(e,c,f): extension.
law(x1,y1,0, x2,y2,0, ...): according to a law.
pend: pendulum variation.
       fade in.
       fade out on second part.

audio transf


        Does module(audio(id))*=d.


        Does audio(id) translated t (seconds).

audio var

audio(id)time(t)var sin(f1,a1,p1, a2,a2,p2)dist(df,da,dp)

Frequency varies in [f1,f2] with f1 and f2 in [0,1] and freqency df in [0,1] (1 default).
Amplitude varies in [a1,a2] with a1 and a2 in [0,1], and freqeency fa in [0,1] (1 default).
Phase varies in [p1,p2] with p1 and p2 in [-PI,PI] and freqeency fp in [0,1] (1 default).


Creates audio id time t initialized at v.
w=calloc(44100)limit(32767);audio(1)time(1)var(w);play audio(1);
play the same sound as audio(1)time(1)sin(.22,.5); audio(1)time(1)sin(.22,.5);play audio(1);


Creates audio id time t with rectangular signal period dt between -v and v.
var(v1,v2): between v1 and v2.

audio var

        Returns the audio (between -127 and 128)
above(m): signal reduced to its components >: m.
adjust(a): ajuste the audio signal s according to the law s=s+a*s*sin(i*dan), where n is the size i in [0,n-1].
audio(1): retutns the derivative of the signal audio (speed).
exp(e): returns signal^e.
max: signal reduced to its maximum components.
sort: processing the fft (sort(0.5,1,2,3): half real, real, imaginary, both).
speed: processing the signal speed (according to audio speed time).
        Must be done audio ini(n). The size of the returned vector is 2^n.

audio write


writes signal in audio file nn.wav and property stock in file nn.stock.

write stock audio(id)name("nn")

writes only property stock in file nn.stock and can be recovered by read("nn.stock") format("anyflo"). writes signal in audio file nn.wav and property stock in file nn.stock.



audio(1)read("0.wav"); /* read file 0.wav in object type audio */
generate stock audio(1); /* generate property stock */
write stock audio(1)name("nnn"); /* save stock in nnn.stock */

See also:

alphabet audio directory.
clean audio        coe audio        coe transf image        coe transf vol        cut audio
depth audio        dila audio(id)        dim audio(id)        displ audio        displ audio(id)
edit audio        exec audio vol
ini audio        inv audio
limit audio(id)
max module audio(id)        move audio(id)
no audio
name audio directory.        NP audio        NP audio win        number audio directory.
play audio        poi audio
rand audio
smooth audio        stock audio
time audio        time audio vol        tran audio(id)
var audio        vertex audio
write audio        write audio image        write audio vol        write audio stock
yes audio