audio signal processing
audio
Causes audio playback buffers, an error is returned if
ini(n)audio was not done.
Comments:
1) In interaction mode is not necessary to call this command (which is automatically
processed before each image if the audio mode is active, ie if
ini(n)audio and yes audio have been made).
2) In interpreter mode audio must be called before other commands containing audio as
audio var or displ audio.
audio average
Returns the average of audio signal amplitude.
Options:
envelope: signal envelope.
normal: returned value between 0.0 and 1.0.
sort(s): fft (s=1: real, s=2: imaginare).
displ audio
See displ audio".
audio dim
Returns the number of samples of the audio signal can be modified
ini(n)audio, it takes the value 2^n.
audio dynamic
audio dynamic(n)
Returns (f_min, a_min, f_max, a_max) = coordinates of the extrema of the fft
If n>0: return vectors (a1, f1, a2, f2, a3, f3, ...) obtained by the division in the order recursive n of segments (a_min, a0, a0, a_max)
with a0 = (a_min a_max +) / 2 (n = 0 by default).
Options:
normal: standardized values between 0.0 and 1.0.
dynamic(n): returne 2*(n+1) vectors (n=0 by default).
envelope: for the enveloppe.
sort: for fft.
var: for the signal.
audio end
audio end
Close audio.
audio envelope
audio var envelope(p)
Returns the envelope of the audio signal with precision p (6 by default).
Options:
audio(1): returns the derivative of the signal (speed).
generate: generates the envelope.
sort: processes the fft of the signal.
Note:
yes envelope must be enabled for the envelope is calculated.
audio limit
audio limit
Returns the low pass filters and high-pass of the amplitude of the audio signal
(initialized 0,1 ie without filter).
audio limit(d1,d2)
Change those filters.
audio read
num=audio read name("nn.wav")
Read file nn.wav in the buffer number num (between 0 and 255).
Options:
directory("dir"): from directory dir.
format("fff"): extension fff.
number(n): file number n of directory dir.
See also
alphabet audio directory("dir").
name audio directory("dir").
number audio directory("dir").
Examples:
num=read audio("ima/0.wav ima/1.wav ima/2.wav");
returns num = 0,1,2.
audio play(num);
Plays together sounds ima/0.wav ima/1.wav ima/2.wav.
or:
num=audio read name("0 1 2")directory("ima")format("wav")
num=audio read directory("ima")format("wav");
is equivalent to:
num=audio read name("*.wav")directory("ima");
num=audio read directory("ima")number(11,1);
files number 11 and 1 of directory ima.
WARNING:
do not confuse audio read with read audio.
to play the buffer num.
audio play(num) period
to play the loop buffer num.
Can play multiple sounds simultaneously.
To stop
.
To kill buffers.
audio min
audio min
Returns the minimum amplitude of the audio calculated on NP audio images.
audio module
audio module
Returns the audio amplitude average calculated on NP audio images.
audio normal
audio normal
Returns 1 if the audio signal is normalized (between 0.0 and 1.0), otherwise returns 0.
It will be necessary, to show the standard curves (audio, fate and envelope), to dilate eg 100
(option coe(100) of displ audio).
audio normal(n)
< ----------------->
Change this flag.
audio NP
audio NP
Returns the number of images to calculate the extrema and the module (default is 1).
audio NP(n)
Calculation of extrema and the module on n images (1 by default).
audio period
audio period(p,m)
Returns the standard period (between 0.0 and 1.0) of the audio signal p samples (default 128, p <= 16384), with an approximation m (0 by default).
audio period(p) var
Returns the list of p values accumulated by audio precision (128 default, p<=16384).
audio precision (default 128, p <= 16384).
Note: You must have done audio precision before.
audio play
audio play(n)
Plays buffer number n.
audio play(n)period(p)
Plays the buffer number n on p images periodically (indefinitely default)
You must call audio play period each image to decrement p.
audio play period
Times decrements and returns the number of buffers stopped.
audio play("name.wav""))
Plays file name.wav.
audio precision
audio precision
Returns the precision of the envelope of the audio signal.
audio precision=p
Change this precision.
audio sort precision
Returns the precision of the envelope of the fft of the audio signal.
audio sort precision=p
Changes this precision.
Notes:
1) p<=1: the envelope is identic to the signal.
2) more p is large, more the envelope is flexible.
3) The precisions are set to 6 by default.
audio ring
audio ring("name.wav")
Plays the file name.wav
audio ring("name.wav")period
Play the file name.wav periodically.
audio ring(f,t)
Plays the sound frequency sine f during t ms synchronously.Audible frequencies: f = 37 (severe) f = 9000 (high).
To play multiple sounds simultaneously see audio read.
audio ring law ext
Returns the extension e,c,f p.
audio ring law law
Returns the law L.
audio ring law period
Returns the period p.
audio ring law rem
remove.
audio ring law ring
Returns the coefficient freq.
audio ring law validate(v)
v>0: enables (default).
v<=0: disables.
audio ring law(L)
Plays the sound frequency=law(x,L) where L=0,y1,0,x2,y2,0,...,1,yn,0)
and x varies between 0 and 1 during 25 images.
Options:
ext(e,c,f): with extension e,c,f.
period(p): during p images (25 default).
ring(freq): freq * frequency (1000 default).
Example:
audio ring(1000)law(0,.1,0, .5,1.5,0, 1,.1,0); plays a sound of frequency from 100 Hz, to
1500 Hz and then to 1OO Hz.
Note: requeries the interaction mode.
audio sort generate
Causes calculating the fft audio signal, must be done before each call to a command containing audio sort
(as audio sort var).
Options:
audio(1): returns the derivative of the signal (speed).
var sort: returns fft.
var sort(1): return the real part.
var sort(2): return the imaginary part.
envelope(p): return the envelope with precision p) of the fft of the audio signal.
limit: returns the low pass frequency and high-pass frequency filters
(initialized 0,1 ie without filter).
limit (d1, d2): change these filters.
Notes:
1) yes sort must be enabled for the fft is calculated.
2) in interaction mode it is not needed to call this command (which is called automatically before each image if the audio mode is active
ie if ini(n)audio has been done.
audio speed
When present processes with the speed of the signal.
audio speed var
Returns the speed of the signal.
audio speed sort var
Returns the speed of the fft.
audio speed time
Returns the computation delay of the speed.
audio speed time(t)
Changes this delay.
Note:
The speed is computed one frame on t frames.
audio status
Returns 1 if audio ON (see audio ini,
else returns 0.
audio STOP
audio STOP
Stops all buffers
audio STOP(n)
Stops buffer n.
audio time
audio time(v,dv,n)
Analysis n times the audio buffer by detecting the number of extrema above v (dv = approximation) default dv = 0 and n = 1.
audio traj
audio traj t vol(id)
Returns the property audio of trajectory type t of volume id.
audio(c)traj t vol(id)
Changes this property.
Note:
When audio has been initialized by audio ini, the trajectory type t
of volume id is modulated by c * (audio max).
audio traj
audio traj t vol(id)
Returns audio signal (between -127 and 128)
Options:
above(m): returns the signal reduced to its component s > m.
adjust(a): adjusts the signal s according to the law s=s+a*s*sin(i*dan),
where n is its size and i in [0,n-1].
audio(1): returns the derivative of the signal (speed).
exp(e): returns signal^e.
max: returns the signal reduced to its maximal component.
sort: processes the fft of the signal.
speed: processes the speed of the signal (according to audio speed time).
Note:
It must have been audio ini(n). The returned vector is of
dimension 2^n.
vol
audio vol
Associates a wav file to the transformations of a volume.
Typically:
first build volumes with properties audio transf vol(id)dim(n).
Then associate trajectories: traj(ni)transf(...)vol(id)....
Finally launch the interactive read time interaction func (5,"F5"))
with function F5 containing exec audio vol;
exec audio is executed every image.
Notes:
time audio: allows controling this execution.
write audio vol: saves the resulting audios.
audio audio vol(id)dim(np,dimp)
Creates property audio audio of volume id with np size dimp
values audio.
audio(num)audio vol(id)
Returns vector number num (between 1 and np).
audio(num)audio vol(id)=x1,x2,...,xdimp
Change this vector.
Note:
x1=frequency, x1,x2,...,xdimp=a1,a2,...=amplitude defining a musical note.
When using write audio vol those notes are mixed
with others sounds.
audio func("ff")vol(id)dim(n)
Creates property audio func of volume id with n
values, function ff executed each image. In such a function the number of the
volume currently processed may be retrivied by num=number vol displ;,
the index of the current value may be retrivied by t=time audio vol;,
and thus changed by audio(t)func vol(num)=v;.
See func func_MEN_VOL_FUNC() of file demo1_audio.func.
audio time(t)func vol(id)
Returns the value index t (1 <= ind <= n).
audio time(t)vol(id)=v
Changes this value.
audio time vol(id)dim(n)
Creates property audio of volume id with n values.
audio time(ind)vol(id)
Returns the value index ind (1 <= ind <= n).
audio time(ind)vol(id)=v
Changes this value.
Note
when using write audio vol the audio
property of volume id acts as any
audio transf. Allows producing specific
sounds at given times.
Example:
t=time audio vol;audio time(t,t+6,t+25)vol(pelvis)=0,1,0;
produces a guitar sound.
audio transf vol(id)dim(n)
Creates property audio transf of volume id with n
values transf= rota tran, audio.
exec audio vol must be done every image.
Options:
dim(n,3): np vectors size 3 = (value,speed,acc), interpreted as frequency, module
and phase.
audio time(t)transf vol(id)
Returns the value index t, with t= time audio vol(id)
audio(t)transf vol(id)=...; allows compute this value each image.
audio time(t)transf vol(id)=v
Changes this value.
audio acc transf vol(id)dim(n)
Creates property audio acc transf of volume id with
n values acc transf.
audio speed transf vol(id)dim(n)
Creates property audio speed transf of volume id with
n values speed transf.
Note:
audio time(t)transf acc vol(id)
audio time(t)transf speed vol(id)
exec audio vol must be done every image.
See also
coe module vol(id)
coe transf vol(id)
meta transf vol(id)
For writing the result.
Note
Can be written at one time:
audio vol(id)acc speed rota tran coe(c)dim(np);
Example:
audio vol(id)dim(250);coe audio vol(1)=1.5;
can be written:
audio vol(id)coe(1.5)dim(250);
Other example:
audio vol(id)acc tran coe(.75)smooth(12)dim(125);
Principle
Command audio(id)time(t) allows build an object type audio duration t
(seconds). Options allow specify some proprties.
audio audio
audio(id1)audio(id2)
Does: audio(id2) = copy of audio(id1).
Options:
acc: does: audio(id2) = acceleration of audio(id1).
add: does audio(id2),=audio(id1).
coe(c): amplitude multiplier.
speed: does: audio(id2) = speed of audio(id1).
audio(id1,id2,...,idn)audio(id0)
Mixes audios id1,id2,... in audio id0:
id0=sum(audio(idi)).
Options:
average: id0=sum(audio(idi))/n: avoids frequencies overflow.
coe(c1,c2,...): id0=sum(ci*audio(idi)).
interpol: interpolating.
audio(id1,id2,...,idn)audio(id0)add
Concatenes audios id1,id2,...,idn in audio id0.
Options:
add(s): mixing (idi,idi+1) on s seconds (for i in [1,n-1[).
audio audio time
audio(id1)time(t1,t2,...,tn)audio(id2)time(t0)
Adds segment[t1,t2,...,tn] from audio id1 at address t0
of audio id2.
Options:
coe(c1,c2,...,cn): modulating the add with coiefficients (c1,c2,...,cn).
ext(e,c,f): with extension (e,c,f).
write("nnn.wav"): writes the result in file nnn.wav.
Example:
audio(1)time(4)sin(.2,1,0);
audio(2)time(4)sin(0,0,0);
audio(1)time(1,1.5,2)audio(2)time(2)coe(0,1,0)ext(e,c,0);
write audio(2)name("A.wav");system("A.wav");
screen;displ audio(2)poi(-400,0,400,100)dila(1);
audio audio audio time
audio(id1)audio(id2)audio(id3)time(t)
Builds audio id3 = audio(id1)[0,t[ + mix(audio(1)[t1,t1[,audio(id2[0,t1-t[) + audio(id2[end1-t,t2]):
crossfade betweed the 2 audios id1 and id2, with
options.
t1=time(id1), t2=time(id2).
audio dynamic
As any others objects of anyflo an audio id can be provided with dynamic
properties:
mass(m): m = mass
spring(stif,visc): stif = stiffness, visc = viscosity.
audio image
audio image(id)
Returns the property audio of image id.
audio image(id)=v
Assignes v to the property audio of the image id.
Notes:
v=a1,a2,...: the buffers numbers will be played according to the speed of the correspondant pixel
of the image id
v=-freq: a ring of frequence freq * spe will be played (where
spe is the module of the average speed of the image id).
The duration of the sound is specified by period
image(id).
It is necessary to make yes dynamic.
audio image transf
As audio vol associates a wav file to the transformations of an image.
coe audio image(id)=c: multiplier of the frequency.
audio image(id)transf dim(ni)
audio win(x1,y1,x2,y2)image(id)dim(n)
Creates property audio win of image id with n values.
Each time the image id is changed the correspondant value is set to the
average luminance of window(x1,y1,x2,y2) of the image id (x1,y1,x2,y2 in [0,1]).
audio win(x1,1,y1,1,x1,2,y1,2,
x2,1,y2,1,x2,2,y2,2,
...,
xnw,1,ynw,1,xnw,2,ynw,2)
image(id)dim(n)
Creates nw properties audio win of image id with n values.
Each time the image id is changed the correspondant value is set to the
average luminance of window (xi,1,yi,1,xi,2,yi,2)
of the image id.
audio win(nb)image(id)dim(n)
Creats an audio win(xi,j,yi,j,xi,j,yi,j)
image(id)dim(n) with nb*nb sub windows (i,j) in [1,nb]*[1,nb]
and n values creating a specific audio. When
write audio is invoked the sum of audios
[1,nb*nb] is computed in audio nb+1 which is saved.
Options:
speed: using speed transformation of the image.
audio(n1)win(n2)image(id)
Returns the value n1 of property win n2 of image id.
write audio image saves the resulting audios.
Note:
when write audio image(id)name(quot;...")speed
is invoked (with option speed), the speed variation of the image is used as coefficient
of the wave form.
audio speed win(x1,y1,x2,y2)image(id)dim(n)
Creates property audio speed win of image id with n values.
Each time the image id is changed the correspondant value is set to the
average speed of luminance of window(x1,y1,x2,y2) of the image id
(x1,y1,x2,y2 in [0,1]) with same syntax as image win.
audio acc win(x1,y1,x2,y2)image(id)dim(n)
Creates property audio speed win of image id with n values.
Each time the image id is changed the correspondant value is set to the
average acceleration of luminance of window(x1,y1,x2,y2) of the image id
(x1,y1,x2,y2 in [0,1]) with same syntax as image win.
audio win network(num)image(id)
Creats an audio win(xi,j,yi,j,xi,j,yi,j)
network(num)image(id)dim(n) with nb*nb sub windows (i,j)
in [1,nb]*[1,nb] and n values associated with network
number num.
Note:
Each time the image id is changed the correspondant value is set to the
validation of neural num for the motif which is the luminances of sub windows.
audio(n)win(w)network(n)image(id)
Returns the value number n of window w of image id
associated at network n.
audio(n)win(w)network(n)image(id)=v
Changes this value to v.
audio memory
audio(id)memory(id2)
builds signal from memory id2
Options:
dim(n): resized to n.
format: only x coordinates of memory points.
audio smooth
audio smooth transf vol(id)
Returns the audio smooth transf property of volume id, with
transf = rot, tran.
audio smooth transf vol(id)=n
Changes this property.
audio smooth derivative transf vol(id)
Returns the audio smooth dedrivate transf property of vol id, with
derivate = acc, speed.
audio smooth derivative transf vol(id)=n
Changes this property.
audio stock
audio(id)stock(ind)
Returns the vector ind of property stock of audio id.
audio(id)time(t)...stock("nnn")
generates and writes the property stock of audio id
in file nnn.stock (i.e. the averages of frequencies and amplitudes
each 1/25 second).
See alos stock audio(id),
displ audio(id) and
write stock audio.
Music
How to generate a musical note
First generate a sinusoidal waveform, for example:
audio(1)time(4)sin(.5,1,0);
Then change the waveform, for example:
audio(1)time(0,.1,.2,1)module(0,1,.2,0);
Music synthesis from images
First transform an interactive process capturing sensors (such as
audio signal, motion captures, ...) in a tmp.mem file using
SENSOR_WAV function of sensor.func tool, then:
First method: dynaphone (from a dynamic model):
build a dynamic scene (objects equipped with
dynamic properties) associated with
SIMUL sensor (reading the tmp.mem file) by commands
audio transf vol.
activate yes dynamic
start interaction func(5,"F5")
each image do exec audio vol(id)
and, at the same time, record images using
interaction write image.
At the end of the processus write the result with write audio vol in a file
tmp.wav.
Second method: imaphone (from an animated image):
Create an image(id) with property
audio image(id)transf
activate yes dynamic
start interaction func(5,"F5")
each image read a recorded image namexxx.eee,
do exec audio image(id)
At the end of this processus
write the result with write audio image in a file
tmp.wav.
It is enough to mount tmp.wav with the generated images.
audio period
audio(id)period(t1,t2,t3,t4)
Repeats the signal between times [t1,t2] on [t3,t4] (0,1 default) (normalized) of audio
object id.
audio rand
audio(id)time(t)rand(c)
signal randomly initialized with coefficient c (0.1 default)
on t seconds (1 default).
audio puls
audio(id)puls(f,df)
signal initialized with a sine frequency df: sin(f);f+=df;
Options:
puls(f1,df1,f2,df2,...): df1=sin(f2);f2+=df2;...
coe(c): c*f; (default 0.1).
audio read
audio(id)read("nn.wav")
signal initialized with the audio file nn.wav.
audio spe
audio(num)set module
Returns audio volume of directx audio buffer number num.
audio(num)set module(v)
Changes this volume.
audio sin
audio(id)time(t)sin(f,a,p)
f1=a*sin(f*t+p) with t=duration.
Options:
dist(d): d = time increment (1 default).
rand(r1,r2,dn): random in (r1,r2)dn = amplitude of variation in (0,1).
sin(fi,pi,ai)module(mi):
interpolating modules.
audio(id)time(t1,t2)sin(f,p,a)
inserts, on an existing audio(id), signal:
a*sin(f*tt+p) with tt in [t1,t2].
Optione:
coe(c): wave form coefficient c normalized between 0.0 and 1.0
(attack).
audio(id)time(t)sin(f1,p1,a1,
f2,p2,a2,
...,fn,pn,an)
signal=interpolation(fi,ai,pi) on t seconds.
Options:
adjust(n)ext(e,c,f): extending the 3D curve
(fi,ai,pi) to n points.
audio(id)time(t)sin(f1,p1,a1,
f2,p2,a2,
...,fn,pn,an)add
signal=sum(a1*sin(fi*tt+pi)) on t seconds.
audio(id)time(ti,)sin(fi,
ai,pi)abs
inserts, on an existing audio(id), signal:
a*sin(f*tt+p) with tt in [t1,t2].
Optiions:
coe(c1,c2): signal = c1 * (old signal) + c2 * signal (0,1 default).
ext(e,c,f): extension.
audio sin law
audio(id)time(t)sin(f0,a0,p0)
law(y1,y2,...,yn)
Frequency fluctuates around f0 as
law(0,y1,0, 1/n,y2,0, ...,1,yn,0).
Options:
dist(d1,d2,...): time increments (0.01 default).
module(m1,m2,...): modules multiplied by mi (1 default).
audio(id)time(t)sin(f0,a0,p0)acc
Same with an accelerated law.
audio(id)time(t)sin(f0,a0,p0)dec
Same with a decelerated law.
audio(id)time(t)sin(f0,a0,p0)pend
Same with a poendulum law.
audio sin sin
audio(id)time(t)sin(f0,a0,p0)
sin(f1,a1,p1,
f2,a2,p2,...)
Frequency fluctuates around f0 as:
fi+1=ai*sin(fi*ti)+pi;
Options:
dist(t0,t1,...): increments of times (1,.1,.01,... default).
exp(e): sin ^ e.
audio sin interpol
audio(id)time(t)sin(f1,p1,a1, f2,p2,a2, ...)interpol(0,t1,t2,...,1)
interpolates:
frequencies between fi and fi+1
phases between pi and pi+1
amplitudes between ai and ai+1
at time ti (normalized between 0.0 and 1.0).
Options:
limit(L): if L is a 3D line (x=frequency).
max(m): frequency maximum (20000 default).
audio time
audio(id)time(s)
Builds an empty signal duration s seconds.
Options:
number(c,s):
c = number of channels (1 default).
s = datas size: 8 (char), 16 (short) (default).
audio time module
audio(id)time(t1,t2,...)module(m1,m2,...)
Changes the waveform of an existing audio object id by seting the amplitude
to mi at times ti (in [0,1]).
Options:
acc: accelerated variation.
coe(c): module ^= c, increases (c<1) or decreases (c>1) the variations.
dec: decelerated variation.
ext(e,c,f): extension.
law(x1,y1,0, x2,y2,0, ...): according to a law.
pend: pendulum variation.
Examples:
audio(1)time(0,1)module(0,1);
fade in.
audio(1)time(.5,1)module(1,0);
fade out on second part.
audio transf
audio(id)dila(d)
Does module(audio(id))*=d.
audio(id)tran(t)
Does audio(id) translated t (seconds).
audio var
audio(id)time(t)var sin(f1,a1,p1, a2,a2,p2)dist(df,da,dp)
Frequency varies in [f1,f2] with f1 and f2 in [0,1] and
freqency df in [0,1] (1 default).
Amplitude varies in [a1,a2] with a1 and a2 in [0,1], and
freqeency fa in [0,1] (1 default).
Phase varies in [p1,p2] with p1 and p2 in [-PI,PI] and
freqeency fp in [0,1] (1 default).
audio(id)time(t)var(v)
Creates audio id time t initialized at v.
Example:
w=calloc(44100)limit(32767);audio(1)time(1)var(w);play audio(1);
play the same sound as audio(1)time(1)sin(.22,.5);
audio(1)time(1)sin(.22,.5);play audio(1);
audio(id)time(t)time(dt)var(v)
Creates audio id time t with rectangular signal period dt
between -v and v.
Options:
var(v1,v2): between v1 and v2.
audio var
Returns the audio (between -127 and 128)
Options:
above(m): signal reduced to its components >: m.
adjust(a): ajuste the audio signal s according to the law
s=s+a*s*sin(i*dan), where n is the size i in [0,n-1].
audio(1): retutns the derivative of the signal audio (speed).
exp(e): returns signal^e.
max: signal reduced to its maximum components.
sort: processing the fft (sort(0.5,1,2,3): half real, real, imaginary, both).
speed: processing the signal speed (according to audio speed time).
Notee:
Must be done audio ini(n). The size of the returned vector is 2^n.
audio write
audio(id)write("nn")
writes signal in audio file nn.wav and property stock
in file nn.stock.
write stock audio(id)name("nn")
writes only property stock in file nn.stock and can be
recovered by read("nn.stock") format("anyflo").
writes signal in audio file nn.wav and property stock
in file nn.stock.
Use
Example
audio(1)read("0.wav"); /* read file 0.wav in object type audio */
generate stock audio(1); /* generate property stock */
write stock audio(1)name("nnn"); /* save stock in nnn.stock */
See also: