Neural movements

Some links may not be active (if the application is not in the folder env / applications / mouv_gen the distributed version).



This application shows:

1) How build neural networks of type multi layers perceptron

2) How build trajecoiries

2 modes:
rotations defined by:
       angle an and axis xr,yr,zr
       giving 2 trajectort types: rotation and axe.
des quaternions of ethe form w,x,y,z, with:
       x,y,z such as module(x,y,z)=sin(a/2) made with a liste
       quaternion w,x,y,z as 4D unit vecteur (module 1).
The movement of volume i is defined by quaternions trajectory obtained by spherically interpolating a quaternions list
qi,1 qi,2 ... qi,nq de nq.
For several volumes Épaule,Humerus,Radius,Poignet,Doigts there will be as many trajectories.

4) How use these features

pour programmer un bras pouvant:
       interactively imitate the real arm capture.
       interact "intelligently" with a real actor he learns postures (under development).


For programming in C out of anyflo this application it will be necessary to:
1) programming neural networks.
2) programming quaternions and spherical interpolation.
3) programming interface for real-time interaction.

Movements definition

Rotational movements are defined in a file .rota of the form:

r1,1 r1,2 ... r1,nq
r2,1 r2,2 ... r2,nq
rnv,1 rnv,2 ... rnv,nq

Where ri,j=a,x,y,z is a rotation angle a and axis x,y,z.
The quaternions movements are definde in a file .quat of the form:

q1,1 q1,2 ... q1,nq
q2,1 q2,2 ... q2,nq
qnv,1 qnv,2 ... qnv,nq

Where qi,j=w,x,y,z is a quaternion.

These movements files can be built from capture of those of a real actor.

A real-time interaction require are interfaced the capture and interpretation.


Foilder env

*.rota: matrix files describing movements.


Java scrypte launching video format anyflo.


configuration file in folder distribution_eng/env of root distribution_eng.


Function in anyflo language.


This file.


Java scrypte launching the application.


Java scrypte launching the neural network demo.
demo1_network.func: demo program (in anyflo language).
corps_simple.func: construction program (in anyflo language) of a simple body (used by demo1_network().


Main menu
EXPL: explanations submenu.
NET: network creation submenu.
GEN: genetic generation.
INI: initialization
ILLUM: yes/no illum.
PEAU: yes/no skin display.
EXT: yes/no ext.
FUNC: displays functions names, click: displays the text.
STEPbySTEP: step by step mode, click on CONTINUE (green): next image, click on X (red) exits this mode.
Warning: dont click on RETURN, launches again the program...

A flashing message indicates whenever the procedure, the menu box that need to click pulse for about 4 seconds.
EXIT: to exit the program.

EXPL: explanation submenu

HEL: help.

ROTA: click on bone name (Épaule, Humerus, Radius, etc...) then change rotation angle rota and axis, the red arm moves accordingly.
Click on ALL() selects all bones.
Click in the green frame changes the eye.

COMPLEX: isomorphism of planar rotations and complex numbers:
Change the components a and b of the unit complex number z=a+i*b represents the corresponding angle of cosinus a and of sinus b.

QUAT: isomorphism of spatial rotations and quaternions:
Change the rotation angle rota and the axis axe of ethe unit quaternion q=w+i*a+j*b+k*c, the red point P displayed on sphere radius 1 represents the corresponding rotation angle 2*acos(w) and axis OP.
Click in the green frame changes the eye.

MOUV: spherical interpolation of two quaternions:
Two quaternions q1=w1,x1,y1,z1 and q2=w2,x2,y2,z2 are represended by 2 points P1(red) et P2 (green) of the sphere radius 1.
Change the interpolation coefficient u, the point P representing the interpolated quaternion between q1 and q2 is displayed on the sphere between P1 and P2.
Change the quaternions on the scales.
The spherical interpolation formula is:
c1=(sin((1-u)*a))/sin(a) c2=(sin(u*a))/sin(a) q=c1*q1+c2*q2
AUTO(): automatic interpolation.
Click in the green frame changes the eye.

One POSITION is represented by a quaternion.
One MOUVEMENT Élémentaire est représenté par une liste de quaternions interpolés. et de sinus b.
One MULTIPLE MOVEMENT (for exemple the arm bones) is represented by a list of such lists : a quaternions matrix.
Such movement has been stored on the disk by a file .rota

NET: network creation submenu

HEL: help de ce sous menu.
MIME: mimetic network creation sub submenu.
OTHER: network other type creation sub submenu .
Sub sub menu MIME
       HEL: help.
       Click on CREATE:
       Click on ALEA: creates a random learning pair.
       Cliquer sur READ: click on nnn.rota, the file is loaded as (learning pair (with law=motif, ie ouptut=input).
       Click on TRAIN: training, the error curve is displayed and must tend towards zero. If there is no convergence can click NOISE for randomly change the matrix of synaptic weights or on coe1 and coe2 are.constants learning. You can also randomly change motifs and laws by clicking on a small value of the scale alea.
       Click on MATRIX: displays the matrix of synaptic weights.
OTHER: developing.

GEN: movements generation

HEL: help.
Cliquer sur ROTA traitement des rotations ou
sur QUAT quaternions ou
TRAIN: displays training.
MATRIX: displays synaptic weights matrix.
Cliquer sur INTER: interactive mode
A small green arm appears in a left small frame, click on it changes its position used as model for the right appearing arm. The small arm movement is passed as input of the neural network, providing as output the arm movement. According to the choosed mode ROTA or QUAT rotations or quaternions are used.
If TRAIN is active training is displayed: parameters can be changed (as explainde in TRAIN).
You can see, on the error curve, the neural network auto configuring in real time.
You can click on the green laws (verts) to provoke the corresponding movement.




Main menu:
mouv_neur(): main function.
externes(): external variables declaration.
init(): initializations.
generer_menu(): menus generation.
MEN(): 1st function of the main menu MEN.
HEL(): help function of the main menu.
INI(): reinitializations.
ILLUM(): yes/no illum.
PEAU(): displays skins.
EXT(): yes/no ext.
FUNC(): functions display.
fonc_FUNC(): called each image if FUNC is active.
STEPbySTEP(): step by step mode.
EXIT(): exit the program.
DEUX_POINTS(): inhibits the display of scales, menus, messages and texts.

Submenu EXPL

Submenu EXPL
EXPL(): submenu EXPL 1st function.
EXPL_HEL(): help of EXPL.
fonc_EXPL(): called each image if EXPL is active.
EXPL_ROTA(): rotation explanations.
fonc_EXPL_ROTA(): called each image if EXPL_ROTA is active.
EXPL_COMPLEX(): explanations about the unit complex numbers associated to a planar rotation.
fonc_EXPL_COMPLEX(): called each image if EXPL_COMPLEX is active.
EXPL_QUAT(): explanations about the unit quaternions associated to a spatial rotation.
fonc_EXPL_QUAT(): called each image if EXPL_QUAT is active.
MOUV(): explianation about the quaternions spherical interpolation.
fonc_EXPL_MOUV(): called each image if EXPL_MOUV is active.
AUTO(): automatic interpolation.
EXPL_MOUVS(): expanations about several movements.
fonc_EXPL_MOUVS(): called each image if EXPL_MOUVS is active.
AUTO(): automatic interpolation.

NETWORK(): 1st function of submenu CREATE.
fonc_NETWORK(): called each image if CREATE is active.
SET_PARAM_NETWORK(): computer network parameters.
NETWORK_MIME(): mimetic network.

NETWORK_MIME(): 1st function of submenu CREATE.
       fonc_NETWORK_MIME(): called each image if NETWORK_MIME is active.
NETWORK_MIME_CREATE(): creates a hidden layers network (of type perceptron):
       NETWORK_MIME_ALEA: creates identical motif and law as a learning pair. (displayed green and red).
       NETWORK_MIME_READ: click on , the file is loaded as a learning pair.
       fonc_READ(): called each image if NETWORK_MIME_READ is active.
       NETWORK_MIME_TRAIN: launching training.
       NETWORK_MIME_MATRIX: displays the matrix.
CREATE_QUAT(): validates the quaterniuons trajectories.

Sous menu GEN
GEN(): 1st function of the submenu GEN.
GEN_HEL(): help GEN.
fonc_GEN(): called each image if GEN is active.
GEN_INTER(): interaction.
       fonc_GEN_INTER(): called each image if GEN_INTER is active.
GEN_ROTA(): rotations validation.
GEN_QUAT(): quaternions validation.
GEN_TRAIN(): displaying training.
GEN_MATRIX(): matrix display.


F0(): called once at the beginning of the interaction.
F5(): called each image after the volumes display, allows displaying scales and process with et texts on the image.

User functions:
dim_screen(): computes dimensions according to the screen size.
set_message(): initializing a flashing message.
coder_bras(): encoding the arm movement.
lire_rota(): read a movements file nnn.rota in the current memory.
cou_vol(): changes the selectioned volume color.
ALL(): selection all volumes.
aff_quat(): displays a quaternion.
rendre_visible(): displays volume or skin.
rendre_invisible(): inhibits the display.
modif_param(): displays the parameters input scales.
VOL_QUAT(): returns the arm position evaluation with respect to memory num.
tourner_Vol0(): rotatesvolumes Vol0 in memory -1 (INTER mode).
creer_reseau(): creates a network.
aff_reseau(): affichage d´un réseau.
Vol0_Vol(): Vol0 -> réseau _> Vol..
motif_loi_alea(): randomly generates a learning pair.
tester_creer(): returns -1 if CREATE is not active.

Fonctions de construction:
cons_bras(inc): building an arm index inc.
cons_rotule(): building a rotle (shoulder and wirst).
cons_doigt(): building a finger.
cons_doigt_1(): building a phalange.
cons_peau_main(): building skins.
cons_peau(): building a skin.

Coding and decoding
CODAGE(): spatial rotation transformation (an,xr,yr,zr) into a quatrenion (w,x,y,z) then encoding on 4*Nb_Bit characters 0 and 1.
DECODAGE_ROTA(): decoding a genome (this genome coding a rotation) into a rotation.
DECODAGE_ROTA_QUAT(): decoding a genome (this genome coding a rotation) into a quaternion.
DECODAGE_QUAT_QUAT(): decoding a genome (this genome coding a quaternion) into a quaternion.

Tajectories generation:
GEN_TRAJ_ROTA(): rotation trajectories generation.
GEN_TRAJ_QUAT(): quaternion trajectories generation by spherical interpolation.

Les variables

Sem_AUTO: automatic interpolation (1), manually (0).
Inc=200: increment value of the number of the volumes discribing an arm to another.
Lead,Epaule,Humerus,Radius,Poignet,Pouce,Indexe,Majeur,Annulaire,Petit,Doigts: noms des os.
Peau_Bras,Peau_Main,Peau_Pouce,Peau_Indexe,Peau_Majeur,Peau_Annulaire,Peau_Petit: skins names.
Nb_Quat: number of quaternions that make up a basic movement.
Nb_Vol: animated volumes number.
Nb_Mem: number of memories encoding read movements.
Ni: trajectory images number.
Num: number of the selectioned volume.
Pend: pendulum (or not) movement.
Amp: amplitude coefficient.
Alea: randomly change coefficient.


Movement recognition
Movements generation

Movement recognition

Just describe movements in files .rota, which will be loaded as learning pairs. They will be recognized, after learning, as well as the neighbor movements

Movements generation

Not learned movements passed as input to the network give, as output, new movements that are neither learned nor arbitrary averaged movements, in this sense we can say that the network improvises.