Stereo music visualization through manifold harmonics

Stereo music visualization through manifold harmonics
Thomas Lewiner, Clarissa Marques, João Paixão, Scarlett de Botton, Allyson Cabral, Renata Nascimento, Vinícius Mello, Adelailson Peixoto, Dimas Martinez, Thales Vieira

Abstract:

Music visualizations are nowadays included with virtually any media player. They usually rely on harmonic analysis of each sound channel, which automatically generate parameters for procedural image generation. However, only few music visualizations make use of 3d shapes. This paper proposes to use spectral mesh processing techniques, here manifold harmonics, to produce 3d stereo music visualization. The images are generated from 3d models by deforming an initial shape, mapping the sound frequencies to the mesh harmonics. A symmetry criterion is introduced to enhance the stereo effects on the deformed shape. A concise representation of the frequency mapping is proposed to allow for an animated gallery interface with genetic reproduction. Such galleries let the user quickly navigate between visual effects. Rendering such animated galleries in real-time is a challenging task, since it requires computing and rendering the deformed shapes at a very high rate. This paper introduces a direct GPU implementation of manifold harmonics filters, which allows the displaying of the animated galleries.

Downloads:

PDF paper (10.3 MB)
movie (31.4 MB)
YouTube

BibTeX:

@article{stereomh_tvc,
    author = {Thomas Lewiner and Clarissa Marques and João Paixão and Scarlett de Botton and Allyson Cabral and Renata Nascimento and Vinícius Mello and Adelailson Peixoto and Dimas Martinez and Thales Vieira},
    title = {Stereo music visualization through manifold harmonics},
    year = {2011},
    month = {october},
    journal = {The Visual Computer},
    volume = {27},
    number = {10},
    pages = {905--916},
    publisher = {Springer},
    doi = {10.1007/s00371-011-0617-4},
    url = {\url{http://thomas.lewiner.org/pdfs/stereomh_tvc.pdf}}
}