Event Calendar:
![[Previous Month]](fileadmin/template/main/calendar/img/left_day.gif)
![[Next Month]](fileadmin/template/main/calendar/img/right_day.gif)
Contact
Dipl.-Inf. Mario Kleiner |
| Address: | Spemannstr. 41 72076 Tübingen |
| Room number: | 1.A.03 |
| Phone: | +49 7071 601 1783 |
| Fax: | +49 7071 601 616 |
| E-Mail: |  mario.kleiner |
As part of a PhD thesis in computer science, i am working on the development of tools and methods, mostly software, to facilitate research in cognitive science. One focus is on video and computer graphics methods for dynamic facial perception research, together with Cristobal Curio, Martin Breidt and Katharina Dobs. Another project is the improvement and maintenance of the Visual Psychophysics Toolbox Version 3, a cross-platform, free software toolkit for cognitive science (http://www.psychtoolbox.org). I also contribute to the development of the GNU/Linux operating system and related parts of the free software ecosystem with the goal of further improving its already excellent suitability for neuro-science research.
Psychophysics Toolbox 3: Developing a modern toolkit for Psychophysics
Mario Kleiner, Cristóbal Curio, Heinrich Bülthoff
Introduction
One of the pillars of both basic and clinical behavioral sciences is the testing of subjects in perceptual tasks. The increasing power and functionality of computer technology has greatly expanded the scope and range of stimuli available for the psychophysical study of both vision and audition, as well as the sophistication of response-contingent experimental designs that may be implemented as part of such study. General purpose computing systems are not designed for psychophysics, therefore deploying these systems in the laboratory requires the time-consuming and technically challenging development and verification of specialized software. Clearly there is much to be gained through the sharing of effective solutions in the form of software toolkits. A popular option is Psychophysics Toolbox 3, a cross-platform, free and open-source software toolkit for the Matlab and Octave programming environments [1].
Goals
Our goal is to improve Psychophysics Toolbox 3 for a large variety of testing paradigms. We develop novel or more efficient methods for stimulus presentation, response collection, and for coping with variations and flaws in general purpose computing platforms. A long term goal is improvement of the underlying infrastructure itself, for example, the third party toolkits employed by us, and specifically the Linux operating system, to promote the use of Linux for neuro-science applications which sometimes require extensive precision, robustness and flexibility more easily achievable by an open-source computing platform.
Methods
Our development efforts are motivated both by feedback from users of the toolkit via its internet forum, and even more so by exploring the capabilities and limitations of the toolkit as part of in-house [2][3] and external research collaborations [4] which require implementation of challenging stimulation paradigms on general purpose commodity hardware. We take advantage of the latest developments in hardware, exploring how paradigms can be implemented more efficiently by use of new features, e.g., the programmable shading hardware of modern graphics cards. We collaborate with other open-source software projects, e.g., the Linux graphics and real-time developer community, helping to improve the functionality and suitability of such systems for very demanding neuro-science applications. This is done by participating in the design process of new features and also by contributing code to those projects. We also occasionally consult vendors of specialized research equipment on optimal design of their hardware for good interoperability with Psychtoolbox 3.
Initial results
Improvements have been made in various areas. One unique feature is the ability for users with limited programming skills to make use of efficient graphics hardware accelerated image processing (Fig. 1) and rendering of procedural dynamic stimuli (Fig. 2). Advanced video-capture and playback functionality allows for implementation of closed-loop action-perception and social interaction studies [4]. The timing precision and robustness of visual and auditory stimulus presentation [5] and response collection has been improved [6]. Some of this work has also found its way into the graphics subsystem of the Linux operating system, which may benefit other toolkits and similar applications as well [7].
Figure 1: Illustration of Psychtoolbox GPU image processing pipeline vs. regular drawing toolkits. Top: Typical toolkits directly draw to the OpenGL backbuffer. Bottom: Example processing graph in Psychtoolbox. Blue are intermediate frame buffers. Red are GPU accelerated processing operators. In this stereo configuration, each image for each eye would be drawn separately, then exposed to some per-eye image processing. The resultant images would be merged together, e.g., into an anglyph stereo image, then subjected to post-processing, e.g., gamma correction and written to the final framebuffer. All processing would happen transparently for the usercode and fast due to GPU hardware acceleration.
Figure 2: Two examples of advanced stimuli. Left: A structure from motion stimulus, generated from a 3D shape in real-time via fast GPU based random sampling and tracking of 3D surface points (color added for illustration – this is a rotating earth globe). Right: Linear superposition of 100 gabor patches in real-time via use of GPU accelerated alpha-blending and floating point framebuffers. While such stimuli can also be generated offline in other toolkits, Psychtoolbox allows to generate them with minimal amount of code and in real-time.
References
1. Kleiner M, Brainard D, Pelli D (2007) What‘s new in psychtoolbox-3?, Perception 36(ECVP 2007 Abstract Supplement) 14.
2. Curio C, Kleiner M, Breidt M, Bülthoff HH (2010) The virtual face mirror project: revealing dynamic self-perception in humans, 4th International Conference on Cognitive Systems (CogSys 2010) 4 1.
3. Conrad V, Bartels A, Kleiner M, Noppeney U (2010) Audiovisual interactions in binocular rivalry, Journal of Vision 10(10:27) 1-15.
4. Redcay E, Dodell-Feder D, Pearrow MJ, Mavros PL, Kleiner M, Gabrieli JDE, Saxe R (2010) Live face-to-face interaction during fmri: a new tool for social cognitive neuroscience, NeuroImage 50(4) 1639-1647.
5. Kleiner M (2010) Visual stimulus timing precision in psychtoolbox-3: tests, pitfalls and solutions, Perception 39(ECVP Abstract Supplement) 189.
6. Li X, Liang Z, Kleiner M, Lu Z-L (2010) Rtbox: a device for highly accurate response time measurements, Behavior Research Methods 42(1) 212-225.
7. See <http://www.phoronix.com/scan.php?page=news_item&px=ODcxNQ>
Academic Background
1993 - 2001
Diploma in Informatik (~ Masters in Computer Science) at the University of Tuebingen, Germany.
2002 - present
PhD student in Informatik, working at the MPI for Biological Cybernetics, Dept. Prof. Dr. Buelthoff.
Articles (9): | |
|---|---|
| • |
Giani AS  , , , Kleiner M , and Noppeney U (April-2012) Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses
NeuroImage 60(2) 1478–1489. 
|
| • |
Curio C , Giese MA , Breidt M , Kleiner M and Bülthoff HH (January-2012) Facial Action Perception After-Effects in Humans investigated by Animation
- . submitted |
| • |
, Schwaninger A , , Kleiner M and Knappmeyer B (December-2011) Recognition of emotion in moving and static composite faces
Swiss Journal of Psychology 70(4) 233-240.
|
| • |
Conrad V , Bartels A , Kleiner M and Noppeney U (August-2010) Audiovisual interactions in binocular rivalry
Journal of Vision 10(10:27) 1-15.
|
| • |
, , , , Kleiner M , and (May-2010) Live face-to-face interaction during fMRI: A new tool for social cognitive neuroscience
NeuroImage 50(4) 1639-1647.
|
| • |
, , Kleiner M and (February-2010) RTbox: a device for highly accurate response time measurements
Behavior Research Methods 42(1) 212-225.
|
| • |
Cunningham D , Kleiner M , Wallraven C and Bülthoff H (July-2005) Manipulating video sequences to determine the components of conversational facial expressions
ACM Transactions on Applied Perception 2(3) 251-269.
|
| • |
, , and Kleiner M (2005) Sonic self-avatars and self-motion in virtual environments
. |
| • |
, , and Kleiner M (2005) When what you see is what
you hear: Auditory-visual integration and presence in virtual environments
CyberPsychology & Behavior . |
Conference papers (11): | |
| • |
Curio C , Giese MA , Breidt M , Kleiner M and Bülthoff HH (August-2008) Probing Dynamic Human Facial Action Recognition From The Other Side Of The Mean
5th Symposium on Applied Perception in Graphics and Visualization (APGV 2008), ACM Press, New York, NY, USA, 59-66.
|
| • |
Curio C , Giese MA , Breidt M , Kleiner M and Bülthoff HH (April-2008) Exploring Human Dynamic Facial Expression Recognition with Animation
International Conference on Cognitive Systems (CogSys 2008), Karlsruhe, Germany, 1-6.
|
| • |
Curio C , Breidt M , Kleiner M , Vuong QC , Giese MA and Bülthoff HH (July-2006) Semantic 3D motion retargeting for facial animation
3rd Symposium on Applied Perception in Graphics and Visualization (APGV 2006), ACM Press, New York, NY, USA, 77-84.
|
| • |
, , and Kleiner M (2005) Travelling without moving: Auditory scene cues for translational self-motion
ICAD05.
accepted
|
| • |
Kleiner M , Wallraven C , Breidt M , Cunningham DW and Bülthoff HH (December-2004) Multi-viewpoint video capture for facial perception research
Workshop on Modelling and Motion Capture Techniques for Virtual Environments (CAPTECH 2004), 55-60.
|
| • |
Cunningham DW , Kleiner M , Wallraven C and Bülthoff HH (August-2004) The components of conversational facial expressions
1st Symposium on Applied Perception in Graphics and Visualization (APGV 2004), ACM Press, New York, NY, USA, 143-149.
|
| • |
Kleiner M , Schwaninger A , Cunningham DW and Knappmeyer B (August-2004) Using facial texture manipulation to study facial motion perception
1st Symposium on Applied Perception in Graphics and Visualization (APGV 2004), ACM Press, New York, NY, USA, 180-180.
|
| • |
, , and Kleiner M (2004) Auditory Presence, Individualized Head-Related Transfer Functions, and Illusory Ego-Motion in Virtual Environments
Proceedings of Presence 2004, 141-147.
|
| • |
, and Kleiner M (2004) Perception of Self-motion and Presence in Auditory Virtual Environments
Proceedings of Presence 2004, 252-258.
|
| • |
Cunningham DW , Breidt M , Kleiner M , Wallraven C and Bülthoff HH (September-2003) The inaccuracy and insincerity of real faces
3rd IASTED International Conference on Visualization, Imaging, and Image Processing (VIIP 2003), Acta Press, Anaheim, CA, USA, 7-12.
|
| • |
Cunningham DW , Breidt M , Kleiner M , Wallraven C and Bülthoff HH (May-2003) How Believable are Real Faces: Towards a Perceptual Basis for Conversational Animation
16th International Conference on Computer Animation and Social Agents (CASA 2003), IEEE, Los Alamitos, CA, USA, 23-29.
|
Contributions to books (1): | |
| • |
Curio C , Giese MA , Breidt M , Kleiner M and Bülthoff HH : Recognition of Dynamic Facial Action Probed by Visual Adaptation, 47-65.
In: Dynamic Faces: Insights from Experiments and Computation, (Ed) , MIT Press, Cambridge, MA, USA, (December-2010).
|
Technical reports (1): | |
| • |
Kleiner M , Wallraven C and Bülthoff HH : The MPI VideoLab: A system for high quality synchronous recording of video and audio from multiple viewpoints, 123, Max Planck Institute for Biological Cybernetics, Tübingen, Germany, (May-2004).
|
Posters (15): | |
| • |
Schindler A , Kleiner M and Bartels A (November-2011): Decoding egocentric space in human posterior parietal cortex using fMRI, 41st Annual Meeting of the Society for Neuroscience (Neuroscience 2011), Washington, DC, USA.
|
| • |
Dobs K , Kleiner M , Bülthoff I , Schultz J and Curio C (September-2011): Investigating idiosyncratic facial dynamics with motion retargeting, 34th European Conference on Visual Perception, Toulouse, France, Perception, 40(ECVP Abstract Supplement) 115.
|
| • |
Giani AS , , , Kleiner M , and Noppeney U (June-2011): Using steady state responses in MEG to study information integration within and across the senses, 17th Annual Meeting of the Organization for Human Brain Mapping (HBM 2011), Québec, Canada.
|
| • |
Kleiner M (October-2010): Visual stimulus timing precision in Psychtoolbox-3: Tests, pitfalls &
solutions, 11th Conference of Junior Neuroscientists of Tübingen (NeNa 2010), Heiligkreuztal, Germany.
|
| • |
Kleiner M (August-2010): Visual stimulus timing precision in Psychtoolbox-3: Tests, pitfalls and solutions, 33rd European Conference on Visual Perception, Lausanne, Switzerland, Perception, 39(ECVP Abstract Supplement) 189.
|
| • |
Conrad V , Kleiner M , Hartcher-O‘Brien J , Bartels A , Bülthoff HH and Noppeney U (June-2010): Audio-visual interactions in binocular rivalry using the Shepard illusion in the auditory and visual domain, 11th International Multisensory Research Forum (IMRF 2010), Liverpool, UK.
|
| • |
Curio C , Kleiner M , Breidt M and Bülthoff HH (January-2010): The Virtual Face Mirror Project: Revealing Dynamic Self-Perception in Humans, 4th International Conference on Cognitive Systems (CogSys 2010), Zürich, Switzerland.
|
| • |
Conrad V , Bartels A , Kleiner M and Noppeney U (July-2009): Auditory influences on the temporal dynamics of binocular rivalry, 10th International Multisensory Research Forum (IMRF 2009), New York, NY, USA.
|
| • |
Noppeney U , Ostwald D , Kleiner M and Werner S (June-2008): The prefrontal cortex accumulates object evidence through differential connectivity to the visual and auditory cortices, 14th Annual Meeting of the Organization for Human Brain Mapping (HBM 2008), Melbourne, Australia, NeuroImage, 41(Supplement 1) S150.
|
| • |
Curio C , Giese MA , Breidt M , Kleiner M and Bülthoff HH (August-2007): Aftereffects in the recognition of dynamic facial expressions, 30th European Conference on Visual Perception, Arezzo, Italy, Perception, 36(ECVP Abstract Supplement) 146.
|
| • |
Curio C , Giese MA , Breidt M , Kleiner M and Bülthoff HH (July-2007): Perception of Dynamic Facial Expressions Probed by a New High-Level After-Effect, 10th Tübinger Wahrnehmungskonferenz (TWK 2007), 10 1.
|
| • |
Noppeney U , Ostwald D , Kleiner M and Werner S (June-2007): Accumulation of object evidence from multiple senses, 13th Annual Meeting of the Organization for Human Brain Mapping (HBM 2007), 13 1.
|
| • |
Curio C , Giese MA , Breidt M , Kleiner M and Bülthoff HH (May-2007): High-level after-effects in the recognition of dynamic facial expressions, Journal of Vision, 7(9) 994.
|
| • |
Kleiner M , Vuong QC , Bülthoff HH and Thornton IM (August-2004): Recognising Famous Gaits, 27th European Conference on Visual Perception, Budapest, Hungary, Perception, 33(ECVP Abstract Supplement) 99.
|
| • |
Schwaninger A , Cunningham DW and Kleiner M (November-2002): Moving the Thatcher Illusion, 10th Annual Worksop on Object Perception and Memory (OPAM 2002), Kansas City, KS, USA.
|
Theses (1): | |
| • |
Kleiner M : Ein stereobasiertes Verfahren zum dreidimensionalen Tracking von Markern in menschlichen Gesichtern, (August-2001).
Diplom thesis |
Talks (2): | |
| • |
Noppeney U , Ostwald D , Werner S and Kleiner M (July-2008): The prefrontal cortex accumulates object evidence through differential connectivity to the visual and auditory cortices, 9th International Multisensory Research Forum (IMRF 2008), Hamburg, Germany(189) 118.
|
| • |
Kleiner M , and (August-2007): What's new in Psychtoolbox-3?, 30th European Conference on Visual Perception, Arezzo, Italy, Perception, 36(ECVP Abstract Supplement) 14.
|