Contact

Dipl.-Inf. Mario Kleiner

 

Picture of Kleiner, Mario, Dipl.-Inf.

Mario Kleiner

Position: Research Technician  Unit: Alumni Bülthoff

As part of a PhD thesis in computer science, i am working on the development of tools and methods, mostly software, to facilitate research in cognitive science. One focus is on video and computer graphics methods for dynamic facial perception research, together with Cristobal Curio, Martin Breidt and Katharina Dobs. Another project is the improvement and maintenance of the Visual Psychophysics Toolbox Version 3, a cross-platform, free software toolkit for cognitive science (http://www.psychtoolbox.org). I also contribute to the development of the GNU/Linux operating system and related parts of the free software ecosystem with the goal of further improving its already excellent suitability for neuro-science research.

Psychophysics Toolbox 3: Developing a modern toolkit for Psychophysics

Mario Kleiner, Cristóbal Curio, Heinrich Bülthoff

 

Introduction

One of the pillars of both basic and clinical behavioral sciences is the testing of subjects in perceptual tasks. The increasing power and functionality of computer technology has greatly expanded the scope and range of stimuli available for the psychophysical study of both vision and audition, as well as the sophistication of response-contingent experimental designs that may be implemented as part of such study. General purpose computing systems are not designed for psychophysics, therefore deploying these systems in the laboratory requires the time-consuming and technically challenging development and verification of specialized software. Clearly there is much to be gained through the sharing of effective solutions in the form of software toolkits. A popular option is Psychophysics Toolbox 3, a cross-platform, free and open-source software toolkit for the Matlab and Octave programming environments [1].

 

Goals

Our goal is to improve Psychophysics Toolbox 3 for a large variety of testing paradigms. We develop novel or more efficient methods for stimulus presentation, response collection, and for coping with variations and flaws in general purpose computing platforms. A long term goal is improvement of the underlying infrastructure itself, for example, the third party toolkits employed by us, and specifically the Linux operating system, to promote the use of Linux for neuro-science applications which sometimes require extensive precision, robustness and flexibility more easily achievable by an open-source computing platform.

 

Methods

Our development efforts are motivated both by feedback from users of the toolkit via its internet forum, and even more so by exploring the capabilities and limitations of the toolkit as part of in-house [2][3] and external research collaborations [4] which require implementation of challenging stimulation paradigms on general purpose commodity hardware. We take advantage of the latest developments in hardware, exploring how paradigms can be implemented more efficiently by use of new features, e.g., the programmable shading hardware of modern graphics cards. We collaborate with other open-source software projects, e.g., the Linux graphics and real-time developer community, helping to improve the functionality and suitability of such systems for very demanding neuro-science applications. This is done by participating in the design process of new features and also by contributing code to those projects. We also occasionally consult vendors of specialized research equipment on optimal design of their hardware for good interoperability with Psychtoolbox 3.

 

Initial results

Improvements have been made in various areas. One unique feature is the ability for users with limited programming skills to make use of efficient graphics hardware accelerated image processing (Fig. 1) and rendering of procedural dynamic stimuli (Fig. 2). Advanced video-capture and playback functionality allows for implementation of closed-loop action-perception and social interaction studies [4]. The timing precision and robustness of visual and auditory stimulus presentation [5] and response collection has been improved [6]. Some of this work has also found its way into the graphics subsystem of the Linux operating system, which may benefit other toolkits and similar applications as well [7].


Illustration of Psychtoolbox GPU image processing pipeline vs. regular drawing toolkits.

Figure 1: Illustration of Psychtoolbox GPU image processing pipeline vs. regular drawing toolkits. Top: Typical toolkits directly draw to the OpenGL backbuffer. Bottom: Example processing graph in Psychtoolbox. Blue are intermediate frame buffers. Red are GPU accelerated processing operators. In this stereo configuration, each image for each eye would be drawn separately, then exposed to some per-eye image processing. The resultant images would be merged together, e.g., into an anglyph stereo image, then subjected to post-processing, e.g., gamma correction and written to the final framebuffer. All processing would happen transparently for the usercode and fast due to GPU hardware acceleration.

 

Figure 2: Two examples of advanced stimuli.

Figure 2: Two examples of advanced stimuli. Left: A structure from motion stimulus, generated from a 3D shape in real-time via fast GPU based random sampling and tracking of 3D surface points (color added for illustration – this is a rotating earth globe). Right: Linear superposition of 100 gabor patches in real-time via use of GPU accelerated alpha-blending and floating point framebuffers. While such stimuli can also be generated offline in other toolkits, Psychtoolbox allows to generate them with minimal amount of code and in real-time.

 

References

 

1. Kleiner M, Brainard D, Pelli D (2007) What‘s new in psychtoolbox-3?, Perception 36(ECVP 2007 Abstract Supplement) 14.

 

2. Curio C, Kleiner M, Breidt M, Bülthoff HH (2010) The virtual face mirror project: revealing dynamic self-perception in humans, 4th International Conference on Cognitive Systems (CogSys 2010) 4 1.

 

3. Conrad V, Bartels A, Kleiner M, Noppeney U (2010) Audiovisual interactions in binocular rivalry, Journal of Vision 10(10:27) 1-15.

 

4. Redcay E, Dodell-Feder D, Pearrow MJ, Mavros PL, Kleiner M, Gabrieli JDE, Saxe R (2010) Live face-to-face interaction during fmri: a new tool for social cognitive neuroscience, NeuroImage 50(4) 1639-1647.

 

5. Kleiner M (2010) Visual stimulus timing precision in psychtoolbox-3: tests, pitfalls and solutions, Perception 39(ECVP Abstract Supplement) 189.

 

6. Li X, Liang Z, Kleiner M, Lu Z-L (2010) Rtbox: a device for highly accurate response time measurements, Behavior Research Methods 42(1) 212-225.

 

7. See <http://www.phoronix.com/scan.php?page=news_item&px=ODcxNQ>

Academic Background

1993 - 2001

Diploma in Informatik (~ Masters in Computer Science) at the University of Tuebingen, Germany.

2002 - present

PhD student in Informatik, working at the MPI for Biological Cybernetics, Dept. Prof. Dr. Buelthoff.

Preferences: 
References per page: Year: Medium:

  
Show abstracts

Articles (10):

Giani AS, Belardinelli P, Ortiz E, Kleiner M and Noppeney U (November-2015) Detecting tones in complex auditory scenes NeuroImage 122 203–213.
Redcay E, Dodell-Feder D, Mavros PL, Kleiner M, Pearrow MJ, Triantafyllou C, Gabrieli JD and Saxe R (October-2013) Atypical brain activation patterns during a face-to-face joint attention game in adults with autism spectrum disorder Human Brain Mapping 34(10) 2511–2523.
Conrad V, Kleiner M, Bartels A, Hartcher O'Brien J, Bülthoff HH and Noppeney U (August-2013) Naturalistic Stimulus Structure Determines the Integration of Audiovisual Looming Signals in Binocular Rivalry PLoS ONE 8(8) 1-8.
Redcay E, Kleiner M and Saxe R (June-2012) Look at this: the neural correlates of initiating and responding to bids for joint attention Frontiers in Human Neuroscience 6(169) 1-14.
Giani AS, Ortiz EB, Belardinelli P, Kleiner M, Preissl H and Noppeney U (April-2012) Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses NeuroImage 60(2) 1478–1489.
Chiller-Glaus SD, Schwaninger A, Hofer F, Kleiner M and Knappmeyer B (December-2011) Recognition of emotion in moving and static composite faces Swiss Journal of Psychology 70(4) 233-240.
Conrad V, Bartels A, Kleiner M and Noppeney U (August-2010) Audiovisual interactions in binocular rivalry Journal of Vision 10(10:27) 1-15.
Redcay E, Dodell-Feder D, Pearrow MJ, Mavros PL, Kleiner M, Gabrieli JDE and Saxe R (May-2010) Live face-to-face interaction during fMRI: A new tool for social cognitive neuroscience NeuroImage 50(4) 1639-1647.
Li X, Liang Z, Kleiner M and Lu Z-L (February-2010) RTbox: a device for highly accurate response time measurements Behavior Research Methods 42(1) 212-225.
Cunningham D, Kleiner M, Wallraven C and Bülthoff H (July-2005) Manipulating video sequences to determine the components of conversational facial expressions ACM Transactions on Applied Perception 2(3) 251-269.
pdf

Conference papers (8):

Curio C, Giese MA, Breidt M, Kleiner M and Bülthoff HH (August-2008) Probing Dynamic Human Facial Action Recognition From The Other Side Of The Mean, 5th Symposium on Applied Perception in Graphics and Visualization (APGV 2008), ACM Press, New York, NY, USA, 59-66.
pdf
Curio C, Giese MA, Breidt M, Kleiner M and Bülthoff HH (April-2008) Exploring Human Dynamic Facial Expression Recognition with Animation, International Conference on Cognitive Systems (CogSys 2008), Karlsruhe, Germany, 1-6.
Curio C, Breidt M, Kleiner M, Vuong QC, Giese MA and Bülthoff HH (July-2006) Semantic 3D motion retargeting for facial animation, 3rd Symposium on Applied Perception in Graphics and Visualization (APGV 2006), ACM Press, New York, NY, USA, 77-84.
pdf
Kleiner M, Wallraven C, Breidt M, Cunningham DW and Bülthoff HH (December-2004) Multi-viewpoint video capture for facial perception research, Workshop on Modelling and Motion Capture Techniques for Virtual Environments (CAPTECH 2004), 55-60.
pdf
Cunningham DW, Kleiner M, Wallraven C and Bülthoff HH (August-2004) The components of conversational facial expressions, 1st Symposium on Applied Perception in Graphics and Visualization (APGV 2004), ACM Press, New York, NY, USA, 143-149.
pdf
Kleiner M, Schwaninger A, Cunningham DW and Knappmeyer B (August-2004) Using facial texture manipulation to study facial motion perception, 1st Symposium on Applied Perception in Graphics and Visualization (APGV 2004), ACM Press, New York, NY, USA, 180.
pdf
Cunningham DW, Breidt M, Kleiner M, Wallraven C and Bülthoff HH (September-2003) The inaccuracy and insincerity of real faces, 3rd IASTED International Conference on Visualization, Imaging, and Image Processing (VIIP 2003), Acta Press, Anaheim, CA, USA, 7-12.
pdf
Cunningham DW, Breidt M, Kleiner M, Wallraven C and Bülthoff HH (May-2003) How believable are real faces? Towards a perceptual basis for conversational animation, 16th International Conference on Computer Animation and Social Agents (CASA 2003), IEEE, Los Alamitos, CA, USA, 23-29.
pdf

Contributions to books (1):

Curio C, Giese MA, Breidt M, Kleiner M and Bülthoff HH: Recognition of Dynamic Facial Action Probed by Visual Adaptation, 47-65. In: Dynamic Faces: Insights from Experiments and Computation, (Ed) C. Curio, MIT Press, Cambridge, MA, USA, (December-2010).

Technical reports (1):

Kleiner M, Wallraven C and Bülthoff HH: The MPI VideoLab: A system for high quality synchronous recording of video and audio from multiple viewpoints, 123, Max Planck Institute for Biological Cybernetics, Tübingen, Germany, (May-2004).
pdf

Posters (19):

Schindler A, Kleiner M and Bartels A (November-2011): Decoding egocentric space in human posterior parietal cortex using fMRI, 41st Annual Meeting of the Society for Neuroscience (Neuroscience 2011), Washington, DC, USA.
Schindler A, Kleiner M and Bartels A (October-2011): Decoding Egocentric Space in human Posterior Parietal Cortex using fMRI, 12th Conference of Junior Neuroscientists of Tübingen (NeNA 2011), Heiligkreuztal, Germany.
Giani A, Erick O, Belardinelli P, Kleiner M, Preissl H and Noppeney U (October-2011): Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses, 12th Conference of Junior Neuroscientists of Tübingen (NeNA 2011), Heiligkreuztal, Germany.
Dobs K, Kleiner M, Bülthoff I, Schultz J and Curio C (September-2011): Investigating idiosyncratic facial dynamics with motion retargeting, 34th European Conference on Visual Perception, Toulouse, France, Perception, 40(ECVP Abstract Supplement) 115.
Giani AS, Ortiz EB, Belardinelli P, Kleiner M, Preissl H and Noppeney U (June-2011): Using steady state responses in MEG to study information integration within and across the senses, 17th Annual Meeting of the Organization for Human Brain Mapping (HBM 2011), Québec City, Canada.
pdf
Kleiner M (October-2010): Visual stimulus timing precision in Psychtoolbox-3: Tests, pitfalls & solutions, 11th Conference of Junior Neuroscientists of Tübingen (NeNa 2010), Heiligkreuztal, Germany.
Kleiner M (August-2010): Visual stimulus timing precision in Psychtoolbox-3: Tests, pitfalls and solutions, 33rd European Conference on Visual Perception, Lausanne, Switzerland, Perception, 39(ECVP Abstract Supplement) 189.
Conrad V, Kleiner M, Hartcher-O‘Brien J, Bartels A, Bülthoff HH and Noppeney U (June-2010): Audio-visual interactions in binocular rivalry using the Shepard illusion in the auditory and visual domain, 11th International Multisensory Research Forum (IMRF 2010), Liverpool, UK.
Redcay E, Dodell-Feder D, Pearrow MJ, Kleiner M, Mavros PL, Wang J, Gabrieli JDE and Saxe R (April-18-2010): Do you see what I see? The neural bases of joint attention during a live interactive game, 17th Annual Meeting of the Cognitive Neuroscience Society (CNS 2010), Montréal, Canada, Journal of Cognitive Neuroscience, 22(Supplement) 71.
Curio C, Kleiner M, Breidt M and Bülthoff HH (January-2010): The Virtual Face Mirror Project: Revealing Dynamic Self-Perception in Humans, 4th International Conference on Cognitive Systems (CogSys 2010), Zürich, Switzerland.
pdf
Conrad V, Bartels A, Kleiner M and Noppeney U (July-2009): Auditory influences on the temporal dynamics of binocular rivalry, 10th International Multisensory Research Forum (IMRF 2009), New York, NY, USA.
Noppeney U, Ostwald D, Kleiner M and Werner S (June-2008): The prefrontal cortex accumulates object evidence through differential connectivity to the visual and auditory cortices, 14th Annual Meeting of the Organization for Human Brain Mapping (HBM 2008), Melbourne, Australia, NeuroImage, 41(Supplement 1) S150.
Curio C, Giese MA, Breidt M, Kleiner M and Bülthoff HH (August-2007): Aftereffects in the recognition of dynamic facial expressions, 30th European Conference on Visual Perception, Arezzo, Italy, Perception, 36(ECVP Abstract Supplement) 146.
Curio C, Giese MA, Breidt M, Kleiner M and Bülthoff HH (July-2007): Perception of Dynamic Facial Expressions Probed by a New High-Level After-Effect, 10th Tübinger Wahrnehmungskonferenz (TWK 2007), Tübingen, Germany.
Noppeney U, Ostwald D, Kleiner M and Werner S (June-2007): Accumulation of object evidence from multiple senses, 13th Annual Meeting of the Organization for Human Brain Mapping (HBM 2007), Chicago, IL, USA, NeuroImage, 36(Supplement 1) S109.
pdf
Curio C, Giese MA, Breidt M, Kleiner M and Bülthoff HH (June-2007): High-level after-effects in the recognition of dynamic facial expressions, 7th Annual Meeting of the Vision Sciences Society (VSS 2007), Sarasota, FL, USA, Journal of Vision, 7(9) 994.
Schwaninger A, Kleiner M, Chiller-Glaus S, Hofer F and Knappmeyer B (August-2006): Recognition of emotion in moving and static composite faces, 29th European Conference on Visual Perception, St. Petersburg, Perception, 35(ECVP Abstract Supplement) 212.
Kleiner M, Vuong QC, Bülthoff HH and Thornton IM (September-2004): Recognising Famous Gaits, 27th European Conference on Visual Perception, Budapest, Hungary, Perception, 33(ECVP Abstract Supplement) 99.
pdf
Schwaninger A, Cunningham DW and Kleiner M (November-2002): Moving the Thatcher Illusion, 10th Annual Worksop on Object Perception and Memory (OPAM 2002), Kansas City, KS, USA.

Theses (1):

Kleiner M: Ein stereobasiertes Verfahren zum dreidimensionalen Tracking von Markern in menschlichen Gesichtern, Eberhard-Karls-Universität Tübingen, Wilhelm-Schickard-Institut für Informatik, (August-2001). Diplom thesis

Talks (5):

Kleiner M (August-25-2013) Invited Lecture: B1: Introduction to Matlab and PsychophysicsToolbox, 36th European Conference on Visual Perception (ECVP 2013), Bremen, Germany, Perception, 42(ECVP Abstract Supplement) 2.
Kleiner M (October-2011) Abstract Talk: Fairy tales & horror stories: Common misconceptions and traps about use of computers for psychophysical testing, 12th Conference of Junior Neuroscientists of Tübingen (NeNA 2011), Heiligkreuztal, Germany 10.
Noppeney U, Ostwald D, Werner S and Kleiner M (July-2008) Abstract Talk: The prefrontal cortex accumulates object evidence through differential connectivity to the visual and auditory cortices, 9th International Multisensory Research Forum (IMRF 2008), Hamburg, Germany(189) 118.
Kleiner M, Brainard D and Pelli D (August-2007) Abstract Talk: What's new in Psychtoolbox-3?, 30th European Conference on Visual Perception, Arezzo, Italy, Perception, 36(ECVP Abstract Supplement) 14.
pdf
Chiller-Glaus S, Kleiner M, Hofer F, Knappmeyer B and Schwaninger A (March-2006) Abstract Talk: Erkennung von emotionalen Gesichtsasudrücken in bewegten und statischen Gesichtern, 48. Tagung Experimentell Arbeitender Psychologen (TeaP 2006), Mainz, Germany 220.

Export as:
BibTeX, XML, Pubman, Edoc, RTF
Last updated: Tuesday, 18.11.2014