Serial dependence in static and dynamic orientation stimuli
Supervisor: Dr Christopher Benton
School of Psychological Science, University of Bristol
Abstract
Visual input is inherently noisy, for example due to eye and head movements. Yet, the perception of the visual world is stable and continuous. Fischer and Whitney (2014) suggested that perception may exploit correlations in previous and current visual input to achieve a better signal-to-noise ratio and stable perception. This is known as serial dependence and has recently been extended to oriented motion stimuli (Alais, Leung, Van der Burg, 2017). These authors found an asymmetry, where both positive and negative serial dependence occurred in interleaved static and motion stimuli.
This study aimed to further investigate serial dependence in static and dynamic orientation stimuli and to explore the possibility of two separate processing mechanisms for static and dynamic orientation, as suggested by Alais et al. (2017). Slight attractive serial dependence was found for interleaved stimulus conditions; static – dynamic and dynamic – static. Repulsive serial dependence was found for static – static stimuli and dynamic – dynamic stimuli, which was unexpected. The author suggests reasons for the outcome of this study; most notably that the task used did not measure the required construct and proposes improvements to that effect.
Serial dependence in static and dynamic orientation stimuli
For most animals, vision is an important aspect to carry out everyday tasks, such as foraging for food or navigating the urban environment. The visual world appears stable and continuous, but upon closer examination, one could ask why it should. Eye and head movements, occlusion and shadows are amongst the items that make visual input quite noisy, yet perception is stable. It has been suggested that an effect called ‘serial dependence’ and its associated ‘continuity field’ may be partially responsible for stable visual perception (Fischer & Whitney, 2014).
Serial dependence describes the finding that visual perception in humans seems to be biased by previous and current perception, where perception is systematically drawn towards what was previously seen (Fischer & Whitney, 2014). This bias is spatiotemporally tuned and orientation selective in its ‘continuity field’ (Fischer & Whitney, 2014). The visual system may rely on the fact that the visual world does not change dramatically from moment-to-moment and may integrate information over time to achieve a better signal-to-noise ratio and average out the fluctuations in visual input (Alais, Leung, & Van der Burg, 2017; Fischer & Whitney, 2014).
Serial dependence has been found in various domains, ranging from facial attractiveness judgements (Kondo, Takahashi, & Watanabe, 2012), auditory pitch judgements (Arzounian, de Kerangal, & de Cheveigné, 2017), eye gaze direction (Alais, Kong, Palmer, & Clifford, 2018), to numerosity (Fornaciai & Park, 2018) and perception of timing (Roseboom, 2019). Participants in these studies have shown their perception of a stimulus to be attracted towards the previously perceived stimulus in a consistent manner and these dependencies further support the notion of an averaging process.
Recently, Alais et al. (2017) investigated dependencies in motion perception. Building on the work by Fischer and Whitney (2014), the authors hypothesised that serial dependency would be disadvantageous in dynamic stimuli, as it would reduce sensitivity to change. Alais et al. (2017) found a surprising asymmetry when interleaving motion stimuli with static orientation stimuli, where motion stimuli produced a positive dependency, but static stimuli produced a negative dependency. The authors suggested that there are two separate mechanisms for processing static and dynamic orientations, where static orientation information is able to influence dynamic orientation information, but not vice versa (Alais et al., 2017).
These contrasting results encouraged this investigation into serial dependence in static and dynamic orientation stimuli. This experiment aims to replicate the positive serial dependency established by Fischer and Whitney in static orientation stimuli and to find positive serial dependence in dynamic stimuli, similar to Alais et al. (2017), and to investigate the relationship between static – dynamic stimuli and dynamic – static stimuli. It is hypothesised that only the former should produce positive serial dependence, in line with the idea that only static orientation can affect subsequent dynamic orientation processing (Alais et al., 2017).
Methods
Participants
In total, 11 participants took part in this study, nine females and two males, with a mean age of 22.90 years (SD= 3.23). Participants needed to have normal or corrected-to-normal vision. The University of Bristol’s Faculty of Science Research Ethics Committee approved the study (Ref: 90482) and all participants gave written informed consent. Participants were reimbursed £20 for their time.
Materials
Stimuli were created using the Psychophysics Toolbox for MATLAB (Brainard & Vision, 1997; Kleiner et al., 2007; MATLAB R2019a, MathWorks, 2019; Pelli, 1997) and presented on a 1280- × 960-pixel CRT monitor screen, 400 mm x 300 mm in length and running at 75 Hz, at a viewing distance of 50 cm.
Participants indicated their orientation judgement responses using a Microsoft Sidewinder controller. The right trigger button would be used to achieve clockwise rotation of the response circles and vice versa for the left trigger button. To submit a chosen orientation, participants pressed button ‘A’. Button ‘Y’ could be used to terminate the experiment. Procedure
Participants completed 129 trials each, separated into eight blocks. Participants were given a five-minute break after four trial blocks were completed. There were four types of stimulus sequence; static stimulus – static stimulus, static stimulus – dynamic stimulus, dynamic stimulus – static stimulus and finally, dynamic stimulus – dynamic stimulus. In total, 128 stimulus relationships were investigated, with 32 of each type of relationship. Ten practice trials were introduced for each participant after four had been tested, as participants struggled to understand the task without having practiced it first. The experiment lasted two hours in total.
Participants sat 50 cm away from the CRT screen, using the controller to make their orientation judgements. This study used two types of stimuli; static Gabor patches and dynamic Gabor patches that had a sinusoidal contrast modulation, to achieve a flickering effect. The stimuli sequences were fully randomised.
A disc at the centre of the screen was used as a fixation point, at a visual angle of 1.15° (1 cm). Participants were told to fixate on this point throughout the entire trial length. The disc also served as an indicator for process and response accuracy. As trials progressed, the disc filled in with white, so that at the end of each trial, the entire disc would be white. If participants succeeded in giving an orientation judgment that lay within 10° of the correct orientation, a black dot would flash up in the centre of the fixation disc.
Static Gabor patches demonstrated a peak contrast of 25% Michelson, a 1.5° s.d. Gaussian contrast envelope and a spatial frequency of 0.5 cycles per degree. These were presented at 6.5° eccentricity for 500 ms and followed by a noise patch for one second, all at the same location.
The dynamic Gabor patches had a sinusoidal contrast modulation and had an added temporal frequency of 8 Hz, which resulted in a flickering perception and.
Noise patches were made up of static Gabor stimuli, but with a von Mises concentration of zero, which resulted in all orientations being sampled, with a spatial frequency of 0.5 cycles per degree and were used to minimise the risk of negative aftereffects.
A 250 ms gap followed the noise patch, after which the two orientation circles appeared, also at a visual angle of 1.15°. These two discs were at 180° opposite each other and had the function of a response bar, without the orientation inherent to using an actual bar. The orientation of these two circles could be adjusted using the trigger buttons and their initial orientation was randomised across trials, but always appeared in the same location as the stimulus. There was a two second delay after making the orientation judgement and accepting with button ‘A’, where participants only saw the fixation disc, before the next trial commenced.
Results
The data of participants were pooled and analysed using MATLAB (MATLAB R2019a, The MathWorks, 2019). Participants’ responses were categorised according to the four relationships that were of interest : static – static, static – dynamic, dynamic – static and dynamic – dynamic.
For each condition, an error plot was calculated, where the y-axis denotes the error on the current trial (degrees), and the x-axis denotes the relative orientation of the previous trial (degrees). The error on each trial was calculated as the reported orientation minus the presented orientation (Fischer & Whitney, 2014). The relative orientation was the difference in orientation between the Gabor seen on the current and previous trial (Fischer & Whitney, 2014). Negative error values show that the reported orientation was more counter-clockwise than the actual orientation of the stimulus. Negative x-coordinate values show that the previous trial was more counter-clockwise than the present trial. The amplitude of each error plot indicates the degree to which the participants’ errors were attracted towards the preceding stimulus, in sum; the amplitude value denotes serial dependence. To determine the amplitude, the error plot was fitted with a first derivative of a Gaussian curve.
Figures 1 to 4 show the error plots for the various conditions. Figures 1 and 4 show slight repulsive serial dependence in terms of the fitted first derivative of a Gaussian curve, with amplitude values of .43 and -.50 respectively. This may suggest that the perceived orientation was repelled away and that the stimulus appears to be more different to the previously seen stimulus. Figures 2 and 3 seem to show slight attractive serial dependence, with amplitude values of 1.98 and 1.10 respectively.
Figure 1. Error plot for pooled data of static – static stimulus condition. Negative values as the x-coordinate show that the preceding trial was more counter-clockwise than the current trial; negative error values show that the reported orientation was more negative than the actual orientation. The grey line represents the average error and the red line represents the first derivative of a Gaussian curve fit.
Figure 2. Error plot for pooled data of static – dynamic stimulus condition. Positive values as the x-coordinate show that the preceding trial was more clockwise than the current trial; positive error values show that the reported orientation was more positive than the actual orientation. The grey line represents the average error and the blue line represents the first derivative of a Gaussian curve fit.
Figure 3. Error plot for pooled data of dynamic – static stimulus condition. Positive values as the x-coordinate show that the preceding trial was more clockwise than the current trial; positive error values show that the reported orientation was more positive than the actual orientation. The grey line represents the average error and the green line represents the first derivative of a Gaussian curve fit.
Figure 4. Error plot for pooled data of dynamic – dynamic stimulus condition. Negative values as the x-coordinate show that the preceding trial was more counter-clockwise than the current trial; negative error values show that the reported orientation was more negative than the actual orientation. The grey line represents the average error and the pink line represents the first derivative of a Gaussian curve fit.
Discussion
This experiment investigated serial dependence in static and dynamic orientation stimuli. The aim of this study was to replicate the findings of attractive serial dependence in static orientation stimuli, as first established by Fischer and Whitney (2014) and to investigate whether the latter would extend to dynamic orientation stimuli. Additionally, as suggested by Alais and Leung and Van der Burg (2017), it was hypothesised that static orientation would affect dynamic orientation, but not vice versa.
The results of this investigation do not entirely support the hypotheses. Attractive serial dependence was not replicated in the most basic static-static condition, which was unexpected. In fact, the repulsive serial dependence found may indicate that stimuli appeared more different to one another. This could be interpreted as adaptation taking place, similar to the negative tilt after effect (Fritsche, Mostert, & de Lange, 2017), which would improve perceptual sensitivity to change. Slight repulsive serial dependence was also found for the dynamic – dynamic stimuli condition, which does not support the findings of attractive serial dependence in motion (Alais et al., 2017). Generally, the curves seem to fluctuate around zero and do not seem to show a large, or even medium, effect.
Though attractive serial dependence was found for the static-dynamic and dynamic- static conditions, the curves do not fit all the data points very well. The first derivative of a Gaussian curve seems to be roughly constrained between [-60°, 60° ] orientation and the tail ends of the average error curve are not well described by the Gaussian curve.
The hypotheses that orientation from a static stimulus could affect subsequent dynamic stimulus orientation, but not vice versa, were not truly supported. Attractive serial dependence was found for both cases, which opposes validation of the full hypothesis. This may suggest that there is feedback between both stages of orientation processing, rather than just the static – dynamic stage. This would counter the proposal of two separate mechanisms for processing static and dynamic orientations, as suggested by Alais et al. (2017). However, as attractive serial dependence could not be replicated for static – static stimuli and was also not found for dynamic – dynamic stimuli, contradicting the findings of both Fisher and Whitney (2014) and Alais et al. (2017), one should be careful to over interpret the results of the interleaved conditions for the moment.
In conclusion, this study showed quite noisy data with no clearly identifiable effects. This suggested to the experimenters that the task used to detect serial dependence was flawed, especially as attractive serial dependence was not replicated in the static – static stimuli condition, as was expected. The nature of stimuli used may have encouraged visual adaptation, as the stimulus contrast may have been too high, the display duration may have been too long or the circles of the “response bar” may have unintentionally encouraged an orientation perception of its own. Additionally, the adjustment task may have been too difficult for participants; participants may have lacked motivation to execute it well over the many trial blocks. Therefore, further tasks have been developed and these are being piloted for further research. These tasks focus on static – static stimulus orientation for now, in order to find a task that reliably produces serial dependence and it employs a measure of subjective point of equality, as used by Alais et al. (2017). Once a working task has been found, it shall hopefully be modified to investigate orientation from dynamic stimuli and serial dependence.
Technical skills gained:
This scholarship has provided me with valuable insight into the field of psychophysics. I have learnt basic use of MATLAB and Linux to input commands and visualise data. I have also gained an understanding of why and when to use bootstrapping and confidence intervals. Other skills gained:
Throughout my eight weeks, I have gained a further understanding of how to set up an experimental study; from the theoretical conceptualisation, the process of obtaining ethics approval, recruiting and testing participants, to the evaluation of data. I have also acquired an appreciation of the use of piloting tasks before rolling them out to many participants, in order to make sure that the task measures the construct you are trying to detect and provides sensible data.
Generally, I have been able to engage with the topics of visual perception and motion in detail, which my second-year modules have not allowed to such extent. It was very enjoyable to learn about a specific area of psychophysics and to stretch myself further, in terms of my reading, understanding and thinking critically and to absorb as much of the research environment and experience as possible. This scholarship has provided important preliminary research, which will be expanded upon in my third-year dissertation.
Acknowledgements
I would like to thank the Applied Vision Association (AVA) for providing me with this fantastic opportunity and for supporting my research topic. It has encouraged me further to pursue a career in academia and I look forward to investigating this research topic in more detail throughout my dissertation. I would like to thank my supervisor, Dr Christopher Benton, for all his support, endless encouragement and time for discussions and explanations. I would also like to thank Geoff Gallagher, for his general guidance and support, all his help with MATLAB and for providing me with such a kind and positive environment to complete my internship in.
References
Alais, D., Kong, G., Palmer, C., & Clifford, C. (2018). Eye gaze direction shows a positive serial dependency. Journal of vision, 18(4), 11-11.
Alais, D., Leung, J., & Van der Burg, E. (2017). Linear summation of repulsive and attractive serial dependencies: Orientation and motion dependencies sum in motion perception. Journal of Neuroscience, 37(16), 4381-4390.
Arzounian, D., de Kerangal, M., & de Cheveigné, A. (2017). Sequential dependencies in pitch judgments. The Journal of the Acoustical Society of America, 142(5), 3047-3057.
Brainard, D. H., & Vision, S. (1997). The psychophysics toolbox. Spatial vision, 10, 433-436. Fischer, J., & Whitney, D. (2014). Serial dependence in visual perception. Nature neuroscience,17(5), 738.
Fornaciai, M., & Park, J. (2018). Serial dependence in numerosity perception. Journal of vision, 18(9), 15-15.
Fritsche, M., Mostert, P., & de Lange, F. P. (2017). Opposite effects of recent history on perception and decision. Current Biology, 27(4), 590-595.
Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., & Broussard, C. (2007). What’s new in Psychtoolbox-3. Perception, 36(14), 1.
Kondo, A., Takahashi, K., & Watanabe, K. (2012). Sequential effects in face-attractiveness judgment. Perception, 41(1), 43-49.
MathWorks, T. (2019). MATLAB R2019a. Natick, USA.
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial vision, 10(4), 437-442.
Roseboom, W. (2019). Serial dependence in timing perception. Journal of experimental psychology: human perception and performance, 45(1), 100.