THE VISUAL SYSTEM'S CODE IS MATCHED TO THE PROPERTIES OF NATURAL SCENES
David J. Tolhurst,
Department of Physiology, Downing Street, Cambridge CB2 3EG, UK
It is often proposed that the stimulus-response specificity of single visual neurons makes those neurons especially efficient at coding the features found in natural visual scenes. The proposal is, in fact, much more prevalent than is justified by any real experimental evidence, while theoretical approaches do not all agree on the definition of the word "efficient". There seem to be two kinds of experimentally-based approach.
First, we can look at the stimulus specificity of single neurons or of populations of neurons, and ask whether these neurons would respond optimally to features that are really found in natural scenes. For instance, visual neurons have a limited contrast response range, between threshold and response saturation; does this limited range match the range of contrasts actually found in the environment? The second experimental approach is psychophysical. We can measure performance in visual tasks that involve stimuli derived from photographs of natural scenes. We can make the statistics of these stimuli deviate more or less from those of truly natural scenes; or we can make surrogate stimuli from, say, random dots and then force statistical structure into the stimuli to mimic certain aspects of natural scenes. Are there any important visual tasks that are performed best when stimuli have natural rather than unnatural statistics, suggesting that the visual system really does work best with natural scenes?
LOCAL CONTRAST IN NATURAL SCENES
Nuala Brady1 and David J. Field2
1Department of Psychology, University of Manchester, Manchester M13 9PL, UK
2Department of Psychology, Cornell University, Ithaca, NY 14853, USA
Natural scenes exhibit a number of statistical regularities to which the visual system is tuned. The present study investigates the organisation of locally defined contrast in natural images, and relates this to cortical coding of contrast. Fifty-six images were analysed using log Gabor filters whose orientation and frequency bandwidths were chosen to match those of striate cortical cells. The images were log transformed so that the filters responded linearly to a luminance ratio or contrast, and the response of each sensor was calibrated relative to its response to a sinusoidal grating of optimal frequency and orientation. This provides a measure of local contrast in natural scenes which is interpretable in terms of the Michelson contrast of a grating stimulus. There are a number of ways in which the visual representation of contrast appears to be optimal. First, the contrast distribution was used to derive a response function which maximises differential sensitivity to contrast in natural scenes, and this is shown to be similar to the contrast-response functions of striate cells. Secondly, the way in which contrast is encoded at different spatial scales may be related to the scale invariant nature of images. Finally, the range and variability of local contrasts within the population of scenes suggests a need for changes in the dynamic range of contrast sensitive cells such as achieved by 'contrast normalization'.
THE CODING OF NATURAL SCENES IN PRIMARY VISUAL CORTEX.
Oxford University, Laboratory of Physiology, Parks Road, Oxford OX1 3PT.
Visual neurons have evolved to respond to complex natural scenes. However, we traditionally characterise the tuning properties of these cells using simple artificial stimuli such as sinewave gratings. In order to validate the use of such stimuli for determining neuronal selectivity and bandwidth, we need to relate these results to the responses to natural scenes. Although there has been some work on the response distributions of neurons to natural scenes (Dan et al. 1996; Baddeley et al. 1997; Gallant et al. 1998), there has not yet been a comprehensive comparison between the coding of natural scenes and the neuronal classification derived from sinewave gratings.
We recorded from neurons in the primary visual cortex of ferret. Each neuron was characterised for selectivity and bandwidth using drifting sinewave gratings. Then repeated sequences of flashed stationary natural scenes were presented. Using reverse correlation across the image sequence, orientation selectivity could be determined, while bandwidth equivalents were inferred from the shape and statistics of the spike-count distributions. We found considerable variability across our population of neurons. I will present our results on the properties of responses to natural scenes in relation to cell type and the traditional measurements of bandwidth and selectivity.
Dan et al., J. Neuroscience 16:3351, (1996)
Baddeley et al., Proc. R. Soc. Lond. B 264:1775, (1997)
Gallant et al., NeuroReport 9:85, (1998)
THE SECOND-ORDER CONTENT OF NATURAL IMAGES.
Andrew J. Schofield
School of Psychology, University of Birmingham, Edgbaston, Birmingham, B15 2TT
The human visual system is sensitive to both first-order variations in luminance and second-order variations in local contrast. Although there is some debate about the nature of the second-order system and its relationship to first-order processing there is now a body of results showing that the two types of image information are processed separately in the initial stages of vision. However, the amount and nature of second-order information present in the natural environment is unclear. This is an important question because if naturally arising second-order signals carry little information in addition to the first-order signals then the notion of a separate second-order system would lack ecological validity.
A generic model of second-order vision was applied to a number of well calibrated natural images. This model consisted of a first stage of oriented spatial filters followed by a rectifying non-linearity and then a second set of filters. The connectivity between first- and second- stage filters was varied to simulate some of the models that have been proposed for second-order vision. Output images taken from this model indicate that natural occurring second-order signals carry information that cannot be revealed by linear first-order processing. Specifically, the second-order system reveals variations in texture and features defined by such variations.
(Supported by BBSRC grant no S03969. The author wishes to thank Mitch Thompson for supplying the calibrated natural images).
SPATIOCHROMATIC STATISTICS OF LOW-DIMENSIONAL REPRESENTATIONS OF NATURAL COLOURED IMAGES: PRELIMINARY RESULTS
Mitchell Thomson1, Steve Westland2,
1Vision Sciences, Aston University, Aston Triangle, Birmingham B3 7ET
2Dept of Communication and Neuroscience, Keele University, ST5 5BG
Although the spatial characteristics of monochromatic images have been analysed and models for their statistical properties elaborated, and the coding of colour information in the retinal and post-retinal human visual system has been studied extensively, there has arguably been little work on the relationship between the spatiochromatic properties of coloured natural images and human visual processing (although see Burton & Moorhead, 1987).
As a prelude to a major study on the statistics of coloured natural images, we decided to conduct a preliminary analysis of the spatial and chromatic properties of a number of natural scenes. These scenes were acquired under several illuminants using a high-spatial-resolution three-chip colour camera; techniques for calibrating such a device (Thomson & Westland, 1998a)and for increasing the dimensionality of the surface-colour representation (Thomson & Westland, 1998b) are presented elsewhere.
A linear-systems technique related to principal components analysis (Maloney, 1986) was used to decompose the image colour signals into a small number of neariorthogonal bases in wavelength space. By considering the projection of the colour signal at each pixel onto these bases, we were able to recode the colorimetric properties of image sequences in a low-dimensional feature space; this made it possible to disconfound image-intrinsic spatiochromatic correlations from those correlations which would be introduced by a highly correlated colour basis. Second-, third- and fourth-order statistics were computed from the optimized basis; some measures on these statistics were consistent from image to image, a result which may have important consequences for the efficient encoding of coloured natural images by the human visual system.
Burton G J, Moorhead I R 1987 Color and spatial structure in natural scenes. Appl. Optics 26 No.1:157-170
Thomson M G A, Westland S W 1998 Color camera calibration by parametric fitting of sensor responses. Submitted to Color Res. Appl.
Thomson M G A, Westland S W 1998 The intrinsic dimensionality of surface-colour representations under artificial illlumination. Perception, in press (ECVP abstract).
Maloney L T 1986 Evaluation of linear models of surface spectral reflectance with small number of parameters. J. Opt. Sc. Am. A3 No. 10:1673-1683
DETECTION OF PHASE-BASED DISTORTIONS IN FACES AND SINUSOIDAL GRATINGS
Dept of Optometry and Vision Sciences, University of Wales, College of Cardiff, Redwood Building, King Edward VII Avenue, Cardiff, CF1 3XF
Peripheral performance in many simple visual tasks can be equated with that of the fovea by size-scaling, whilst complex tasks and those involving discriminations based upon phase information have produced less clear results. Due to the functional nature of the visual system, facial images are assumed to have an increased relevance compared to more abstract images such as sinusoidal gratings, perhaps being reflected in differing performance variations across the visual field. Thus, human ability to detect phase-randomisation distortions in sinusoidal gratings and facial images was measured at the fovea and at eccentricities up to 10o. Results showed that discriminations based upon phase-randomisation were scaleable for both face and grating stimuli. The fact that this task could be size-scaled implies it is mediated by early cortical mechanisms. Thus, phase-based changes can be detected at an early stage in the visual system with both foveal and eccentric viewing, and any differences between faces and gratings, in terms of additional relevance, do not occur until beyond this level.
NATURAL IMAGE STATISTICS AND VISUAL PROCESSING. ARE THEY MATCHED?
C A P rraga1, D J Tolhurst2, T Troscianko1
1Perceptual Systems Research Centre, Department of Psychology, University of Bristol, 8 Woodland Road, Bristol BS8 1TN, UK
2Department of Physiology, University of Cambridge, Downing Street, Cambridge CB2 3EG, UK
It has been suggested that the overall organisation of the visual system, including the response properties of individual neurons, might be optimised for encoding the statistical information content of natural scenes. However plausible it might be, the suggestion still remains to be fully validated experimentally. Here we propose a new method for investigating whether the presence of natural statistics does indeed optimise the discriminability of natural scenes. Our aim is to use a set of stimuli which, while plausible, still allows good experimental control. A morphed sequence of natural scenes was presented to observers in a modified two-alternative forced-choice experiment. They were asked to discriminate between reference (original) images and a slightly morphed version of these. Discrimination thresholds were obtained by fitting the measured psychometric function with the integral of a normal distribution. The statistics of each morphed sequence were manipulated by controlling the falloff of Fourier amplitude with spatial frequency (alpha), and thresholds for morphed sequences with different alpha values were measured. Eleven different conditions were explored with amplitude slopes ranging from -0.5 (whitened or edge-enhanced pictures) to -2.5 (blurry pictures). The results show that morphed scenes having an alpha value close to that reported for natural scenes (alpha= -1.2) are optimally discriminated by the human visual system. We conclude that natural stimuli are optimally discriminated, and suggest that this method may be suitable for more general investigations with naturalistic stimuli.
A NEURAL-NETWORK VIRTUAL-REALITY MOBILITY AID FOR THE SEVERELY VISUALLY IMPAIRED
B Thomas1, M Eeveringham1, T Troscianko2, N Karia3, D Easty4
1Advanced Computing Research Centre, University of Bristol, Bristol BS8 1UB, UK
2 Perceptual Systems Research Centre, University of Bristol, Bristol BS8 1UB, UK
3 Department of Psychology, University of Bristol, Bristol BS8 1UB, UK
4 Department of Ophthalmology, University of Bristol, Bristol BS8 1UB
Many people who are registered as blind nevertheless retain some residual vision and are said to have `low vision'. Conditions resulting in such low vision include cataracts, diabetic retinopathy, age-related maculopathy, and retinal detachment. In recent years principles from computer vision have been increasingly applied to the requirements of the low-vision subject. A variety of conventional image-processing techniques have been used to enhance the visual appearance of a scene, and devices from the field of virtual reality such as head-mounted displays have been investigated as an aid to low vision. However, a fundamental limitation with conventional image-processing techniques is that they are applied to an entire image with no knowledge of scene content, resulting in unwanted emphasis of noise and unimportant detail.
Our aim is to produce a portable system comprising a processing unit with head-mounted camera and display which will allow a person with low vision to be self-sufficient and mobile in a typical urban environment. Our approach differs from previous research in that it uses a neural-network object classifier to allow images to be enhanced in a way which considers the identity of objects in the scene. Primarily, our system transforms an original image into a classified image in which the types of objects in the scene are identified by an object outline filled with a particular high saturatin colour according to the object type, chosen by he user. By classification our system allows the user to identify important objects in a scene simply by their colour, requiring no erception of shape or high spatial frequencies, and minimal contrast sensitivity. The resultant images are very simple and ncluttered and we expect that users would adapt quickly to the system. Results obtained to date suggest that the system is apable of providing registered-blind users with useful visual information. We are now working on improving the speed and classification accuracy of the system, and investigating the applicability of our techniques to specific conditions.Registration (click on one of the buttons below):