Modelling human experience

Every interface makes assumptions about the person using it: what they will notice,
what they can hold in mind, how much they can be asked to take in at once, and how
long they will tolerate waiting for it. Those assumptions are usually implicit, and at
the margins they are usually wrong. We make them explicit, as computational models of
perception and cognition that a system can be designed and evaluated against.

Our models are visual, because that is where the ground truth is richest. We design
neural networks and training strategies that respect the pooling from local to global
image perception, and we build psychophysical properties of vision into the
architecture, which markedly increases the robustness of the resulting sensory models.
Integrated into image and video compression, models of this kind deliver bitrate
savings of sixteen per cent through encoder-only adaptation, and because the expensive
model evaluation can be taken out of the mode-decision loop they run in real time,
which makes perceptually informed irrelevance reduction practical for energy-efficient
media delivery.

Measures fitted to aggregated human ratings inherit whatever the aggregate conceals.
We have shown systematic blind spots in distribution-based metrics when they are held
against human perceptual judgement of generated faces, and we treat a disagreement
between a model and an observer as the more informative case, not as an outlier.