Importance And Risks of Using Perception Scales for Evaluating IEQ

Despite the ubiquitous usage of perception scales for evaluating indoor environmental quality, there is hardly any public discourse on the benefits, challenges, and risks associated with their use. This article addresses these topics together with specific guidelines and points of attention for the successful implementation and use of perception scales in research and practice.

Keywords: Perception scales, Indoor environmental quality, Individual differences, Reliability, Research biases, Survey, Questionnaire, Post-occupancy evaluation.

Introduction

People spend most of their lives indoors. It is thus important to understand the impact of indoor environmental conditions or indoor environmental quality (IEQ) on people’s health, comfort, well-being, satisfaction, and productivity. For the characterization of the IEQ and its effect on people, several types of assessment methods exist. The methods defined as “objective” a) rely on sensors either measuring specific aspects of the IEQ like air temperature, illuminance, carbon dioxide concentration or sound pressure level, b) measure physiological reactions of people like skin temperature or heart rate, or c) collect their presence and behaviours in buildings (e.g., opening or closing windows). Subjective methods are usually based on questions directed to the users of the indoor environment and vary from quantitative – based on scales – to qualitative approaches – based on open-ended questions. The following describes the importance and potential risks associated with the use of perception scales, which are used ubiquitously in research and practice. This article concludes with a few general guidelines and points of attention for a successful implementation of perception scales for evaluating perceived IEQ in research and practice.

Importance of perception scales

The ubiquitous use of scales and their importance is supported by numerous factors challenging objective and qualitative measures.

The effect of the IEQ on people’s health and productivity could be presumably measured – at least in part – objectively, relying heavily on environmental sensors and quantifiable metrics. A number of potentially hazardous elements in indoor environments such as high carbon monoxide or radon concentrations cannot be perceived by occupants and have to be detected using physical measurements. Likewise, self-reported productivity or performance assessments may not correspond to objectively measurable indicators of productivity. While purely objective approaches are valuable for gathering data on IEQ, they are demonstrating somehow limited potential when it comes to capturing the nuanced and subjective nature of human comfort, well-being, or satisfaction.

Significant individual differences exist in how we, as humans, perceive various aspects of our built environment. This variability influences the effect of the IEQ on people and the way in which people assess IEQ at multiple levels, from physiological and sensory responses to behaviours. This interindividual variability, i.e. difference between responses of individuals under the same environmental conditions and time, arises from a broad range of physiological, psychological, socio-economic, demographic, and cultural factors. For example, in the context of thermal comfort, and from a thermophysiological perspective, differences in morphology, body size and composition, and functionality affect thermoregulation and body temperature. Interindividual differences can also be attributed to sex, age, fitness level, health status and geographic or ethnic acclimatization. In addition, intraindividual variability, i.e. differences in responses that the same person experiences under the same environmental conditions at different points in time stems from a range of factors including thermal adaptation to different seasons, acclimatization throughout the day due to circadian rhythms, altered physiological states for example during the menstrual cycle or when experiencing fever, personal preferences, emotional states, and prior experiences.

Objectively measurable reactions of individuals are physiological signals, so an increasing number of studies is trying to replace subjective scales with them. While first attempts based on data from laboratory studies show potential pathways, research is far from being able to rely on these alone for field study or practice applications because physiological signals like variations in heart rate, heart rate variability, or galvanic skin responses are not showing unique patterns to a single stressor. Even for relatively direct signals like skin temperature signals to thermal perception, the relationship varies according to individual characteristics which affect thermoreceptors’ activity and functioning [1]. It is only through subjective assessments that we can better understand these relationships. Therefore, we must primarily rely on people’s subjective experiences and their corresponding reports about such experiences when we try to assess these states as objective measurements alone cannot account for the diversity of individual perceptions.

Perception scales offer significant benefits for evaluating IEQ compared to subjective but qualitative approaches. While traditional interviews and open-ended questions are useful for gathering detailed background information, they are often time-consuming and complex to analyse. In contrast, scales provide a standardized, quantifiable method to assess how occupants experience and perceive their environment. Unlike qualitative methods, perception scales allow for consistent data collection across different contexts and times, enhancing the reliability of findings. By capturing responses from all occupants, not just those who vocalize dissatisfaction, perception scales provide a more comprehensive understanding of the overall environmental quality. Scales help identify subtle issues that might not yet be severe enough to prompt complaints but could impact comfort, well-being, or productivity over time. Additionally, perception scales can highlight differences in individual sensitivities, preferences, and needs, enabling targeted and effective interventions to optimize IEQ.

Challenges and risks related to the usage of perception scales

Despite their importance and advantages, challenges and risks are related to the selection of the most suitable scales for a given purpose and their interpretation.

When selecting or preparing perception scales for evaluating IEQ, it is essential to ensure they accurately assess what they are intended to measure, capturing the full spectrum of occupants' experiences and reflections. For example, evaluating the subjective thermal experience requires distinct scales for different constructs, such as thermal sensation, comfort, preference, and acceptability. These constructs differ in meaning: thermal sensation refers to an occupant's immediate discriminative experience (ranging from cold to hot, as defined by the ASHRAE thermal sensation scale), while thermal comfort, preference, and acceptability reflect an occupant’s evaluation of their thermal sensation [2]. This distinction is significant because thermal, as well as other domain, evaluations are results of reflection upon the generated sensation and can be influenced by the multitude of above-mentioned factors (Figure 1). Furthermore, single aspects of a domain may not be sufficient. For instance, evaluating the visual experience should not be based solely on the evaluation of light quantity (typically assessed through illuminance), but should include factors like the absence of glare, colour rendering or the quality of the view [3]. Similarly, assessing air quality and acoustic experiences should involve complex variables, such as perceived freshness of air or noise disturbance, which currently lack standardized scales. The absence of clear guidance for such comprehensive scales in non-thermal domains suggests a need for more nuanced and multidomain approaches to capture the full scope of IEQ. In addition, questions and scales may need to be adapted to the audience. For instance, children typically require simplified and clearer questions depending on their age and cognitive ability [4].

Figure 1. Relationship between thermal sensation and thermal comfort votes for 5 clusters of respondents showing the influence on the relationship by personal and environmental factors. (Figure based on Schweiker et al. [5])

Despite the need for adjusting scales to the targeted group, IEQ studies can only lead to scalable and generalizable results, if researchers and practitioners use the respective tools and methods consistently. A recent review of multi-domain IEQ studies suggested that it is common that “a different number of points and different labels were used, even though the same assessment category was involved” [6]. The review also pointed to inconsistent use of dimensions in analogue scales, which all together “disables the comparison of results from different studies and poses a problem for conducting large-scale meta-analyses”. Unfortunately, “the practical fitness and interpretative potential of commonly deployed formats for eliciting and representing people's response remains a formidable challenge. We simply miss a conclusive treatment (e.g., a rigorous meta-study) of the expressive power and consistency of typical scales and formats used in IEQ research” [7] and even current international standards suggest different scales.

Within this line of argument, the quality of scales related to IEQ is seldom assessed by mean of reliability measurements, like Cronbach’s alpha. Their use is essential to consider the internal consistency of questionnaires to ensure that they accurately measure a single construct, which is crucial for maintaining the validity of the data collected and ensuring reliable results across different contexts and populations.

Further challenges arise during data acquisition and analysis to avoid potential biases affecting outcomes. When preparing a questionnaire, semantic bias may arise when a scale is translated into a different language so that the original meaning of thermal comfort scale results is altered (see, e.g., Pitts [8]). When a person fills out a questionnaire, response biases complicate the interpretation of data gathered from perceptual scales. For example, Sadick et al. [9] investigated the reliability of humans as environmental sensors and highlighted the potential for the existence of an acquiescence bias, that is, the tendency to provide positive feedback. Similarly, in a large data collection campaign, a central tendency bias was observed; responders tended to avoid extreme category responses, which can skew the data collected from perception scales and affect model development [2]. Also, cognitive biases on thermal comfort assessment may occur, triggered by social desirability in respondents’ assessments and decision-making processes [10]. Similarly, respondents may employ cognitive shortcuts or conform to social norms [11]. Furthermore, anchoring bias happens when participants may involuntarily adopt previous sensorial scores as benchmarks through repeated reporting, potentially influencing subsequent evaluations based on such established reference points [12].

Moreover, the use of appropriate statistical tools is of paramount importance to obtain reliable outcomes. For example, if the data type is not considered in the selection of the statistical tools (e.g., statistical test, regression type), the results can result artificially altered, as shown by Favero et al. [13].

Summary and guidance

In summary, scales, if carefully designed and applied, provide a structured way to record and document people’s conscious subjective perception of indoor environmental conditions. Specifically, people can develop and express judgments about the intensity of their perceptual experiences in relative terms. They can judge if thermal conditions are warmer or cooler, visual conditions are brighter or dimmer, or acoustic conditions are louder or quieter and whether they perceive such conditions as comfortable or uncomfortable.

In contrast, objective measurements of IEQ parameters or physiological reactions lack the sensitivity to detect how personal stressors, both psychological and environmental, might alter an individual’s comfort perception at any given time. For example, someone experiencing fatigue or stress might react differently to the same IEQ conditions compared to when they are well-rested and relaxed.

A multisensory approach combining objective and subjective measurements is recommended and crucial for capturing the full spectrum of human experience, which should be the key driver to address via an effective architectural and building technology design. This approach involves the use of scales to understand how individuals perceive these factors through various senses—sight, hearing, touch, smell, and thermal perception in combination with measuring environmental parameters. Adding the monitoring of physiological responses in real-time, we can support the identification of how environmental changes affect the human being, tailoring indoor and outdoor environments to meet individual needs and mitigate potential environmental stressors and risks. This combination can be used to establish certain correlations between the objectively measurable intensity of such stimuli (e.g., temperature, luminance, sound level) and the corresponding subjective reports of perceived intensity in terms of scales (e.g., thermal sensation, level of glare, loudness level).

When preparing surveys using perception scales, when designing experiments, and when analysing collected data, care must be taken to reduce the impact of unwanted biases and alterations of outcomes due to fallacious hypotheses. When administering perception scales, it is also important to educate participants about the meaning of the scales using clear examples. This procedure is already followed in some domains, such as glare assessment in visual comfort studies, where the perception of glare can be highly subjective. Providing examples helps standardize responses and reduces ambiguity, leading to more accurate and meaningful data.

Moreover, to capture the broader context of an occupant's experience, perception surveys should also include questions that go beyond immediate environmental factors. For instance, asking about recent sleep quality, caffeine intake, or physical activity can reveal contextual effects influencing an individual's perception and evaluation of their environment. The same thermal sensation, for example, could lead to different evaluations depending on these contextual factors. Such additions would be helpful to go beyond the understanding of average responses towards advancing our understanding of the factors contributing to the wide observed responses’ variance.

Acknowledgements

The authors thank Boris Igor Palella (Università degli Studi di Napoli Federico II), Francesca Romana d’Ambrosio (Università degli Studi di Salerno), Hansjürgen Gebhardt (ASER), Rune Korsholm Andersen (DTU), and Simon Hodder (Loughborough University) for the fruitful discussions within the framework of a more comprehensive research article to be published on this topic.

References

[1]     Vellei, M., et al., Dynamic thermal perception under whole-body cyclical conditions: Thermal overshoot and thermal habituation. Building and Environment, 2022. 226: p. 109677.

[2]     Schweiker, M., et al., Challenging the assumptions for thermal sensation scales. Building Research and Information, 2017. 45(5): p. 572-589.

[3]     Carlucci, S., et al., A review of indices for assessing visual comfort with a view to their use in optimization processes to support building integrated design. Renewable and Sustainable Energy Reviews, 2015. 47: p. 1016-1033.

[4]     Haddad, S., et al. Questionnaire design to determine children’s thermal sensation, preference and acceptability in the classroom. in Conference: PLEA 2012 - 28th International Conference: Opportunities, Limits & Needs, towards an Environmentally Responsible Architecture. 2012. Lima, Peru.

[5]     Schweiker, M., et al., Evaluating assumptions of scales for subjective assessment of thermal environments – do laypersons perceive them the way, we researchers believe? Energy and Buildings, 2020.

[6]     Chinazzo, G., et al., Quality criteria for multi-domain studies in the indoor environment: Critical review towards research guidelines and recommendations. Building and Environment, 2022. 226: p. 109719.

[7]     Mahdavi, A. and C. Berger. Critical reflections on proxies of indoor-environmental quality. in Proceedings of 43rd AIVC-11th TightVent & 9th Venticool Conference. 2023.

[8]     Pitts, A. The language and semantics of thermal comfort. in Windsor Conference. 2006. Windsor, UK: NCEUB.

[9]     Sadick, A.-M., et al., Reliability of human environmental “sensors”: Evidence from first- and third-person methods. Building and Environment, 2020. 186: p. 107303.

[10]   Van de Mortel, T.F., Faking it: social desirability response bias in self-report research. Australian Journal of Advanced Nursing, The, 2008. 25(4): p. 40-40.

[11]   Allcott, H., Social norms and energy conservation. Journal of Public Economics, 2011. 95(9): p. 1082-1095.

[12]   Raccuglia, M., et al., Anchoring biases affect repeated scores of thermal, moisture, tactile and comfort sensations in transient conditions. International Journal of Biometeorology, 2018. 62(11): p. 1945-1954.

[13]   Favero, M., A. Luparelli, and S. Carlucci, Analysis of subjective thermal comfort data: A statistical point of view. Energy and Buildings, 2023. 281: p. 112755.