Reliability of a standard test for wheelchair cushion envelopment characteristics
Jenna Freedman, Patricia Karg, David Brienza, Sandra Arias-Guzmán, Keerthana Kalliat, Alexandra Delazio
University of Pittsburgh, Department of Rehabilitation Sciences and Technology (Pittsburgh)
Pressure injuries are damage to the skin and underlying tissue due to prolonged pressure, typically near bony prominences. [1, 2] Due to reduced mobility and a large portion of time spent in the seated position, wheelchair users are particularly prone to develop pressure injuries.  Treatment for pressure injuries can also be quite costly. A poor distribution of load is an extrinsic risk factor for pressure injuries, but the risk can be lowered by improving the load’s pressure distribution. Immersion and envelopment are two measures used to characterize the distribution of a load on wheelchair cushions. 
Immersion indicates the relative stiffness of a cushion. It is defined by the depth at which an object sinks into a support surface. Ideally, pressure is reduced at a site of a bony prominence as the bony prominence increasingly immerses into a cushion. A support surface’s ability to conform around the contours of an object defines envelopment. A cushion with greater envelopment properties allows for a greater distribution of loads and a reduction of peak pressures.  Only limited research has been conducted on the impact of these two characteristics on wheelchair cushions.  The International Organization for Standardization (ISO) published a standardized method (ISO 16840-12) on measuring the immersion and envelopment properties of cushions, which was updated and adopted as a RESNA standard. [5, 6] The standard was developed to improve methods used by the Centers for Medicare and Medicaid services (CMS) for coding verification of “adjustable skin protection” cushions. Currently, coding language and tests allow for products of varying performance in this category. This paper presents the results of the application of the standardized method on 20 cushions, and the intralaboratory repeatability and reliability. Additionally, it discusses and compares cushions for their ability to accommodate changes in size and mass.
This testing was conducted based on the protocol described by the RESNA WC-3, Section 12 standard.  Twenty-one cushions of varying materials, including air, foam, gel, honeycomb structured, and a combination of materials, were tested. This sample of cushions is representative of the six fundamental Healthcare Common Procedure Coding System (HCPCS) codes; a minimum of three cushions represent each code. Cushions were set up based on manufacturer procedures. One air cushion was unable to accommodate the indenter and loads for this test without bottoming out (cushion D) and was excluded.
Two rigid, dual, semispherical indenters (radii of 220 mm and 255 mm) were constructed with 18 pressure sensors (TD101.0463.9898.00.U, STS, USA). A gel pad was placed on each sensor surface per the standard to improve load transfer from the test surface. All pressure sensors were calibrated prior to testing. Figure 1 depicts the instrumented indenter and the grouping of sensors into four elevations. Elevation 1 occurred at the lowest points on the indenter representing the ischial tuberosities (ITs), while Elevation 4 corresponded to the trochanters. Each elevation is separated by a height of 5 mm. Immersion and average pressures at the four elevations were obtained for each cushion using the two different indenters at two different loads.
A standard load of 425 N and an overload of 525 N was applied to each cushion using the indenters and a loading rig (Alliance RF 100, MTS Corporation). The standard load of 425 N was applied for 130 seconds, after which the load increased to 525 N for 60 seconds. This was repeated three times for each indenter. Cushions were allowed to recover for a minimum of 5 minutes between trials.
Cushion thickness was recorded before applying the loads. To determine the immersion for each load, the thickness beneath the basepoints of the indenter was subtracted from the cushion thickness. For each sensor, pressure data were collected and averaged for the last 10 seconds of each loading period. Subsequently, the four elevation metrics were calculated by averaging the corresponding pressure sensors (Figure 1). These five measurements were calculated for each loading conditions (i.e., large indenter at standard load, large indenter at overload, small indenter at standard load, and small indenter at overload).
The repeatability coefficient (RC) and intraclass correlation coefficients (ICC) were computed to examine the test’s reliability and reproducibility. The RC, also defined as test-retest reliability, signifies the repeatability of a metric. The RC is a precision measure representing the 95% confidence interval for the difference between two measurements that are collected using the same methods. As described by Bland and Altman, the RC corresponds to the within-subject standard deviation, sw, multiplied by 1.96 and the √2. A one-way analysis of variance (ANOVA) was run to determine the within-subject standard deviation of each metric.  ICCs quantify the degree to which groups resemble each other in terms of a quantitative characteristic. To calculate the ICC for each metric, a two-way mixed effects model with absolute agreement at a 95% confidence interval was used. This analysis produced two outputs, a single measure ICC and an average measure ICC. The single measure ICC describes the reliability of using a single repetition as the outcome of the test, while the average measure ICC describes the reliability of averaging several repetitions of the test. [8, 9] A cushion’s capacity to adjust to the varying loads and indenter sizes was examined by calculating the differences between the load conditions.
The average immersion results for each cushion under the four loading conditions are displayed in Figure 2. The lowest immersion values tend to correspond to the large indenter at standard load while the highest immersion values correspond to the smallest indenter at overload. Since the large indenter has the smallest force-to-area ratio at the standard load and the small indenter has the highest force-to-area ratio at the overload, the immersion pattern follows expectations. The high ICC values and low RC values displayed in Table 1 indicate a high repeatability of trials for the immersion metric. The RC values for the small indenter are lower than the large indenter, indicating more variability within cushions using the large indenter.
The elevation metrics describe the load distribution and the amount the cushion envelops the indenter. A representation of the envelopment metrics is displayed in Figure 3a. It contains the elevation values of the large indenter at standard load (425 N) for each cushion. Typically, the largest pressure values are at Elevation 1, the ITs, and the lowest pressure values at Elevation 4, the trochanters. This pattern is seen with all four loading conditions apart from a few cushions. As seen in Figure 3a, cushions L, R, and U do not follow this pattern; Cushion L deviates the most and offloads pressure from the ITs. Load distribution between cushions can also be compared using Figure 3a. For example, Cushions U and P nearly distribute the load uniformly across the elevations, while cushion F has a large discrepancy amid elevations. Figures 3b and 3c illustrate the difference in the corresponding pressure parameters between the load conditions. A cushion with smaller differences suggests a greater ability to adjust to varying load conditions. The high ICC values in Table 2 demonstrate the high repeatability of the elevation metrics. The RC values varied across loading conditions, indicating varying precision.
The five test metrics - Immersion, Elevation 1, Elevation 2, Elevation 3 and Elevation 4, are demonstrated to have high repeatability and reliability from the calculated ICCs. Based on the 95% confidence interval, values of 0.90 and higher are considered to have excellent reliability.  The single measure ICC represents the reliability of a single trial. Under the four loading conditions, the single measure ICCs were at least 0.94 indicating that the test produces accurate individual trials. The average measure ICCs (at least 0.98) were high amongst the different conditions. This suggests that the protocol is more reliable and repeatable if data from multiple trials is averaged.
The RCs demonstrate that the immersion at overload has a lower variability than the immersion at standard load for both the small and large indenter. Due to the relatively low RC values compared to the respective data ranges for each metric, the immersion RCs also validate the repeatability of the metric. The RCs for the elevation metrics varied, with the least precision occurring at Elevations 1 and 4 for the small indenter. Given that the metrics produced high ICCs and low RCs compared to the range and generally high variability of the measures across cushions, it can be concluded that the test is able to distinguish between cushions. However, Elevation 1 and Elevation 4 of the small indenter are more limited than the other metrics in distinguishing between cushions. 
The cushions’ capacities to adapt and adjust to different load conditions can be examined through comparing the differences in the elevation pressures between the two load magnitudes (Figure 3b) and indenter sizes (Figure 3c). Cushion L (skin protection and positioning cushion) and cushion P (adjustable, skin protection cushion) demonstrate the smallest differences in pressure values. Since these cushions demonstrate small pressure differences under both conditions, it can be inferred that the cushions have a greater capability to adjust to “macro” changes, like the varying loads. Additionally, the small differences suggest that these “macro” changes do not appear to greatly affect the load distribution pattern.
Limitations of the data collection include the unknown effect of adding the gel to the surface of the pressure sensors. A limitation of the testing method is the inability to test some cushions (cushion D). There is still research needed to improve the test method as well as examine the value of the information the test produces. Future research should be conducted comparing the repeatability of the protocol between two tests of the same set of cushions, two sets containing different samples of the same models of cushions, and interlaboratory. There may be an interest in further exploring the relationships between immersion and envelopment metrics, and different types of metrics that may be more sensitive to cushion variations.
The contents of this paper were developed under a grant from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR grant number 90REGE0001). NIDILRR is within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). The contents do not necessarily represent the policy of NIDILRR, ACL, or HHS, and you should not assume endorsement by the Federal Government.
 Grey, J. E., Harding, K. G., & Enoch, S. (2006). Pressure ulcers. BMJ (Clinical research), 332(7539), 472-475.
 Taule, T., Bergfjord, K., Holsvik, E. E., Lunde, T., Stokke, B. H., Storlid, H., ... & Rekand, T. (2013). Factors influencing optimal seating pressure after spinal cord injury. Spinal Cord, 51(4), 273.
 Wantanabe, L. (2017, September 1). IMMERSION, ENVELOPMENT AND OFF-LOADING. Mobility Management. Retrieved from https://mobilitymgmt.com/Articles/2017/09/01/Seating-Strategies.aspx
 Linden, M., & Sprigle, S. (2007). Methodology to Measure the Adjustability of Skin Protection Features of Wheelchair Cushions.
 International Organization for Standardization (2015). ISO/TS 16840-12: Wheelchair seating -- Part 12: Apparatus and method for cushion envelopment testing. Geneva, Switzerland.
 RESNA (2018). RESNA American National Standard for Wheelchairs - Volume 3: Wheelchair Seating. Rehabilitation Engineering and Assistive Technology Society of North America. Arlington, VA.
 Bland, J. M., & Altman, D. G. (1999). Measuring agreement in method comparison studies. Statistical methods in medical research, 8(2), 135-160.
 Bafana, R. P. (2005). Development, evaluation and implementation of wheelchair seat cushion testing standards (Master’s Thesis, University of Pittsburgh).
 Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of chiropractic medicine, 15(2), 155-163.