Using INTEGRATE-HTA for the assessment of videorasterstereography in idiopathic scoliosis

Abstract

Objective

For the assessment of videorasterstereography compared with radiography for monitoring idiopathic scoliosis, new tools given by the INTEGRATE-HTA methodology were included within the frame set by the authority. In contrast to the usual linear approach by covering the single domains' safety and efficacy, costs, and ethical, legal, socio-cultural and organizational issues (ELSOI) separately, the methodology is based on the assumption of interactions of different aspects of the domains. From the very beginning, these interactions are captured systematically using various tools and are repeatedly discussed with stakeholders.

Methods

Relevant databases were systematically searched for studies and question catalogues were processed in order to be able to search for specific aspects in orienting bibliographic research. An initial logic model was developed and successively expanded. A synoptic table showing multiple mapping of aspects to domains, a complexity checklist, and a semantic complex highlighting the socio-cultural impact of the disease were newly developed.

Results

Four diagnostic studies with high risk of bias were included. For the clinical outcome, Cobb angle standard deviations up to 8.2 degrees and average measurement differences up to 8.8 degrees were determined. These differences in measurement accuracy correspond to those observed for purely radiographic measurements in other studies. In addition to freedom from radiation as the greatest advantage, there were clear arguments in favor of videorasterstereography when considering ELSOI, for example less harm.

Conclusions

Using INTEGRATE-HTA contributes to identify interactions between domains and to significantly expand the perspective on a technology. It improves patient-centered understanding and facilitates discussions.

Background

INTEGRATE-HTA is a completed EU project to develop concepts and methods for a patient-centered integrated assessment of complex technologies (1;2). Technologies are considered complex if they are characterized by an increased number of interacting components, the number and type of groups or organizational levels affected by an intervention, different behaviors of stakeholders, variability of outcomes, and the degree of flexibility (3).

In contrast to the usual linear approach by partial treatment of the individual domains, the INTEGRATE-HTA methodology is based on the assumption of the interactions of different aspects of the domains. From the very beginning, the interactions are systematically recorded by going back and forth between the five steps using various tools and are repeatedly discussed with stakeholders. Step 1 consists of defining the HTA objective and stakeholders, step 2 comprises the development of a logic model, which allows the identification of single aspects and context from a system perspective, step 3 includes evidence collection and assessment, step 4 finalization of the logic model, and step 5 deliberative decision making.

Videorasterstereography is a purely light-optical procedure without any release of ionizing radiation for the three-dimensional surface measurement of spinal deformities. Using a light projector, a slide with a grid of horizontal lines is projected onto the patient's back, which is recorded by a camera. The changed grid is registered by software and a mathematical form analysis generates a three-dimensional image of the surface (virtual plaster cast). The result is a plastic illustration of the deformity and a graphic representation of the frontal and lateral projection including the surface rotation (4). In order to investigate comparable technologies and measurement parameters, we focused on the Formetric System, which is distributed by only one manufacturer in Germany (Diers International, Wiesbaden).

Idiopathic scoliosis is a three-dimensional axial deviation of the spinal column without known cause, which is a relatively common malformation with an incidence of more than 1 percent (5). Depending on its severity, the disease is treated conservatively by physiotherapy, corset therapy, or surgical measures. Regular monitoring is essential to adjust the therapy in case of progression. In adolescent scolioses, the risk of progression increases depending on age, gender, skeletal maturity, and the angle of curvature of the scoliosis (6).

This article describes our experience with the INTEGRATE-HTA methodology for the research question: Can videorasterstereography replace a radiological examination in the aftercare of idiopathic scoliosis? (7).

Methods

New Tools Given by and Developed within INTEGRATE-HTA and Stakeholder Involvement

According to the INTEGRATE-HTA methodology and by intensive exchange of ideas and “reflective thoughts,” we developed an initial logic model, which was steadily expanded during the course of the HTA process until its final version (8;9). The integration of domains took place in a cycloid manner and by recurring reflection about the structure of the logic model every time when discovering a new interaction or using a new checklist or question catalogue.

In order to review the complexity of the technology in general, a complexity checklist was designed as a new tool for an initial assessment based on Wahlster et al. (9) as well as a synoptic table for the integrated assessment across domains (Supplementary Table 1), which shows the main aspects identified with multiple assignments to the individual domains and the graphics designed specifically for this purpose to illustrate the interactions of individual aspects of domains.

In addition, text mining on ELSOI in relevant publications on the diagnosis of idiopathic scoliosis using videorasterstereography was performed, and the results were graphically presented in a harvest plot. For further investigation of the socio-cultural significance of the disease, the tool “semantic complex” was developed.

Experts from radiology, ethics, law, and economics were regularly involved throughout the entire HTA process to discuss the logic model, individual aspects, the interaction of aspects and domains, and to edit texts. In addition, stakeholders were interviewed at a patient meeting even before the database searches regarding the logic model, relevant aspects of the disease for body, soul, and social participation and asked to weight selected aspects of the disease and the two technologies as well as to identify additional possibly relevant subgroups. After the meeting, the contacts were used for occasional queries by e-mail, and the manufacturer was interviewed several times by telephone.

A request was made via the German Federal Association for Scoliosis Self-Help (GSASSH) for a data set to the German Asklepios Katharina-Schroth-Clinic, specialized in scoliosis, in order to be able to derive statements on adherence in corset therapy by means of calculations.

A survey of orthopedists in private practice with experience in scoliosis treatment by e-mail was used to obtain information on the availability of videorasterstereographic devices in Germany. Information on the orthopedists interviewed was taken from a list on the Web site of the GSASSH.

Data Sources and Searches

According to the Standards for Reporting of Diagnostic Accuracy (STARD) guideline (10), the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement (11), and the PICO scheme, the following criteria for study inclusion were determined: patients of all ages with a diagnosis of idiopathic scoliosis and a Cobb angle of at least 10 degrees were included. The index test was videorasterstereography and the reference test conventional X-ray, both for diagnosis in the follow-up of idiopathic scoliosis. The outcome parameters were mortality, morbidity, health-related quality of life, health-related social function level, disease management, and adverse events. The Cobb angle is the decisive outcome parameter for or a therapeutic decision in idiopathic scoliosis and was, therefore, defined as a clinical outcome parameter. Studies had to be available as full-text publications. The systematic searches were carried out in MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and in study registers via ClinicalTrials.gov and International Clinical Trials Registry Platform Search Portal. In parallel, a search for relevant systematic reviews was conducted in MEDLINE, Embase, Cochrane Database of Systematic Reviews, and HTA Database.

A systematic search for health economic studies was also conducted in MEDLINE, Embase, and the HTA database. For the ELSOI domains, various question catalogues were thematically processed in order to be able to search for specific aspects and questions by means of orienting bibliographic research (12–16). The research was carried out in MEDLINE, ETHMED, EthxWeb, “Juris Rechtsportal,” and in regional registers, laws, ordinances, and interest-based sources of information (for complete searches, see (7)).

Study Selection, Data Extraction, Quality Assessment

Two reviewers independently screened titles, abstracts, and full texts and decided about the eligibility of articles. Any disagreements were resolved by discussion. For the clinical domain, we extracted the following data: first author, year of publication, sample size, patients’ characteristics, study objective, study location, and outcome parameter. Where necessary, authors were contacted for further information. The data extracted was cross-checked. The risk of bias was assessed using the tool QUality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 (17).

Results

The search for benefit assessment yielded 215 entries, of which four publications on diagnostic studies (18–21) met the predefined inclusion criteria.

Risk of Bias of the Diagnostic Studies

The risk of bias was rated as high for all studies, primarily due to an unclear selection of patients. This potential selection bias at the beginning has an impact on all aspects evaluated with the QUADAS-2 instrument: The index test and reference standard can be performed only on the population affected by selection bias; therefore, the results are contaminated. The same applies to patient flow and timing. Furthermore, the significance of the results is weakened by a monocentric design (except in Knott et al. (20)) and the small to very small study populations.

Characteristics, Populations, and Results of the Diagnostic Studies

In all studies (Supplementary Table 2), videorasterstereography was compared with conventional X-ray. Three studies were carried out in Germany at the University of Münster, and the study by Knott et al. in 2016 (20) was performed in six centers in the USA and one center in Germany. In three studies, the patients included had a Cobb angle below 50 and 52 degrees. Only in the study by Hackenberg et al. (19), in which the patients underwent surgery, the Cobb angle ranged from 36 to 92 degrees. The age of the patients in three studies ranged from 8 to 31 years. In the 1994 study by Drerup and Hierholzer, no information was given on the age of the patients. In the studies by Knott et al. (20) and Liljenqvist et al. (21), a majority of the patients were female (n = 148 of 193 and n = 81 of 95, respectively); in the other two studies, no information on gender was provided.

In all four diagnostic studies, videorasterstereographic examinations in the standing position and radiometric spinal radiographs in the standing position were performed on the same day.

The standard deviations for the Cobb angle between the radiographic and the videorasterstereographic measurements were between 7.9 and 8.2 degrees in Drerup and Hierholzer (18) and Liljenqvist et al. (21). In the study by Knott et al. (20), the average difference between the measurement procedures was 5.8 degrees for the thoracic scoliosis curve and 8.8 degrees for the lumbar scoliosis curve. In the study by Hackenberg et al. (19) with pre- and postoperative measurements, the videorasterstereographically determined kyphosis angle after surgery was 33.3 percent larger than the radiometrically determined angle and the videorasterstereographically determined lordosis angle was 28.7 percent smaller than the radiometrically determined angle.

Health Economic Evaluation and Dissemination of Technology

No health economic studies could be identified for the health economic evaluation. The determination of the intervention costs is based on the assumption of a suspected progression per patient and year, which should be checked by means of a conventional X-ray examination and for which videorasterstereography can be used alternatively. In this case, videorasterstereography results in total costs per patient and year of 151.58€ for static 3D spinal column measurement, or 192.39€ for static 4D spinal column measurement (with additional freely selectable image sequences and mathematical averaging to reduce postural variations in individual images) according to the German medical fee schedule for privately insured patients (22;23). The total costs for conventional X-rays amount to 66.58€ according to the fee schedule for patients with statutory health insurance (24).

According to the manufacturer, 850 of its devices have so far been installed in the medical profession (n = 700) and in medical supply stores or by orthopedic technicians (n = 150) in Germany. The survey of registered orthopedic specialists (n = 226) showed that of n = 159 who could be contacted by e-mail, n = 28 of n = 64 respondents (43.8 percent) used such a device. It can, therefore, be assumed that the manufacturer's 700 installations are distributed among various specialist areas and are also used in sports medicine or rehabilitation facilities, for example. Outpatient access for patients is, therefore, not nationwide possible and is associated with effort.

INTEGRATE-HTA

Two Examples for Interactions of Aspects

Using the complexity checklist as a new tool, various interactions could be uncovered in addition to the logic model. As an example, “uncertain causality” and “various interacting components” of the complexity checklist are explained in detail (Table 1). An uncertain causality between the diagnostic method and its effects on the course of therapy was assumed for both adherence and compliance of patients. The examination by means of videorasterstereography can be performed in a normal room and in the presence of relatives, which means that a minor patient can be reassured. Because in videorasterstereography only the surface of the back is measured, a patient can, in contrast to X-rays, wear an open brassiere, stand with her back to the examiner, and does not have to wear lead protection. As a result, the specific sense of shame of adolescents can be reduced.

Table 1. Complexity checklist^a

GKV, Statutory health insurance; MTRA, medical technical radiology assistant; StrSchG, Radiation protection act; StrSchV, Radiation protection ordinance.

^a The table content resulted from the development of the initial and final logic models by reflective thoughts, answering checklists and question catalogues, using and developing several tools, the patient meeting, discussions with domain experts, and the involvement of further stakeholders.

^b The example is a complement of the authors.

Aspect 1–5 Wahlster et al. 2016, Table 1.

Aspect 6–8 Wahlster et al. 2016, p. 5.

Aspect 9–11 Wahlster et al. 2016, p. 15.

The actual measurement takes only six seconds and has the advantage that an average value is calculated from many individual images of a sequence. This allows the patient to continue breathing normally during the measurement, whereas during X-rays the patient has to hold his breath as only one single image is taken. It is conceivable that this examination situation has a positive effect on conveying the relevance of individual therapeutic measures. For example, the reliable and time-defined wearing of the corset, which can be a challenge for adolescents, could be followed more closely if a patient feels that he or she is being cared for carefully and attentively even during the diagnostic process. Another uncertain causality arises for videorasterstereography in the form of a conflict between financial and health aspects, especially for the relatives of adolescent patients. The greater time and effort required for diagnosis using videorasterstereography, due to its lower prevalence, is sometimes difficult to convey to adolescents in puberty. In general, the effects of a lack of information about an increased cancer risk, in particular, breast cancer risk, due to frequent X-rays, which has been observed in various cohort studies with long follow-up (25–27), can be described as uncertain causality.

In the comparison of the technologies, the level of information was identified as the interacting component: for example, the level of awareness has an influence on how the demand of patients and physicians for videorasterstereography develops in comparison with conventional X-ray. Consequently, distribution and use can change with an impact on all HTA domains. At present, the socio-economic status of patients and their relatives, but also of physicians and their practices, for example in rural, structurally weak regions, must also be recorded as an interacting component. Depending on the level of information and access to information about the technologies, interactions of aspects between the HTA domains are initiated. As an example of two relevant aspects that affect and interact with all domains, the following may be mentioned: a patient who has been able to obtain information about videorasterstereography could be denied access because this is not reimbursed by the SHI and he himself cannot afford the examination. The same applies to the mobility of patients, which can be restricted by their socio-economic status. However, if there is a lack of information, access is completely blocked.

Semantic Complex, Evaluation of Adherence in Corset Therapy, and Textmining

The ethical and socio-cultural significance of the disease is illustrated by the developed semantic complex (Table 2), which contains linguistic idioms for the terms “back” and “posture.” It illustrates how deeply the topic is anchored in a society and the importance it attaches to it. The ethical and socio-cultural relevance becomes particularly clear in a corset, which, in the case of idiopathic scoliosis, often has to be worn by a population that is vulnerable due to its age. The corset highlights the obvious flaw and represents a stigmatization.

Table 2. Semantic complex idiopathic scoliosis^a

^a The table content is based on reflective thoughts.

Furthermore, the corset severely restricts the patients’ freedom of movement and social participation. Activities are avoided that intensify sweating or that allow a view of the exposed body (e.g., outdoor swimming pool). For female patients in particular, the physical appearance, additionally emphasized by a corset, represents a mental strain and can lead to the clothing style being geared toward concealing the blemish. Nevertheless, adherence is of great importance for the success of corset therapy and the prevention of progression.

For the evaluation of adherence in corset therapy, the Asklepios Katharina-Schroth-Clinic provided a data set for the survey period from 01 September 2016 to 01 September 2018. Data from n = 2,225 patients (female 88.2 percent) with an average age of 14.6 years (range 6–63) could be used to calculate the average target and actual wearing time. The analysis showed that the average difference between target and actual wearing time was −4.6 h/day (average target value: 19.6 h/day, average actual value: 15 h/day). In total, 4.8 percent of the patients wore the corset longer, 23.1 percent wore the corset for the same length of time, and 72.0 percent wore it for less than prescribed by the treating physician. The results on corset adherence lead to the hypothesis that for patients there is a conflict between psychological and physical stress as well as the restrictions in social participation caused by wearing the corset and the psychological stress caused by fear of progression, for example, if the corset was not worn for a sufficiently long time.

In order to identify additional aspects for the individual domains and to check the awareness of publishing and clinically active physicians about the ELSOI of the technology, introductory and discussion parts of relevant publications were examined by means of text mining and shown in a harvest plot (Supplementary Figure 1). The text mining revealed a few text passages in which aspects other than the benefit aspects of the technologies were addressed. Exposure to radiation and the avoidance of radiation exposure associated with videorasterstereography were the most frequently discussed topics.

Final Logic Model and Interaction Model

The final logic model (Figure 1) provides an overview of the context of the technology comparison, the relevant aspects of each domain, the details of patients, intervention, and outcomes, as well as aspects for implementation.

Figure 1. Final logic model

Figure 2 shows the interaction model of the relevant aspects of the individual domains for videorasterstereography. There is no hierarchization of aspects on a vertical level. On the horizontal level, the domains were arranged according to the EUnetHTA core model (28).

Figure 2. Interaction model

In the presentation, a high complexity of videorasterstereography becomes clear; in particular, the number of individual ethical aspects, as well as the density of interactions of the ethically and socio-culturally significant aspects, stand out.

Discussion

The measurement results of videorasterstereography differ from those of conventional X-rays in terms of quality and quantity due to the different imaging techniques used. In contrast to 2D measurement by radiography, which is used to visualize bony structures, videorasterstereography provides a 3D quantification of the deformity. In the four included studies, all of which showed a high risk of bias, standard deviations between 7.9 and 8.2 degrees (18;21), an average difference of 5.8 and 8.8 degrees (20), and differences between pre- and postoperative measurements of 33.3 and 28.7 percent were found (19). The differences in measurement accuracy, thus, correspond to those obtained for purely radiographic Cobb angle measurements in studies, which can vary by up to 5° for the same observer and by up to 7° for different observers (29–31). The fact that a potential of videorasterstereography can be deduced from these results is supported by the fact that the purely light-optical procedure, in contrast to conventional X-rays, can be used frequently for diagnostic purposes without hesitation from a medical point of view. By means of more closely meshed control examinations, the deviations of the measurement results of videorasterstereography from those of conventional X-rays could be compensated for, and the risk of malignant neoplasms in general and in breast cancer patients in particular could be reduced in the vulnerable population of adolescents.

With regard to access to services, there is social inequality depending on socio-economic status, because videorasterstereography is a self-paying service or is covered by private health insurance. Patients with a low socio-economic status are, therefore, denied access, even though they should be informed about the advantages and disadvantages. There is an interaction both with the right to self-determination, restricted for the reasons mentioned above, and with access to videorasterstereography, because the devices are not widely available and patients may have to accept long journeys and waiting times that they may not be able to afford. In addition, videorasterstereography offers greater patient comfort in many respects and makes the examination, which is characterized by anxiety and possible sense of shame, easier.

Discussion of the Methodology

The technology comparison, which was initially classified as low complexity, turned out to be highly complex: The multiple assignment of individual aspects to the domains resulted in a high density of interactions. In contrast to Bond and Weeks (32), who report on first experiences with the INTEGRATE-HTA methodology in Canada, we do not interpret the overlaps as redundancies, but as condensations. The interactions of different aspects on different levels or between different domains reveal the importance of a technology for a society. The fact that an aspect can be not only of socio-cultural importance but also of ethical and economic significance at the same time should be made clear to decision makers so that they can grasp the relevance and depth of a decision. Furthermore, in the present HTA, relevant aspects for technology comparison were collected from the beginning, successively supplemented, and transferred to tables for cross-domain presentation. In contrast to the Canadian authors, who examined all domains separately, we did not have the difficulty of failing a posteriori to integrate the individual domains due to our cycloid manner by going back and forth between the steps as visualized by Wahlster et al. (9;32). On the contrary, the parallel coverage of different aspects and the multiple assignment of individual aspects to domains created a certain fascination from the very beginning, because the technology comparison had originally been classified as of low complexity. In contrast to Bond and Weeks (32), who did not develop either an initial or a final logic model due to time pressure and resource constraints, we devoted distinctly more time to its development by reflective thoughts and repeated discussions on how to integrate aspects of domains adequately each time we answered one of the numerous checklists and question catalogues. From our point of view, this is indispensable for the success of the INTEGRATE-HTA methodology resulting in a largely smooth flow, because all aspects and their interactions have already been captured by discussion. The logic model turned out to be the core piece of the INTEGRATE-HTA method.

In its nonlinear approach, the application of the INTEGRATE-HTA methodology presents a certain challenge. A graphic presentation of the evidence accompanying the HTA process in the form of logic models, harvest plot, and interaction model can additionally promote motivation. The application of the methodology contributes to the identification of interactions between domains and significantly broadens the perspective on a technology. It improves patient-centered understanding and facilitates discussion significantly.

Outside the framework set by the commissioning authority, the lack of searching guidelines for evaluations of the technology may be called a limitation, though. In addition, the last step according to the INTEGRATE-HTA methodology (9), “HTA decision making,” is not part of HTA reports in Germany. Although the preliminary HTA report is sent to relevant committees and subjected to a public commenting procedure before publication, a discussion of results with stakeholders and decision makers is not planned. In the sense of a not only interdisciplinary but also transdisciplinary exchange of information (33), such a procedure would be highly desirable and should be discussed and implemented for future HTA processes. An event in the form of a public dialogue at the end of the HTA process would be conceivable. Decision makers (patient) and other stakeholders could also be invited to such an event. This forum would give citizens a say, and concerns, unanswered questions, uncertainties, and limitations of the evaluation process could be discussed and taken into account in the decision-making process as envisaged in the INTEGRATE-HTA methodology.

Conclusions

Videorasterstereography is considered to be an alternative to conventional X-rays and an additional option to clinical examinations, which could lead to a reduction of radiographs used in follow-up examinations. Thus, X-rays could be taken only if there is a suspicion of progression after videorasterstereography and additional information is necessary for a decision on therapy. X-rays are recommended as obligatory by professional associations for the initial diagnosis, the indication for corset therapy and verification of this as well as pre- and postoperative examinations. For general follow-up, however, videorasterstereography should always be considered as an alternative in order to reduce or avoid radiation exposure and the associated risk of malignant neoplasms.

Patients should be regularly informed about the risks of radiation exposure and the alternative to conventional X-rays. Professional societies and cost units should work toward a widespread use of X-ray passports and regular entries of examinations. In addition, a dialogue between professional societies should also be initiated on the importance of the measurement inaccuracies of both procedures for initial diagnosis.

By applying INTEGRATE-HTA, the complexity of videorasterstereography could be captured. The methodology can be used like a construction kit, from which one can choose flexibly and according to the research question.