Client Socio-Demographic and Service Receipt Inventory – European Version: development of an instrument for international research

Abstract

Background

Cross-cultural comparison of mental health service utilisation and costs is complicated by the heterogeneity of service systems. For data to be locally meaningful yet internationally comparable, a carefully constructed approach to its collection is required.

Aims

To develop a research method and instrument for the collection of data on the service utilisation and related characteristics of people with mental disorders, as the basis for calculating the costs of care.

Method

Various approaches to the collection of service use data and key stages of instrument development were identified in order to select the most appropriate methods.

Results

Based on previous work, and following translation and cross-cultural validation, an instrument was developed: the Client Socio-Demographic and Service Receipt Inventory – European Version (CSSRI–EU). This was subsequently administered to 404 people with schizophrenia across five countries.

Conclusion

The CSSRI – EU provides a standardised yet adaptable method for collating service receipt and associated data alongside assessment of patient outcomes.

The clinical and social burden imposed on individuals, families and communities by schizophrenia contains an economic dimension (Knapp et al, 1999). Costs are incurred at all levels of society, either directly through expenditure or unpaid time spent on providing health and social care and support, or indirectly in terms of lost opportunities (such as for leisure or work). The estimation of these costs, and descriptions of associated patterns of service uptake or utilisation, assist in a comprehensive assessment of the resource consequences of schizophrenia and its treatment.

Yet standardised methods or instruments for international research have not been developed for these tasks. Methods for the measurement of service use and costs need to be more sensitive to the local context than do the rating of psychiatric symptoms or the assessment of behavioural traits and personal abilities. For example, the approach to measurement must take into account the structure or system of treatment and care, and factors relating to patients' access to specific services. The broader socio-economic and cultural contexts will also be relevant, for they will influence, inter alia, the prevailing level of unemployment, and the expectations regarding roles of families and wider local communities in supporting people with mental health problems (Johnson et al, 1997).

The aim of the work described in this paper was to develop an internationally usable method for gathering data on service utilisation and other domains relevant to the economic analysis of mental health care. Specifically, the work represented a core element of a five-country EPSILON (European Psychiatric Services: Inputs Linked to Outcome Domains and Needs) Study, the primary aims of which were to produce standardised versions of instruments in key areas of mental health services research in five European languages, test their reliability, and employ them in a five-country, cross-sectional study of people with schizophrenia (Becker et al, 1999). The five research sites were Amsterdam (The Netherlands), Copenhagen (Denmark), London (England), Santander (Spain) and Verona (Italy).

METHOD

Analytical perspective

For the purposes of mental health economics research, it is desirable to measure service use and costs comprehensively, since the broad personal and social impacts of schizophrenia typically result in a need for contact with many different service agencies, including health services, social services, housing and criminal justice services (Weisbrod et al, 1980; Clark et al, 1994; Knapp et al, 1999). This comprehensive perspective is particularly important for multinational studies, since different countries have established different boundaries between health and other services, and these boundaries have been known to shift over time as a result of changes in government policy or other forces. Also, the balance of responsibilities between the public sector (state) and other agencies may similarly vary from country to country. Data collection should therefore range beyond the immediately observable health service inputs to include other service supports, contacts with other agencies (such as housing and criminal justice) and non-service implications of mental ill health (particularly the costs of lost employment and productivity, and the economic burden falling on family caregivers ¹ ). Moreover, data should be obtained on the frequency and intensity of any service contacts, in order to examine service patterns and to estimate costs accurately.

International research on service utilisation patterns, costs and other economic dimensions of mental health care is complicated by the need to reflect the contexts within which people live and receive their care. Arguably this is true of any research tool, but the problems of economic research which crosses international boundaries are especially acute when the objects of comparison are themselves heavily influenced by social, economic, political, historical and cultural structures and forces peculiar to those contries. To a greater degree than for the other instruments developed or tested in this study, therefore, we needed to ensure that the instruments assessing resource use captured the core features of each of five health care and other systems covered by the study, as well as being sufficiently standardised to permit meaningful international comparison. A core feature was thus to balance local relevance with international generalisability.

Sources of resource utilisation and socio-economic data

Selecting the most appropriate method of data capture for economic studies depends on a number of factors, including the primary purpose of the study, the availability of funding and the data collection methods to be used for other evaluative information in the broader study. In this paper we do not discuss what methods to employ in each possible circumstance or in response to each type of eventuality (for helpful discussion of these issues, see Drummond et al, 1997; Hargreaves et al, 1998; Johnston et al, 1998). Instead we focus on the type of study within which we would be examining service use and associated patterns. The main features of that type of study are in fact quite common in mental health services research :

1. (a) a representative sample of people with mental health problems (in this case schizophrenia) treated by ‘ordinary’ services;

2. (b) the aim of making comparisons between samples or sites (in this case, comparisons between countries);

3. (c) a cross-sectional design, with the possibility of repeating the observations later on sampled individuals;

4. (d) a limited research budget, making it necessary either to rely on extant information sets and/or to collect interview-based data concurrently with clinical and associated data.

The two broad options for data collection that presented themselves were to use existing information held by service-providing or funding agencies, or to rely on individual informants. One of the study sites (Verona) has a psychiatric case register which contains health service utilisation data, and other sites had some electronic data (for example, secondary health care information systems in London and Copenhagen). However, none of the local ‘ routine information’ systems was sufficiently compatible with the others to provide the basis for comparative research. Even if there had been some compatibility, there would be the question of data breadth: do extant systems keep data on all relevant services ? We know from previous research that schizophrenia sufferers use many different services (Knapp et al, 1999); so even if each site had computerised data, would these cover all relevant services ? In the (unlikely) event that they did, there would then be the considerable challenge of merging data-sets designed for different health care systems, to meet different local management needs, and using different software. Data capture via electronic information systems was therefore rejected at an early stage. Information on services used by individual people will usually be held by service providers, and some professionals (such as general practitioners) should have a reasonably broad view of service utilisation. The disadvantage of relying on service providers to produce these data is that records or professionals' knowledge will often be incomplete, partial (generally covering only the agency's own responsibilities) and difficult to access or expensive in researcher time. These latter factors were important in the present study, and we concluded that we could not rely on agency information holdings in the five sites to provide the range and quality of data needed.

Another way of collecting these data is to ask individuals, either through interview or a self-completion questionnaire. Postal or self-completion methods (including diary cards) have been used in some previous studies (Mauskopf et al, 1996; Gosden et al, 1997) but were ruled out here, partly because we feared a low response rate, but mainly for the pragmatic reason that other study objectives already required face-to-face interviews, and there is a long track record of collecting service utilisation and associated data alongside clinical data (Beecham, 1995).

Three potential groups of respondents could be interviewed: case managers (keyworkers or similar), family members, or patients themselves. If there is a case (or care) manager or keyworker to coordinate services for the patient, they might be a good respondent, although this would depend on the breadth of their responsibilities and knowledge (Widlak et al, 1992). Case management has been implemented in some guise in all the study sites, but because the particular modes of operation and service structures differ in these different sites, standardised reporting of service uptake by case managers would be problematic. Where a patient lives with their family, another data source could be other family members (who may act as informal case managers, of course). In some sites it transpired that a majority of the sample lived with at least one relative (72% in Santander, 50% in Verona), but elsewhere this was much less common (20% in Amsterdam and London, and only 4% in Copenhagen). Family members were interviewed for another part of the research study (to complete the Involvement Evaluation Questionnaire (IEQ); van Wijngaarden et al, 2000, this supplement), but it was felt that this data source was not sufficiently widely available for the purposes of the description of service use and cost calculations.

The patient is the only person who should have all or most of the information on which particular services have been accessed, how often and for how long. A potential concern, however, is that the patient may not report service utilisation accurately, either because of their clinical condition, or because they exhibit the common human failing of poor recall. We did in fact choose to ask patients for these data, but we needed to take especial care with instrument design to improve the likely accuracy of the information provided (for example, by providing clearly defined and identifiable categories of service or state benefits).

The comparative merits of retrospective and prospective data collection have been discussed elsewhere (Johnston et al, 1998). Prospective data collection essentially requires maintaining a diary of all service contacts, whereas retrospective collection involves occasional completion of an interview, reflecting back on services used in the previous few weeks or months. (Data collection should not be confused with design: prospective trials can of course use retrospective methods for collecting service use data.) In this study, we adopted a 3-month retrospective period, which is sufficiently long to pick up the wide range of services that individuals might take up but without stretching the respondent's powers of recall (there is evidence to suggest that interviewees significantly under-report frequent events when asked to report retrospectively over a 6-month period; Jobe et al, 1990).

Instrument development

It is possible to distinguish a number of stages in the development of an instrument for collecting service use and related (Table 1). The first task was to identify criteria for selection or development of an instrument. Four requirements were identified :

Table 1 Stages of instrument development

1.	Identify the main desirable characteristics of an instrument in this area
2.	Examine existing instruments
3.	Select an existing instrument for adaptation or develop a new instrument from first principles, aiming for face validity
4.	Translate into other languages
5.	Organise and conduct focus group discussions to refine the content and translation
6.	Revise both the original instrument (in the light of content recommendations) and the translations (in the light of terminological or language recommendations)
7.	Employ the instrument in its various languages in an ‘exploratory’ cross-national sample
8.	Re-examine the performance of the instrument in terms of, for example, response rates to individual questions, ease of completion, preferred aggregation of service categories
9.	Make further revisions to the instrument so as to make it ready for more widespread use

1. (a) It should span the domains of accommodation and living circumstances, employment and income, and service utilisation, so as to allow description of the economic and related circumstances of individual people and the service or care ‘packages’ that support them.

2. (b) It should record the frequency and intensity of service use, so that service costs can be calculated as accurately as possible.

3. (c) After translation, it should be able to be used alongside the other instruments chosen for the EPSILON Study, and it should also be suitable, after only modest adaptation, for use in other European countries.

4. (d) It should be understandable by respondents (people with schizophrenia) and manageable for use in interviews conducted by trained researchers.

The second stage was to establish whether there existed an instrument that would meet these requirements. If so, we could move on to its employment in the empirical part of the study, examining its performance in use with samples of people with schizophrenia. If no existing instrument satisfied our requirements, as was the case here, the next task was to develop or adapt one. Rather than start from very first principles, we chose to build on an existing instrument, the Client Service Receipt Inventory (CSRI). The CSRI has been widely employed and has a multitude of forms, having been used in over 100 studies since it was first developed in England in the mid-1980s (Beecham & Knapp, 1992, introduce this instrument in the context of a wider discussion of cost research methods in mental health). Although it has been used outside the UK, the CSRI has not previously been subjected to the degree of developmental work or scrutiny that we employed in this study. In particular, close attention was paid to the categorisation of housing, employment and service use items of the inventory in order to generate an instrument capable of international use (see below).

A set of baseline questions was generated which covered the topics of interest (initially in English, subsequently translated into the other four languages). This baseline schedule was then subjected to a process of cross-cultural validation and refinement, based on the discussions in focus groups at each site. The final stage in the development of this resource utilisation measure was to administer it to a sample of individuals at each site, and further revise the instrument in the light of any difficulties or misinterpretation.

RESULTS

The form of the instrument

The baseline version of the Client Socio-Demographic and Service Receipt Inventory - European Version (CSSRI-EU) was constructed around five main sections (Table 2). A manual was prepared which contained explanatory notes for particular questions or items in the schedule that required additional information, definition or guidance.

Table 2 Key domains and variables of the Client Socio-Demographic and Service Receipt Inventory - European Version (CSSRI-EU)

Section	Key variables
Socio-demographics	Age, gender, marital status, ethnicity, mother tongue, years of schooling, educational level
Usual living situation	Living situation (alone, with relatives, etc.), type of accommodation, household composition
Employment and income	Employment status, occupational category, days of work lost, state benefits, source/level of income
Service receipt	Hospital in-patient days, out-patient/day care attendances, community-based service contacts (mental health, social services and primary care), criminal justice service contacts
Medication profile	Name/type of drug, dosage level and frequency

Socio-demographic information

A range of categorised socio-demographic variables, including date of birth, gender, marital status, ethnic group, mother tongue, years of schooling and level of educational attainment comprised the initial section of the instrument. Although some of these variable (such as age or gender) appear in other instruments, these data were comprehensively recorded here for completeness. Moreover, such data lead naturally on to consideration of other socio-economic circumstances.

Usual living situation

Accommodation represents an important parameter for economic studies of mental disorder, largely because of the high cost of specialist residential care. An individual's living situation (alone, in a family or living with other, non-related residents) is also a potentially significant predictor of cost (and outcomes). Accommodation was divided into domestic, hospital and community residential categories, each with clearly defined sub-categories (for example, tenure of domestic accommodation or staffing cover/intensity in residential care). Changes in accommodation over the retrospective period can be recorded.

For all subjects resident in non-domestic accommodation, completion of a one-page supplement was requested, containing information on the number of (available and occupied) places/beds in the facility, the total complement and cost of care staff, other revenue costs and the average weekly charge or fee per resident place/bed. This supplement, based on a schedule developed for costing mental health residential care in the UK (Chisholm et al, 1997), was an addition to the original CSRI and was completed after the face-to-face interview, in consultation with a facility manager.

Employment and income

This section aimed to elicit information on patients' employment and income circumstances. It is an important source of information for establishing the indirect costs and effects of schizophrenia, such as lost days of work, and also for estimating the living expenses of the patient. Employment status was divided into a number of appropriate categories (paid or self-employed, unemployed, housewife, house-husband, etc.), while occupational categories were based on an international standard classification of occupations (manager/administrator, professional, skilled labourer, etc.). The approach taken regarding state benefits was to identify a number of international categories of benefits or entitlements (unemployment and income support; sickness and disability; housing; other), and to have a list of national variants that fell under these broad categories (for example, ‘ job-seekers' allowance’ in the UK). This enabled us both to make consistent comparisons between study sites and to build up a set of data that has most meaning and use within each individual site. Personal (gross) income was also requested, using bands obtained from national statistics bureaux that reflected the quintiles of gross income in each country (so that the proportions of patients falling into these internationally equivalent income bands could be compared).

Service receipt

A range of psychiatric, social and general medical services were identified which together were considered to make up a comprehensive profile of services available to the patient population in each of the five centres. The main categories were: psychiatric and general medical in-patient hospital admissions and total days; psychiatric and general medical hospital out-patient attendances; community-based day services (frequency and intensity of attendance); and contacts with primary care, social services and community mental health care professionals. Clear definitions were attached to individual service components or categories, in order to be able to compare the different sites, and space was left for inclusion of other services provided to patients that were not specifically identified in the inventory. For each service, the numbers of contacts in the previous 3 months were requested, and, where applicable, the sector of provision (statutory/government, voluntary or private). A final subsection asked for contact with criminal justice services (number of police contacts, nights in custody, psychiatric assessments or court appearances).

Medication profile

A profile of the individual's use of all prescribed medications in the previous month was requested, incorporating the name of the drug, the dosage level and frequency, and whether it was prescribed on a depot basis.

Translations and focus groups

Once the baseline version of the instrument had been developed, the next steps were translation into the other four European languages (Danish, Dutch, Italian, Spanish), by either professional translators or local researchers, followed by cross-cultural validation of the translated instrument. Since the CSSRI-EU is an inventory of socio-economic indicators and service variables rather than a multi-item rating scale of a particular outcome domain, the focus in this study has been on achieving face validity and semantic equivalence within and between individual participating sites, rather than formal exploration of the reliability of the measure between raters, sites or time points (Schene et al, 2000, this supplement). This took place through both informal dialogue and discussion with principal investigators and other interested parties, and more formally through the conduct of focus groups. Focus groups consisted of between six and ten individuals (psychiatrists, other health professionals, social care workers, informal carers and service users), and were intended to address two aspects of the instrument: its content and its language (Knudsen et al, 2000, this supplement).

The CSSRI-EU focus groups generated a number of system-level comments that revolved around the perceived incompatibility of national health, social and welfare structures with the attempted European-wide structures or categories given in the initial version of the CSSRI-RU. These comments related to two sections of the instrument: usual living situation, and employment and income. In particular, focus groups suggested reordering the categories of employment, benefit entitlements and accommodation so as to reflect their own national systems better. These suggestions were incorporated as far as possible, without losing the core requirement of inter-site comparability. For example, four international categories of state benefits were developed (unemployment/income support; sickness/disability; housing; pension), within which sites could specify local variants of these broader categories. Residential care was a further area that required reordering, owing to the heterogeneity of service arrangements in different sites. This problem was overcome by describing the final categories in neutral, broad terms (overnight facility, 24-hour staffed; overnight facility, staffed (not 24 hours); overnight facility, unstaffed).

A second set of comments revolved around country-specific suggestions for enhancing the understanding, definition or measurement of individual items or components included in the service receipt section of the inventory. A particular area of discussion concerned the appropriate classification and definition of day care and support facilities and community-based mental health services. For example, the Dutch system of community mental health centres needed to be correctly classified under the appropriate item in the inventory.

The instrument was then revised, both in its original English form (in the light of focus group recommendations as to content) and in each of four translations (in the light of recommendations about terminology or language).

Exploratory use of the instrument

The tool was now ready for use in empirical research. In this study the sample was selected in each of the five sites by employing the same diagnostic and administrative criteria (adults aged 18-65 inclusive with an ICD-10 diagnosis of schizophrenia: code F20). All patients had been in contact with mental health services during the 3-month period preceding the start of the study (Becker et al, 1999). Fieldwork was conducted over 15 months in the five sites. The main findings regarding service utilisation and costs for this sample are to be reported elsewhere, but a number of methodological issues which emerged in the course of data collection and in the following initial examination of the final dataset can be summarised here.

First, the response rate in terms of completion of the CSSRI-EU was 100%; no refusals to be interviewed were encountered across the sites. This unusually high response rate may be attributable in part to the integration of service utilisation questions with key socio-demographic characteristics (the latter being of fundamental importance to analyses, over and above economic evaluation). Response rates to individual items within the CSSRI-EU were also high (where applicable), indicating few difficulties of interpretation. For items or services not available in, or applicable to, a site (for example, long-stay psychiatric wards in certain sites), zero was entered for all cases.

Structured response categories for a range of socio-demographic and socio-economic indicators (such as living situation, level of educational attainment and state benefit entitlements) performed well. The only clear source of inter-site differences in interpretation related to employment status: specifically, the classification of patients not in open or sheltered employment; in certain sites, such patients were categorised as ‘unemployed’, whereas in other sites a large proportion were classified as ‘retired’, in the sense that they were in receipt of a disability allowance or pension and were not expected or likely to be employed in the future.

As expected, and allowed for in the baseline version of the instrument, a number of specific services had been used over and above those specified. These service contacts were recorded as an ‘other service’ under the appropriate service category, together with a brief text description of the service, and subsequently recoded or described in their own right. For one site (Verona), where the same professionals provide both hospital and community care (continuity of care model), there was evidence of ‘double counting’ of service provision, which subsequently required checking of CSSRI-EU data with that centre's psychiatric case register. A final area of difficulty experienced in the service receipt section of the instrument was coding of the dosage frequency for prescribed medications, because sites adopted different approaches for recording frequency rates for depot v. other drugs.

On the basis of these analyses, a final set of revisions was made to the instrument (and accompanying manual) in order for it to be usable in other research studies. For example, specific services included under ‘other’ for a particular service category were incorporated as necessary, and guidance was modified for coding employment status and frequency of medication use.

DISCUSSION

From the outset of the EPSILON Study it was agreed that one part of the empirical study should look at economic aspects of schizophrenia care. After consideration of a number of methodological options, we developed an instrument - the CSSRI-EU - for use in interviews with patients and/or key staff. The CSSRI-EU is easy to use after a short amount of interviewer training (1-2 hours; the brief manual also provides guidance on completion), can be completed in interviews by people with schizophrenia, takes only about 20 minutes to complete, and provides information useful for a number of evaluative and other purposes both within and across countries. The CSSRI-EU enables us to trace patterns of service use and care in an international context, calculate the associated costs of care, and examine relationships or differences across countries between costs and a range of socio-demographic and clinical characteristics. Each of these capabilities can be usefully employed to improve the planning, provision and evaluation of mental health services for people with schizophrenia.

Since the CSSRI-EU is an inventory of variables required for economic analysis, rather than a multi-item rating scale, the focus in this study has been on achieving face validity within and between individual participating sites, rather than formal exploration of the reliability of the measure between raters, sites or time points. The absence of such reliability measures represents a gap in our understanding of how accurate the CSSRI-EU is in recording rates of service utilisation. While significant but relatively uncommon events such as hospitalisation are readily recalled, there is concern that reporting of the frequency and intensity of contact with community-based service professionals by patients is subject to recall error (Jobe et al, 1990; Clark et al, 1994; Johnston et al, 1998).

This possible source of error can be examined by comparing the values given by patients in the CSSRI-EU with an alternative data source, either another informant (an informal carer or keyworker) or an administrative database. We have already commented on the limitations of these alternative sources of data in an international context, specifically the incomplete knowledge of other informants and the absence of standardised, high-quality information systems across countries. Where they exist, however, well-maintained psychiatric case registers do represent one important data source against which to assess the performance of certain elements of service receipt schedules such as the CSSRI-EU. Such an analysis was not an objective of this study, but recent work in one of the sites (Verona) has considered these issues and showed that the agreement on overall psychiatric costs was high: the concordance correlation coefficient was 0.93 for all patients and 0.97 for patients with a diagnosis of schizophrenia (Mirandola et al, 1999).