Hostname: page-component-f554764f5-sl7kg Total loading time: 0 Render date: 2025-04-16T23:59:20.966Z Has data issue: false hasContentIssue false

Measuring Inclusion for Greek-Speaking Schools: Validation of the Themis Questionnaire

Published online by Cambridge University Press:  10 April 2025

Olga Lyra
Affiliation:
Frederick University, Nicosia, Cyprus
Kyriaki Koullapi*
Affiliation:
Frederick University, Nicosia, Cyprus
*
Corresponding author: Kyriaki Koullapi; Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

This paper presents the content validation process and results of the Themis Inclusion Tool, a questionnaire designed to stimulate teacher reflection on the response to diversity in schools in Cyprus, where, despite efforts, progress is still necessary. We present the adapted form of the Themis questionnaire originally published in English. The Greek version of the questionnaire contains 60 items measured on a 5-point Likert scale, consisting of three dimensions: contexts, resources, and processes. The questionnaire also includes two open-ended questions. The use of the Themis questionnaire is suggested as an effective means to enable teachers to understand challenges with respect to inclusion and for developing more inclusive schools. Thus the aim of this research is to contribute to the lack of updated, validated, and research-based tools for Greek-speaking schools at a time where school self-evaluation processes have been prioritised in educational policymaking.

Type
Original Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2025. Published by Cambridge University Press on behalf of Australian Association of Special Education

Efforts to ensure the rights of all children to education have gradually set increased challenges for general education teachers, who are expected to respond to a very diverse student population by teaching effectively in mixed-ability classes (Kefallinou et al., Reference Kefallinou, Symeonidou and Meijer2020). As research shows (Symeonidou, Reference Symeonidou2022), discriminatory practices are still applied, despite teachers’ good intentions. For instance, general education teachers often opt for transferring students with invisible disability to integration classes, instead of differentiating their teaching practice so that these students can be provided equal learning opportunities with the rest of their classmates within the main classroom (Kefallinou et al., Reference Kefallinou, Symeonidou and Meijer2020).

In this respect, self-assessment processes that enable schools to reflect upon their core values and daily practices as a form of self-evaluation in terms of specific dimensions, such as inclusiveness, are powerful tools in helping teachers acquire a deeper understanding of the culture that is embedded in their school. Further, they enable the initiation of discussions among the school community regarding whether the strategies that they apply and the culture that is predominant in their school are indeed inclusive (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019). The identification of these elements leads, on a next level, to the development of concrete action plans that are based on the identification of blind spots — that is, weaknesses both on a collective and an individual level (Lyra, Reference Lyra2012).

Given the benefits and long-term impact of such improvement processes, it is of great importance that they are introduced into educational systems with a clear inclusion focus. Nevertheless, the absence of practical materials to assist them in navigating this transformative journey remains a challenge. Cyprus is an example that, according to Damianidou and Phtiaka (Reference Damianidou and Phtiaka2018), ‘the extent to which current teaching practices are inclusive seems to raise issues for further consideration’ (p. 1080), since they still provoke considerable discrimination. In fact, there is a lack of updated and validated tools for inclusive school development that enable a holistic approach in the understanding and measurement of dimensions related to the successful implementation of inclusion that are provided in the Greek language, and thus adequate for Greek-speaking educational contexts (i.e., the Greek as well as the Greek Cypriot).

An important requirement for the tools’ effectiveness is the identification, in an initial phase, of teachers’ attitudes and views on barriers to the implementation of inclusive education. Thus, the purpose of this research is to evaluate and adapt to the Cypriot Greek-speaking educational context the questionnaire of the Themis Inclusion Tool (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019), which is a contemporary, recently developed tool that aims ‘to offer an overall evaluation of the response to student diversity in schools’ (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019, p. 13). Research (Carvalho et al., Reference Carvalho, Cruz, Azevedo and Fonseca2022) suggests that once these elements are identified and reflected upon, they will lead to more productive efforts in terms of intentional and continuous actions for improving inclusiveness via the use of self-evaluation tools.

Teachers’ Response to Student Diversity

The difficulty for teachers to respond effectively to student diversity can be understood through Hargreaves’s view (2003), which suggests that teachers’ practices take the form of ‘scripts’ that reside within teachers and have been shaped by their personal experiences and beliefs. This understanding can explain the fact that teachers may feel incapable of responding effectively to students’ diversity, even though they perceive themselves as using inclusive practices (Symeonidou, Reference Symeonidou2022).

Following Hargreaves’s (2003) concept, teachers’ internal scripts prevent them from critically examining whether their teaching practices are indeed inclusive, which in fact acts as a barrier to adopting inclusive practices. In line with this, Kielblock (Reference Kielblock2018) notes that there is a need for instruments that measure teachers’ attitudes in ‘real-world’ practices and are related to the concept of ‘inclusive education for all’. Such instruments would enable teachers to reflect on evolving and adapting their teaching practices towards all students within the classroom, rather than in two separated groups (i.e., students with and students without disability).

According to Azorín-Abellán (Reference Azorín-Abellán2018), although many questionnaires have been developed for measuring the effective implementation of inclusion, there is still need for research-based tools that measure particular aspects of teachers’ response to student diversity in depth, such as sufficient knowledge of how to effectively use available resources to support full and equal participation of all students during the lesson. The use of the Themis questionnaire invites teachers to actively participate in well-organised ‘collaborative, transformative approaches’ (Messiou, Reference Messiou2017, p. 148) as a form of self-evaluation process that enables schools ‘to review progress on their journey to becoming more inclusive’ (Azorín & Ainscow, Reference Azorín and Ainscow2020, p. 58).

The Initial Phase of the Self-Evaluation Process

A self-evaluation process is one that ‘involves teachers in teacher evaluation by giving them ownership of the process of evaluation and creating a sense of awareness of their weak areas they ought to improve’ (Quddus et al., Reference Quddus, Khalid and Khan2019, p. 807). An essential aspect of such processes is the initial phase, where teachers attempt to critically examine their views and identify weaknesses in terms of how they respond to all students’ needs. As research shows, active involvement in a self-evaluation process has a positive impact on strengthening the psycho-emotional state of educators (Quddus et al., Reference Quddus, Khalid and Khan2019). In this way, feelings such as fear and insecurities towards changes to the status quo (Lyra et al., Reference Lyra, Koullapi and Kalogeropoulou2023) can be reduced. This can be achieved in the first steps of self-evaluation, since through the collaborative identification of the school’s weaknesses, teachers feel empowered, mutually motivated, and supported by each other to overcome the barriers (Booth & Ainscow, Reference Booth and Ainscow2002).

Especially regarding the efforts for implementing inclusive education, the initial phase allows educators to first identify and understand their weaknesses both at an individual and collective level and, second, to be able to define a plan for inclusive school development as a collective effort (Carvalho et al., Reference Carvalho, Cruz, Azevedo and Fonseca2022). In this phase, research-based tools for inclusive school development have proven to be particularly helpful (Azorín & Ainscow, Reference Azorín and Ainscow2020). However, the use of updated, culturally appropriate research-based tools for measuring teachers’ attitudes towards student diversity is still not widespread (Quddus et al., Reference Quddus, Khalid and Khan2019) in educational systems such as that of Cyprus, where efforts to implement inclusive practices have stagnated in recent years.

An innovative tool, which is exclusively ‘designed to offer an overall evaluation of the response to student diversity in schools’, is the Themis questionnaire, which was first designed for use in the Spanish educational context (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019, p. 13).

As the tool’s creators point out, inclusion must be understood within specific contexts, as barriers to learners’ participation and achievement vary across different cultural contexts. This implies that tools should be informed by the experiences of those in the relevant context (Azorín & Ainscow, Reference Azorín and Ainscow2020).

The necessity for a new tool that aims to offer an understanding of teachers’ perceptions and response to diversity arose in the Spanish context, an educational landscape that shares similarities with the Greek Cypriot education system. In fact, in the Spanish school system, as in Cyprus, there are three different types of schooling: general schools, special education schools, and special education classrooms located in general schools. Additionally, teachers in Spanish schools are still implementing segregative practices despite efforts to ensure equal opportunities for all (Azorín & Ainscow, Reference Azorín and Ainscow2020), which is very similar to the practices adopted in the Cypriot Greek-speaking educational context (Symeonidou, Reference Symeonidou2022).

Developments in Inclusive Education in Cyprus

In the Cypriot educational system, the Integration and Training of Children With Special Needs Act (House of Representatives, 1999) currently applies. The main goal of this Act is to intensify the efforts for equal education for students with disability and/or additional learning needs (Damianidou & Phtiaka, Reference Damianidou and Phtiaka2018). However, although the above cited law was a significant step in this direction, it is still considered as establishing segregation among students (Damianidou & Phtiaka, Reference Damianidou and Phtiaka2018). Practically, a considerable number of students with disability are still placed in special education schools or in separated classes within general education schools — the so called ‘special education school units’ that are still considered as a form of integration. In addition, research (Damianidou & Phtiaka, Reference Damianidou and Phtiaka2018) has shown that although Cypriot teachers express positive views towards the implementation of inclusive education, they still apply segregative methods of teaching at the micro level of their classroom. Such practices include grouping students based on ability levels without fostering interaction among diverse groups, over-relying on traditional teaching methods such as lecture-based instruction, and neglecting to use differentiated or individualised teaching approaches to address the diverse needs of their students (Symeonidou, Reference Symeonidou2022). Following this stagnant situation and after numerous voices have raised serious concerns over the years regarding social justice and quality education for all, since 2017 the Ministry of Education has intensified efforts for establishing a new legislative framework that ensures inclusive education. Towards this effort, the European Agency for Special Needs and Inclusive Education (2019) has offered support for the identification of weaknesses and, consequently, the changes that must be employed in the Cypriot educational system in terms of legislation, policies, and structures (for details, see SPRA, 2019). Towards this goal, a new system for teacher evaluation has been employed by the Ministry of Education, Sport and Youth (MOECSY; 2019), addressing, among others, self-evaluation processes for schools, a process that has recently been institutionalised by law (MOECSY, 2023).

Considering the above, we argue that the Themis questionnaire, which is designed specifically for the initial phase of school self-evaluation processes (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019), is an appropriate instrument to be introduced to Greek Cypriot schools for measuring teachers’ response to student diversity.

Aim of the Study

The purpose of this study is to adapt and evaluate the questionnaire of the school self-evaluation tool Themis (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019) in the Greek-speaking context. Specifically, in this study, we aim to gather and analyse teachers’ views in Cypriot schools regarding barriers to participation and learning, as well as their responses to diversity. By doing so, we seek to provide a reflective tool for schools to collectively identify areas for improvement and to promote more inclusive practices.

Method

Participants

The participants (Table 1) of the evaluation process of the Greek version of Themis were divided into two groups: (a) the judges (n = 17) and (b) the evaluators (n = 20). For our sampling we used the purposive sampling technique, since the participants of the evaluation process were required to meet specific criteria — that is, researchers with expertise in inclusive education and educational research methods, and school directors and teachers with work experience in mixed-ability classrooms. Among the reviewers of the adapted form of the tool was also an officer from MOECSY, who works as a connecting link between the school and the department of the ministry that is responsible for all affairs related to special education and who encouraged us to continue with the process, pointing out its necessity for Cyprus.

Table 1. The Research Participants

Ethics approval was granted by the Cyprus Pedagogical Institute. Also, written informed consent was collected from the participants who took part in the study.

Instrument

The Themis questionnaire is a component of the Themis Inclusion Tool, designed by Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019), with the goal of providing teachers with a means to reflect on the ways they respond to diversity. This school self-evaluation tool consists of four phases: (a) starting with reflective questions, (b) filling out the questionnaire, (c) analysing the data, and (d) choosing improvement lines. The questionnaire is used during the second phase as part of the broader process of gathering data to support self-reflection and to promote inclusive practices in schools. The design of the instrument was inspired by the Index for Inclusion tool (Booth & Ainscow, Reference Booth and Ainscow2011) as well as three other instruments: (a) the Guía para la reflexion y valoración de prácticas inclusivas (Marchesi et al., Reference Marchesi, Durán, Giné and Hernández2009), (b) the Guía ACADI (Arnaiz & Guirao, Reference Arnaiz and Guirao2015), and (c) the Manchester Inclusion Standard (Moore et al., Reference Moore, Ainscow and Fox2007). As for the tool’s name, it is inspired by the Greek goddess Themis, who symbolises social justice and equality — a view of inclusion that is shared by the authors (Azorín & Ainscow, Reference Azorín and Ainscow2020). The questionnaire consists of three dimensions: contexts (Items 1 to 23), resources (Items 24 to 42), and processes (Items 43 to 65). At the end of the questionnaire there are two open-ended questions: the first refers to three positive and the second to three negative aspects that are considered important with respect to paying attention to diversity in the classroom or school (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019). So far the Themis questionnaire has been translated and adapted to the Portuguese context (Carvalho et al., Reference Carvalho, Cruz, Azevedo and Fonseca2022).

Procedure and Results of Each Phase

In the following, we discuss the steps that we followed for the evaluation of Themis, including the results of each phase, to provide a more detailed description of the procedure conducted by our research team.

Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019) evaluated their newly designed instrument via the content validation process. As the authors explain, this validation process ‘guarantees that the tool (1) actually measures what it purports to measure, (2) fits the aims of the research for which it was designed, and (3) includes all the representative elements of the object under study’ (p. 15). Additionally, according to Hirschmüller et al. (Reference Hirschmüller, Steffen, Fassbender, Clarsen, Leonhard, Konstantinidis, Südkamp and Kubosch2017), content validation is the most appropriate method to evaluate an instrument when cross-cultural adaptation is involved. Thus, we decided to proceed with the adaptation and evaluation of the Themis instrument in the Cypriot Greek-speaking context following the same steps as Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019) — that is, via a content validation process. In fact, the tool’s designers advised us to use the same validation method for the Greek version.

An indicative series of steps for the content validation method is as follows: (a) definition of the aim, (b) selection of the judges, (c) articulated presentation of the tool’s dimensions and indicators, (d) identification of the aim of the evaluation by placing it in a research context, (e) design of the templates, and (f) calculation of the level of agreement (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019). There is no strict series of steps for this method, and similar steps have been suggested by other researchers (Yusoff, Reference Yusoff2019). Instead, the process must be adapted to the context and specific needs of each tool. Figure 1 shows the steps that we followed for the evaluation of the Greek version of Themis.

Figure 1. Themis Evaluation Steps.

Step 1: Preparation of the initial version

First, we translated the questionnaire into Greek. Regarding the translation process, we followed established practices to ensure both linguistic accuracy and cultural appropriateness. As suggested by Wild et al. (Reference Wild, Grove, Martin, Eremenco, McElroy, Verjee-Lorenz and Erikson2005), we utilised forward and backward translation, expert review, and cultural adaptation techniques. These methods, combined with pilot testing, helped ensure the tool’s validity for the target population. To ensure the accuracy of the translation, we relied on two external researchers, who reviewed it. One researcher is an expert in educational research methodology and the other is an expert in the field of inclusive education. With the latter, we held three in-depth discussions, examining the exact translation of the terms related to inclusion. We paid attention to formulating the statements in a way that reflects the original statements as accurately as possible and, at the same time, ensured that the statements could be understood by the teachers who are asked to complete it, adapting questions to the Cypriot Greek-speaking context. According to Beaton et al. (Reference Beaton, Bombardier, Guillemin and Ferraz2000), if a measure is used across cultures, it must be both accurately translated and culturally adapted. For example, in the case of the term ‘co-teaching’, we observed that in the Cypriot educational context, this term is sometimes used to describe the situation where a lesson is taught simultaneously to students from different grades by a single teacher. For this reason, we clarified the meaning of the term in parentheses to ensure its validity at a conceptual level.

Afterwards, we discussed in depth the instrument’s validation method. According to Nunnally (Reference Nunnally1978), one of the most reliable methods of validating psychometric instruments that measure attitudes is content validation, as it is a key element for ensuring that the tool’s content is sufficient. As Colquitt et al. (Reference Colquitt, Sabey, Rodell and Hill2019) note, an important reason why many questionnaires receive negative reviews is because their designers do not focus on the tool’s content validation, preferring instead to use other validation methods.

As Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019) elaborate, the best way to ensure a tool’s content validity is through the inter-judges process, where experienced individuals critically review the tool. The main advantages of this method are its high-quality feedback, ease of use, minimal technical and human resources, and the ability to use various information-gathering strategies.

Step 2: The inter-judges process

Towards this direction, we discussed the individuals who would participate in the process as the judges. There does not seem to be a consensus among researchers regarding the appropriate number of judges. Burns and Grove (Reference Burns and Grove1993), for example, suggest five to 10 participants; Landeta (Reference Landeta2002) suggests seven to 30. The number of the judges who participated in our research was 17.

Initially, we considered including only researchers with expertise in education, inclusive education, and educational research, similar to the designers of Themis. However, as we gained a deeper understanding of the content validation method, we decided to enhance the tool’s validation process by including individuals from the target population as judges. This approach aligns with recommendations by Burns and Grove (Reference Burns and Grove1993) and Rusticus (Reference Rusticus2014).

Expert Researchers

We contacted five experts, four of whom agreed to participate (n = 4). The experts were asked to review the adapted questionnaire and provide detailed feedback. Their contributions were invaluable:

  1. 1. Researcher 1 (educational research methodology) provided insights into wording adaptations and the inclusion of explanatory statements. For example, redundant parenthetical phrases next to dimension names were removed to avoid confusion, ensuring simplicity and clarity (Creswell, Reference Creswell2002).

  2. 2. Researcher 2 (educational research methodology) contributed to discussions about preconditions for instrument validation and factor analysis, which informed subsequent steps.

  3. 3. Researcher 3 (inclusive education) suggested clarifications for terms like ‘inclusion’ and emphasised the need to explain underutilised but impactful teaching methods (e.g., peer tutoring).

  4. 4. Researcher 4 (primary education) highlighted the need for simplicity, noting that many teachers may lack a clear understanding of inclusion, and offered suggestions to refine the questionnaire’s contextual relevance.

The experts’ feedback was systematically documented in a field diary and summarised in Table 2.

Table 2. Indicative Experts’ and Researchers’ Comments

Target Population

To further validate the questionnaire, following the purposive sampling technique, we engaged seven school principals (n = 7) and one ministry official. These participants reviewed the questionnaire and provided feedback on its applicability to the Cypriot Greek-speaking context. A summary of their comments is presented in Table 3. Key issues raised included the inapplicability of certain items (e.g., Questions 19, 37, and 39) due to structural or jurisdictional constraints within the Cypriot educational system and suggestions to refine or remove unclear statements (e.g., Question 33 on computer lab availability).

Table 3. School Principals’ Comments

Subsequently, we involved five elementary school teachers (n = 5). Their feedback echoed the school principals’ concerns about certain items and highlighted the potential of the tool as a reflective instrument for addressing diversity in schools. A summary of their comments is presented in Table 4.

Table 4. Teachers’ Comments

Refinements

Based on the feedback, we removed Items 19, 20, 37, 38, and 39, which were deemed irrelevant to the Cypriot context. Although Item 34 was debated, we retained it with adjustments, considering its broader implications for access to information and communication. Additional clarifications, such as defining the ‘Senior Leadership Team’, were added to ensure clarity.

Tables 3 and 4 summarise the comments from school principals and teachers, respectively, and the overall refinements are detailed in the Discussion.

Step 3: Aggregated individual method

Based on the above suggestions, a new version of the questionnaire was drafted and handed out for a second evaluation round. In this phase, the aggregated individual method was followed, according to the steps that Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019) suggest. This method suggests the choice of reviewers, who are asked to judge specific aspects of the tool independently (e.g., regarding format adequacy; Azorín & Ainscow, Reference Azorín and Ainscow2020) using a Likert scale of 1 to 4, where (1) means none, (2) little, (3) some, and (4) a lot. The reviewers’ sample consisted of 20 individuals. Every reviewer received information about the tool’s purpose, its theoretical background, and the evaluation template based on the aggregated individual method (Corral, 2009, as cited in Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019).

The evaluators’ group comprised three English language Cypriot philologists, who specifically examined the translation of the items, four school principals, and 13 teachers. We used the same evaluation template as the tool’s designers, with the exception of two elements, which were to be removed (categories and reflection questions), according to the current version of the adapted tool. The collected data were analysed with Version 1.2 of the Jamovi statistics package (The Jamovi Project, 2020). Table 5 shows the number of reviewers (N), the mean (M), and median (Mdn) scores, and the standard deviations (SD).

Table 5. Means, Medians, and Standard Deviations for the Various Sections of the Template

As shown in Table 6, the lower rated aspects are related to the impact (M = 3.55), the interest that the tool is expected to raise among teachers (M = 3.65), and the tool’s relevance to the Cypriot educational system reality (M = 3.65). These data support the view that in Cyprus teachers are not yet familiar with inclusive education. The overall scores indicate that this particular version of the questionnaire can be used as the definitive version of the adapted tool of Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019) in the Greek-speaking content. Comparing the results of this version of the tool with the results of the original form of Themis by Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019), ours is satisfactory, as indicated in Table 6.

Table 6. Comparison of Azorín et al.’s (Reference Azorín, Ainscow, Sánchez and Goldrick2019) Study and Our Study’s Results Via the Aggregated Individual Method

Additionally, Cronbach’s alpha was calculated to assess the internal consistency reliability of the instrument. The results demonstrated high reliability for the overall scale (α = .940) as well as for its three subscales: Dimension A (α = .861), Dimension B (α = .840), and Dimension C (α = .896).

Discussion

According to Ainscow (Reference Ainscow2020), the increasing, although still insufficient, efforts of many countries to successfully implement inclusive education shed light on the necessity of using school self-evaluation tools more broadly. As the author puts it, ‘moves towards inclusion are about the development of schools, rather simply involving attempts to integrate vulnerable groups of students into existing arrangements’ (p. 9). For this reason, (numerous) countries around the world (e.g., England and Spain) have introduced school self-evaluation processes for the development of their educational school systems, with great results (Azorín & Ainscow, Reference Azorín and Ainscow2020).

In this respect, the contribution of this research is twofold: first, we introduce to Greek-speaking teachers a ‘diagnostic tool’ (Azorín et al., Reference Azorín, Ainscow, Sánchez and Goldrick2019) that enables a better understanding of their school’s strengths and weaknesses regarding inclusion (Azorín & Ainscow, Reference Azorín and Ainscow2020) by measuring teachers’ actual attitudes towards student diversity. In particular, evaluating the Themis questionnaire for use in the Cypriot Greek-speaking context addresses the lack of validated and reliable tools (Gheyssens et al., Reference Gheyssens, Consuegra, Engels and Struyven2020) for measuring teachers’ response to student diversity, as well as an introduction to school self-evaluation processes — that is, the identification of the schools’ profile focusing on aspects related to inclusive education. Such a collective reflection process motivates teachers to familiarise themselves with and experience the collaborative culture of a school self-evaluation process, which is in line with the philosophy of inclusive culture (Ainscow, Reference Ainscow2020).

Second, the contribution of this research lies in providing an example of the content validation methodological approach for validating and evaluating a measurement tool used in the field of inclusive education. As Yusoff (Reference Yusoff2019) notes, the content validation method is being increasingly recognised as a vital tool for research. To that, Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019) underline that this specific process of content validation ‘elicits opinions as scientific data within a formal process in which the information received guides later decision making’ (p. 15) for tools that (a) measure attitudes and (b) are used in educational contexts with different cultural characteristics.

Cyprus is a European country where, despite efforts, research reveals considerable resistance to fully implementing inclusive education (Symeonidou, Reference Symeonidou2022). This resistance is often linked to teacher-related factors, such as a lack of inclusive culture, collaboration difficulties, and misconceptions about inclusive practices (Lyra et al., Reference Lyra, Koullapi and Kalogeropoulou2023). At the same time, Cyprus is facing an increasingly complex educational landscape, as schools must respond to a highly diverse student population. This includes not only students from different linguistic and cultural backgrounds, resulting from the flow of refugees, but also students with disability. Under these circumstances, schools are striving to implement inclusive education effectively, following international conventions and best practices. The introduction of school self-evaluation processes by the Ministry of Education as part of teacher evaluation, along with the emphasis on inclusive school development (MOECSY, 2023), signals an optimistic shift in the Cypriot educational context.

In this context, the necessity of the Themis questionnaire becomes even more evident. This tool provides a means to assess teachers’ attitudes and practices towards inclusion, directly addressing the challenges schools face in responding to the diverse needs of their students. The feedback gathered through its use can guide schools in identifying strengths and weaknesses in their approach to inclusive education, thus supporting their efforts to effectively meet the needs of all students, including those with varying cultural, linguistic, and disability-related backgrounds. The representative of the Ministry of Education who participated in the adaptation of the tool as an expert noted that this research fills a crucial need for evaluating inclusiveness in schools with a linguistically and culturally appropriate tool. Her motivating comment, ‘Go ahead with this questionnaire. It deals with all aspects of education, and you use targeted and clear vocabulary in it’, along with other participants’ feedback, underscores the demand for reliable tools to support efforts in changing school culture and practices towards inclusion. This need resonates with findings from studies such as Azorín et al. (Reference Azorín, Ainscow, Sánchez and Goldrick2019) and Carvalho et al. (Reference Carvalho, Cruz, Azevedo and Fonseca2022). The same positive response was also reflected in the comments of the evaluators who participated in the validation process of the initial version of the tool. They, too, highlighted the usefulness of this questionnaire, noting, among other things, its potential to support inclusive education efforts. Further, the utility of the tool is also indicated by the results of the ‘aggregated individuals’ phase of our research, where the mean of elements, such as the tool’s relevance to the Cypriot educational system reality was low (M = 3.65) and the corresponding mean of the tool’s evaluation in the Spanish context by its designers was even lower (M = 3.38). This result indicates that the two educational contexts face common challenges regarding teachers’ response to the implementation of inclusive education, showing that much effort is still required for teachers’ familiarisation with inclusive practices.

As presented in Table 6, the comparison between Azorín et al.’s (Reference Azorín, Ainscow, Sánchez and Goldrick2019) original version of the tool and the Greek Cypriot adapted version reveals some notable differences in the evaluation scores across various elements. For instance, the mean score for ‘impact’ in the Cypriot version (M = 3.55) is slightly higher than in the Spanish context (M = 3.38), indicating a relatively better perceived impact of the adapted version in Cyprus. Similarly, ‘usefulness’ shows a marked improvement, with the Cypriot version scoring M = 3.85 compared to M = 3.38 in the original tool, highlighting the increased relevance and utility of the tool for the Greek Cypriot context.

Additionally, elements such as ‘format’, ‘instructions’, and ‘clarity’ exhibit significant positive shifts in the Greek Cypriot version, suggesting that adaptations made to the questionnaire, including language modifications and contextual adjustments, were well-received by the Cypriot respondents. For example, the ‘format’ element increased from M = 2.92 in the original version to M = 3.75 in the adapted version, reflecting a clearer and more user-friendly structure for Cypriot teachers.

In contrast, the ‘relevance’ and ‘interest’ elements show modest improvements, with the Cypriot version scoring M = 3.65 in both categories, as compared to M = 3.38 and M = 3.85, respectively, in the Spanish version. This suggests that although the adapted tool holds significant relevance for the Cypriot educational system, there are still areas that might benefit from further contextual alignment to fully meet the needs of the local educational environment.

Overall, the results highlight the positive reception of the adapted version, suggesting that the adjustments made to the tool have improved its relevance and clarity, though some areas could still be refined further to enhance its impact and utility in the Cypriot context.

Similar results also emerged in the Portuguese study, where, according to Carvalho et al. (Reference Carvalho, Cruz, Azevedo and Fonseca2022), the use of Themis showed that Portuguese teachers still face challenges regarding the three dimensions of the tool (processes, context, resources). Consequently, the fact that the Themis questionnaire brings to the surface the weaknesses of each school proves the need for using instruments designed specifically for analysing these domains (processes, context, resources) in which teachers face challenges 30 years after the international declaration on inclusive education (Azorín & Ainscow, Reference Azorín and Ainscow2020), guiding them through the self-evaluation process towards the development of their school (Ainscow, Reference Ainscow2020).

In conclusion, the evaluation of the Themis questionnaire is part of a larger research project regarding school self-evaluation processes. The next step of our ongoing research is the distribution of the validated Themis questionnaire to Greek Cypriot teachers on a broader scale, and the discussion of the results, with the goal of contributing to Cypriot schools’ efforts for moving forward their ‘journey to inclusion’ (Azorín & Ainscow, Reference Azorín and Ainscow2020).

Footnotes

This manuscript was accepted under the Editorship of Umesh Sharma.

References

Ainscow, M. (2020). Promoting inclusion and equity in education: Lessons from international experiences. Nordic Journal of Studies in Educational Policy, 6(1), 716. https://doi.org/10.1080/20020317.2020.1729587 CrossRefGoogle Scholar
Arnaiz, P., & Guirao, J. M. (2015). La autoevaluación de centros en España para la atención a la diversidad desde una perspectiva inclusiva: ACADI. Revista Electrónica Interuniversitaria de Formación del Profesorado, 18(1), 45101. https://doi.org/10.6018/reifop.18.1.214341 CrossRefGoogle Scholar
Azorín-Abellán, C. M. (2018). The journey towards inclusion: Exploring the response of teachers to the challenge of diversity in schools. Revista Colombiana de Educación, 75, 3958. https://doi.org/10.17227/rce.num75-8100 CrossRefGoogle Scholar
Azorín, C., & Ainscow, M. (2020). Guiding schools on their journey towards inclusion. International Journal of Inclusive Education, 24(1), 5876. https://doi.org/10.1080/13603116.2018.1450900 CrossRefGoogle Scholar
Azorín, C. M., Ainscow, M., Sánchez, P. A., & Goldrick, S. (2019). A tool for teacher reflection on the response to diversity in schools. Profesorado, Revista de Curriculum y Formación del Profesorado, 23(1), 1136. https://doi.org/10.30827/profesorado.v23i1.9142 CrossRefGoogle Scholar
Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 31863191. https://doi.org/10.1097/00007632-200012150-00014 CrossRefGoogle ScholarPubMed
Booth, T., & Ainscow, M. (2002). Index for inclusion: Developing learning and participation in schools. Centre for Studies on Inclusive Education (CSIE). https://www.eenet.org.uk/resources/docs/Index%20English.pdf Google Scholar
Booth, T., & Ainscow, M. (2011). Index for inclusion: Developing learning and participation in schools (Rev. ed.). Centre for Studies on Inclusive Education (CSIE).Google Scholar
Burns, N., & Grove, S. K. (1993). The practice of nursing research: Conduct, critique and utilization (2nd ed.). W. B. Saunders.Google Scholar
Carvalho, M., Cruz, J., Azevedo, H., & Fonseca, H. (2022). Measuring inclusive education in Portuguese schools: Adaptation and validation of a questionnaire. Frontiers in Education, 7, Article 812013. https://doi.org/10.3389/feduc.2022.812013 CrossRefGoogle Scholar
Colquitt, J. A., Sabey, T. B., Rodell, J. B., & Hill, E. T. (2019). Content validation guidelines: Evaluation criteria for definitional correspondence and definitional distinctiveness. Journal of Applied Psychology, 104(10), 12431265. https://doi.org/10.1037/apl0000406 CrossRefGoogle ScholarPubMed
Creswell, J. W. (2002). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (2nd ed.). Prentice Hall.Google Scholar
Damianidou, E., & Phtiaka, H. (2018). Implementing inclusion in disabling settings: The role of teachers’ attitudes and practices. International Journal of Inclusive Education, 22(10), 10781092. https://doi.org/10.1080/13603116.2017.1415381 CrossRefGoogle Scholar
European Agency for Special Needs and Inclusive Education. (2019). Cyprus country policy review and analysis. https://www.european-agency.org Google Scholar
Gheyssens, E., Consuegra, E., Engels, N., & Struyven, K. (2020). Good things come to those who wait: The importance of professional development for the implementation of differentiated instruction. Frontiers in Education, 5, Article 96. https://doi.org/10.3389/feduc.2020.00096 CrossRefGoogle Scholar
Hargreaves, A. (2003). Teaching in the knowledge society: Education in the age of insecurity. Teachers College Press.Google Scholar
Hirschmüller, A., Steffen, K., Fassbender, K., Clarsen, B., Leonhard, R., Konstantinidis, L., Südkamp, N. P., & Kubosch, E. J. (2017). German translation and content validation of the OSTRC Questionnaire on overuse injuries and health problems. British Journal of Sports Medicine, 51(4), 260263. https://doi.org/10.1136/bjsports-2016-096669 CrossRefGoogle ScholarPubMed
House of Representatives. (1999). The Integration and Training of Children with Special Needs Act, N. 113(I)/1999 (I)/1999 Law (Government Gazette n. 3340, 28/07/1999). Government Printing Office.Google Scholar
The Jamovi Project. (2020). Jamovi (Version 1.2) [Computer software]. https://www.jamovi.org Google Scholar
Kefallinou, A., Symeonidou, S., & Meijer, C. J. W. (2020). Understanding the value of inclusive education and its implementation: A review of the literature. Prospects, 49(3–4), 135152. https://doi.org/10.1007/s11125-020-09500-2 CrossRefGoogle Scholar
Kielblock, S. (2018). Inclusive education for all: Development of an instrument to measure the teachers’ attitudes [Unpublished doctoral dissertation]. Macquarie University Sydney, Australia.Google Scholar
Landeta, J. (2002). El método Delphi: una técnica de previsión del futuro [The Delphi method: A technique for forecasting the future] (2nd ed.). Ariel.Google Scholar
Lyra, O. (2012). Führungskräfte und Gestaltungsverantwortung: Inklusive Bildungslandschaften und die Theorie U [Educational leadership and the responsibility for shaping inclusive educational landscapes: The Theory U approach]. Verlag Julius Klinkhardt.Google Scholar
Lyra, O., Koullapi, K., & Kalogeropoulou, E. (2023). Fears towards disability and their impact on teaching practices in inclusive classrooms: An empirical study with teachers in Greece. Heliyon, 9(5), Article e16332. https://doi.org/10.1016/j.heliyon.2023.e16332 CrossRefGoogle ScholarPubMed
Marchesi, A., Durán, D., Giné, C., & Hernández, L. (2009). Guía para la reflexión y valoración de prácticas inclusivas. http://www.oei.es/inclusivamapfre/Guia.pdf Google Scholar
Messiou, K. (2017). Research in the field of inclusive education: Time for a rethink? International Journal of Inclusive Education, 21(2), 146159. https://doi.org/10.1080/13603116.2016.1223184 CrossRefGoogle Scholar
Ministry of Education, Sport and Youth. (2019). Διαμόρφωση νέου συστήματος Αξιολόγησης του Εκπαιδευτικού Έργου και των Εκπαιδευτικών [Development of a new evaluation system for educational work and educators]. https://www.pio.gov.cy/assets/pdf/%CE%91%CE%9E%CE%99%CE%9F%CE%9B%CE%9F%CE%93%CE%97%CE%A3%CE%97%20%CE%95%CE%9A%CE%A0%CE%91%CE%99%CE%94%CE%95%CE%A5%CE%A4%CE%99%CE%9A%CE%9F%CE%A5%20%CE%95%CE%A1%CE%93%CE%9F%CE%A5%20%CE%9A%CE%91%CE%99%20%CE%95%CE%9A%CE%A0%CE%9 Google Scholar
Ministry of Education, Sport and Youth. (2023). ϒιοθέτηση Ενιαίου Σχεδίου βελτίωσης για την αυτοαξιολόγηση των σχολικών μονάδων [Adoption of a comprehensive improvement plan regarding the self-evaluation process of school units]. https://enimerosi.moec.gov.cy/archeia/1/ypp15942a Google Scholar
Moore, M., Ainscow, M., & Fox, S. (2007). The Manchester Inclusion Standard. Manchester City Council.Google Scholar
Nunnally, J. C. (1978). An overview of psychological measurement. In B. B. Wolman (Ed.), Clinical diagnosis of mental disorders: A handbook (pp. 97–146). Plenum Press. https://doi.org/10.1007/978-1-4684-2490-4_4 CrossRefGoogle Scholar
Quddus, L., Khalid, M., & Khan, M. W. A. (2019). Teachers’ self-assessment of their teaching effectiveness at higher secondary level in Pakistan: A case study. KnE Social Sciences, 3(22), 807817. https://doi.org/10.18502/kss.v3i22.5089 Google Scholar
Rusticus, S. (2014). Content validity. In A. C. Michalos (Ed.), Encyclopedia of quality of life and well-being research (pp. 1261–1262). Springer. https://doi.org/10.1007/978-94-007-0753-5_553 CrossRefGoogle Scholar
Symeonidou, S. (2022). Unpacking ableist discourses in Cypriot education policy during the pandemic. European Educational Research Journal, 22(6), 834851. https://doi.org/10.1177/14749041221117885 CrossRefGoogle Scholar
Wild, D., Grove, A., Martin, M., Eremenco, S., McElroy, S., Verjee-Lorenz, A., & Erikson, P. (2005). Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: Report of the ISPOR Task Force for Translation and Cultural Adaptation. Value in Health, 8(2), 94104. https://doi.org/10.1111/j.1524-4733.2005.04054.x CrossRefGoogle ScholarPubMed
Yusoff, M. S. B. (2019). ABC of content validation and content validity index calculation. Education in Medicine Journal, 11(2), 4954. https://doi.org/10.21315/eimj2019.11.2.6 CrossRefGoogle Scholar
Figure 0

Table 1. The Research Participants

Figure 1

Figure 1. Themis Evaluation Steps.

Figure 2

Table 2. Indicative Experts’ and Researchers’ Comments

Figure 3

Table 3. School Principals’ Comments

Figure 4

Table 4. Teachers’ Comments

Figure 5

Table 5. Means, Medians, and Standard Deviations for the Various Sections of the Template

Figure 6

Table 6. Comparison of Azorín et al.’s (2019) Study and Our Study’s Results Via the Aggregated Individual Method