Hostname: page-component-78c5997874-fbnjt Total loading time: 0 Render date: 2024-11-14T05:21:05.704Z Has data issue: false hasContentIssue false
  • English
  • Français

Design and development of sign language questionnaires based on video and web interfaces

Published online by Cambridge University Press:  27 November 2019

Juan Pedro López Open the ORCID record for Juan Pedro López [Opens in a new window] ,
Marta Bosch-Baliarda ,
Carlos Alberto Martín ,
José Manuel Menéndez ,
Pilar Orero ,
Olga Soler  and
Federico Álvarez
Juan Pedro López*
Affiliation:
Grupo de Aplicación de Telecomunicaciones Visuales, Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madrid, 28040Madrid, Spain
Marta Bosch-Baliarda
Affiliation:
Faculty of Translation, Universitat Autònoma de Barcelona, 08193 Bellaterra, Barcelona, Spain
Carlos Alberto Martín
Affiliation:
Grupo de Aplicación de Telecomunicaciones Visuales, Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madrid, 28040Madrid, Spain
José Manuel Menéndez
Affiliation:
Grupo de Aplicación de Telecomunicaciones Visuales, Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madrid, 28040Madrid, Spain
Pilar Orero
Affiliation:
Faculty of Translation, Universitat Autònoma de Barcelona, 08193 Bellaterra, Barcelona, Spain
Olga Soler
Affiliation:
Faculty of Translation, Universitat Autònoma de Barcelona, 08193 Bellaterra, Barcelona, Spain
Federico Álvarez
Affiliation:
Grupo de Aplicación de Telecomunicaciones Visuales, Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madrid, 28040Madrid, Spain
*
Author for correspondence: Juan Pedro López, E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Conventional tests with written information used for the evaluation of sign language (SL) comprehension introduce distortions due to the translation process. This fact affects the results and conclusions drawn and, for that reason, it is necessary to design and implement the same language interpreter-independent evaluation tools. Novel web technologies facilitate the design of web interfaces that support online, multiple-choice questionnaires, while exploiting the storage of tracking data as a source of information about user interaction. This paper proposes an online, multiple-choice sign language questionnaire based on an intuitive methodology. It helps users to complete tests and automatically generates accurate, statistical results using the information and data obtained in the process. The proposed system presents SL videos and enables user interaction, fulfilling the requirements that SL interpretation is not able to cover. The questionnaire feeds a remote database with the user answers and powers the automatic creation of data for analytics. Several metrics, including time elapsed, are used to assess the usability of the SL questionnaire, defining the goals of the predictive models. These predictions are based on machine learning models, with the demographic data of the user as features for estimating the usability of the system. This questionnaire reduces costs and time in terms of interpreter dedication, as well as widening the amount of data collected while employing user native language. The validity of this tool was demonstrated in two different use cases.

Keywords

AccessibilityHTML5human–computer interactioninterfacesign languagevideo
Type
Research Article
Information
AI EDAM , Volume 33 , Special Issue 4: Intelligent Interaction Design , November 2019 , pp. 429 - 441
Copyright
Copyright © Cambridge University Press 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Allen, T (2015) The deaf community as a “special linguistic demographic”: diversity rather than disability as a framework for conducting research with individuals who are deaf. In Orfanidou, EWoll, B and Morgan, G (eds), Research Methods in Sign Language Studies: A Practical Guide. London, UK: Wiley Blackwell, pp. 21–40.CrossRefGoogle Scholar
Berghs, M, Atkin, K, Graham, H, Hatton, C and Thomas, C (2016) Implications for public health research of models and theories of disability: a scoping study and evidence synthesis. Public Health Research 4, 1–166. https://doi.org/10.3310/phr04080CrossRefGoogle Scholar
Bibal, A and Frénay, B (2016) Interpretability of machine learning models and representations: an introduction. Proceedings of the ESANN, ESANN.Google Scholar
Cavender, A, Ladner, RE and Riskin, EA (2006) Mobileasl: intelligibility of sign language video as constrained by mobile phone technology. Proceedings of the 8th International ACM SIGNACCESS Conference on Computers and Accessibility, pp. 71–78, ACM.CrossRefGoogle Scholar
Ciaramello, FM and Hemami, SS (2011) A computational intelligibility model for assessment and compression of American sign language video. IEEE Transactions on Image Processing 20, 3014–3027.CrossRefGoogle ScholarPubMed
Cooper, M, Reid, LG, Vanderheiden, G and Caldwell, B (2016) Techniques for WCAG 2.0-techniques and failures for web content accessibility guidelines 2.0. W3C note (last version: 7 October 2016). World Wide Web Consortium (W3C), October.Google Scholar
De Meulder, M, Krausneker, V, Turner, GH and Conama, JB (2018) Sign language communities. In Hogan-Brun, G and O'Rourke, B (eds), The Handbook of Minority Languages and Communities. London, UK: Palgrave Macmillan, pp. 207–232.Google Scholar
Domínguez, AB (2017) Educación para la inclusión de alumnos sordos. Revista Latinoamericana de Educación Inclusiva.Google Scholar
EasyTV Consortium (2018) Easy tv: easing the access of Europeans with disabilities to converging media and content. Available at http://easytvproject.euGoogle Scholar
Ewart, J and Snowden, C (2012) The media's role in social inclusion and exclusion. Media International Australia 142, 61–63. https://doi.org/10.1177/1329878X1214200108CrossRefGoogle Scholar
Ferber, R, Sheatsley, P, Turner, AG and Waksberg, J (1980) What is a Survey?. Washington, DC: American Statistical Association.Google Scholar
Fontaine, S (2012) Surveying populations with disabilities. Specific mixed-mode methodologies to include sensory disabled people in quantitative surveys. International Conference on Methods for Surveying and Enumerating Hard-to-Reach Populations, New Orleans, October–November 2012.Google Scholar
Friedman, JH (2002) Stochastic gradient boosting. Computational Statistics & Data Analysis 38, 367–378.CrossRefGoogle Scholar
Greco, GM (2016) On accessibility as a human right, with an application to media accessibility. In Matamala, A and Orero, P (eds), Researching Audio Description. London, UK: Palgrave Macmillan, pp. 11–33.CrossRefGoogle Scholar
Guardino, C and Cannon, JE (2016) Deafness and diversity: reflections and directions. American Annals of the Deaf 161, 104–112. https://doi.org/10.1353/aad.2016.0016CrossRefGoogle ScholarPubMed
Haug, T (2011) Adaptation and Evaluation of a German Sign Language Test. Hamburg University Press.Google Scholar
Haug, T (2015) Use of information and communication technologies in sign language test development: results of an international survey. Deafness & Education International 17, 33–48.CrossRefGoogle Scholar
Haug, T and Mann, W (2007) Adapting tests of sign language assessment for other sign languages – a review of linguistic, cultural, and psychometric problems. Journal of Deaf Studies and Deaf Education 13, 138–147CrossRefGoogle ScholarPubMed
HBB4ALL Consortium (2017) Hbb4all: hybrid broadcast broadband for all. Available at http://pagines.uab.cat/hbb4all/Google Scholar
Helms, W, Arthur, JD, Hix, J and Rex Hartson, D (2006) A field study of the wheel – a usability engineering process model. Journal of Systems and Software 79, 841–858. doi: 10.1016/j.jss.2005.08.023CrossRefGoogle Scholar
Internet Engineering Task Force, IETF (2014) RFC 7159: the javascript object notation (JSON). Data Interchange Format. March, 2014. Available at https://tools.ietf.org/html/rfc7159Google Scholar
Kohavi, R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In International Joint Conferences on Artificial Intelligence, (Vol. 14, pp. 1137–1145).Google Scholar
Lane, HL and Grosjean, F (2017) Recent Perspectives on American Sign Language. Psychology Press.CrossRefGoogle Scholar
Lazar, J, Feng, JH and Hochheiser, H (2017) Research Methods in Human-Computer Interaction. Morgan Kaufmann.Google Scholar
Liu, Y, Zhao, T, Ju, W and Shi, S (2017) Materials discovery and design using machine learning. Journal of Materiomics 3, 159–177.CrossRefGoogle Scholar
Longo, L (2017) Subjective usability, mental workload assessments and their impact on objective human performance. IFIP Conference on Human-Computer Interaction. Cham: Springer, pp. 202–223.CrossRefGoogle Scholar
McGlinn, K, Yuce, B, Wicaksono, H, Howell, S and Rezgui, Y (2017) Usability evaluation of a web-based tool for supporting holistic building energy management. Automation in Construction 84, 154–165.CrossRefGoogle Scholar
McKee, M, Schlehofer, D and Thew, D (2013) Ethical issues in conducting research with deaf populations. American Journal of Public Health 103, 2174–2178. https://doi.org/10.2105/AJPH.2013.301343CrossRefGoogle ScholarPubMed
MDN Web Docs (2019) Web technology for developers: JavaScript. Last updated March 2019. Available at https://developer.mozilla.org/en-US/docs/Web/JavaScriptGoogle Scholar
Moustafa, K, Luz, S and Longo, L (2017) Assessment of mental workload: a comparison of machine learning methods and subjective assessment techniques. International Symposium on Human Mental Workload: Models and Applications, pp. 30–50, Springer, Cham.CrossRefGoogle Scholar
Orero, P, Doherty, S, Kruger, J-L, Matamala, A, Pedersen, J, Perego, E and Szarkowska, A (2018) Conducting experimental research in audiovisual translation (avt): a position paper. JosTrans: The Journal of Specialised Translation 30, 105–126Google Scholar
Oztekin, A, Delen, D, Turkyilmaz, A and Zaim, S (2013) A machine learning-based usability evaluation method for eLearning systems. Decision Support Systems 56, 63–73.CrossRefGoogle Scholar
Petrie, H, Hamilton, F, King, N and Pavan, P (2006) Remote usability evaluations with disabled people. Proceedings of the Sigchi Conference on Human Factors in Computing Systems, pp. 1133–1141, ACM.CrossRefGoogle Scholar
Pyfers, L, Robinson, J and Schmaling, C (n.d.) Deliverable 3.1: signing books for the deaf in EU-countries: state of the art.Google Scholar
Sandler, W and Lillo-Martin, D (2001) Natural Sign Languages. In Aronoff, M and Rees-Miller, J (eds), The handbook of linguistics, 533–562.Google Scholar
Shneiderman, B (2004) Designing for fun: how can we design user interfaces to be more fun? Interactions 11, 48–50.CrossRefGoogle Scholar
Smith, R, Morrissey, S and Somers, H (2010) HCI for the deaf community: developing human-like avatars for sign language synthesis.Google Scholar
Tran, JJ, Kim, J, Chon, J, Riskin, EA, Ladner, RE and Wobbrock, JO (2011) Evaluating quality and comprehension of real-time sign language video on mobile phones. The Proceedings of the 13th International ACM Sigaccess Conference on Computers and Accessibility, pp. 115–122, ACM.CrossRefGoogle Scholar
Tran, JJ, Flowers, B, Risken, EA, Ladner, RJ and Wobbrock, JO (2014) Analyzing the intelligibility of real-time mobile sign language video transmitted below recommended standards. Proceedings of the 16th International ACM Sigaccess Conference on Computers & Accessibility, pp. 177–184, ACM.CrossRefGoogle Scholar
Tran, JJ, Riskin, EA, Ladner, RE and Wobbrock, JO (2015) Evaluating intelligibility and battery drain of mobile sign language video transmitted at low frame rates and bit rates. ACM Transactions on Accessible Computing (TACCESS) 7, 11.Google Scholar
Vinayagamoorthy, V, Allen, P, Hammond, M and Evans, M (2012) Researching the user experience for connected tv: a case study. Chi'12 Extended Abstracts on Human Factors in Computing systems, pp. 589–604, ACM.CrossRefGoogle Scholar
Witten, IH, Frank, E, Hall, MA and Pal, CJ (2016) Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann.Google Scholar
World Federation of the Deaf (2018) World federation of the deaf. Available at https://wfdeaf.orgGoogle Scholar
World Wide Web Consortium, W3C (2017) HTML 5.2 W3C Recommendation, 14 December 2017. Available at https://www.w3.org/TR/html5/Google Scholar
Yoshida, Y, Ohwada, H, Mizoguchi, F and Iwasaki, H (2014) Classifying cognitive load and driving situation with machine learning. International Journal of Machine Learning and Computing 4, 210–215.CrossRefGoogle Scholar