Tag Archives: technology

Replacing human readers in examinations

We have completed phase 1 of investigating the possibility of replacing human readers in examinations. This involved analysing past exam papers for suitability with text-to-speech applications (i.e. TextHelp).

  • Some subjects are unproblematic for TextHelp.These subjects are Anthrolopogy, Applied Social Studies, Business & Law, Education, English, Philosophy, Politics and Sociology. A couple have minor issues as a result of formatting (e.g tables reading down instead of across).
  • Some are completely unsuitable (e.g. scientific formulae, computer-language syntax, language not English). These subjects are Nua Ghaeilge, Chemistry, German, Electronic Engineering, French, Computer Science, Mathematics, Experimental Physics, Maths Physics and Spanish.

I suggest that with both of these sets of subjects, no further investigation or testing is required. Several subjects show potential and need to be further examined: History, Media Studies, Ancient Classics, Geography, Biology, Finance, Economics and Music. These subjects are ambiguous for several reasons;

  • Some comprises of unproblematic papers and problematic papers (e.g. Music – text-based papers OK, papers with musical notation unsuitable).
  • Some have a mix of text with significant use of mathematical or other symbols within questions (e.g. Economics, Biology).
  • Some have heavy use of Latin words both within questions and as part of other parts of questions (Ancient Classics).

Our next move is to work out what risks are associated with using technology to read the papers? We will need to consult both the departments and student-users on this. For example, if a candidate uses TextHelp to read a question with, say, the symbol for infinity, the software won’t read that symbol correctly. We need to establish whether this is a serious issue for the candidate on two fronts:

1. Exam integrity: Could a misunderstanding occur which would result in a candidate making a avoidable error in the exam. How would it be different if a (non-specialist) human reader was present? (Academic Dept/Exams Office)

2. Effect of SLD/disability. Can we reasonably say that a candidate with an SLD would be aware that he/she would need to pay special attention to items such as these, but yet benefit from the technology with the remaining text in the paper which is unproblematic? Could awareness of the limitations of the software be part of the training they receive from us? Are there issues around exam stress/anxiety that need to be considered? (Yes, I’d imagine)

Interesting times!  As we have a site licence for TextHelp, we need to go with that (and PDFAloud as it helpfully pops itself into Acrobat) until we are able to consider others software combinations.

Blindness & Responsibility

I was reading the healthcare supplement in today’s Guardian which was about looking after one’s eyes when I came across some articles about the impact of blindness/visual impairment on employment prospects. Apparently two thirds of people with severe visual impairments are unemployed and the same number again have no formal qualifications, and this is in a country (the UK) with quite robust disability legislation. I must admit I was a little shocked at those figures. Why should I be shocked? Well, I work as Assistive Technology Advisor in NUI Maynooth, one of seven Irish universities and one that is rapidly establishing a reputation for excellence in accommodating learners with disabilities. On Tuesday, I was in UCC, my alma mater, to attend a meeting of people in the sector involved in the production and distribution of material in alternative formats. 

Alternative format (AF), as you may know, involves ensuring that text is easily available and accessible to those who have difficulty with printed material. AF may include Braille, audio or PDF, but the consensus is that material in simple text or HTML gives users the most flexibility.

At the meeting were representatives from various Irish universities, many of whom gave presentations on the work they do to help ensure that the ‘print disabled’ have access to core and other texts. It was a fascinating morning and I was struck (and heartened) by the passion shown by these people in the various ways they chose to facilitate learners in their academic pursuits.Image of student with visual impairment on flim set at NUI Maynooth

At NUIM, we have only had one student who was blind (so far). we currently have three students with visual impairments but the bulk of students registered with the Disability Office are those with dyslexia, who also benefit hugely from having easy access to material in accessible digital format. From September we will text-to-speech software available on all PCs across campus, which will greatly benefit many students with milder forms of dyslexia. This, along with the increasing use of Moodle by faculty and moves to encourage good practice in the creation of digital documents, is helping to make the environment at the university conducive to many with print disabilities. But not all.

 

Some 66% of blind or partially sighted people of working age are unemployed, and nearly the same number again (67%) have no formal qualifications, according to latest figures.

So, despite all the good work and the availablity of technological solutions, a shockingly high number of people are not getting any formal qualifications in the UK. What’s going wrong?

As people involved in education/technology in education, we need to push the accessibility agenda as much as possible. If a new tool or elearning strategy is being promoted, we need to question its accessibility to all. Last year, a company approached NUIM with an essay writing module that could be added to Moodle. Luckily, the university approached te Disability Office for our opinion on it before they agreed to purchase. It turned out that, while the content was excellent and the interface was pretty, it was Flash-based and caused unnecessary difficulties for learners using screenreaders such as JAWS. We recommended that the company provide a HTML or text-based accessible version or else we could not purchase (interestingly, we cannot by law purchase inaccessible services!). 

There have been situations alas, where institutions have not consulted others about the accessibility of services and gone ahead, putting themselves at legal risk but also excluding the needs of learners. And often needlessly, sometimes all it takes is a little rewrite, a little consideration in formatting, to ensure greater inclusivity. No great sacrifice.

If we are vigilant here, we can encourage good practice in the creation of accessible elearning strategies which benefit all.

And we can get that shocking statistic down to something more reasonable.