We have completed phase 1 of investigating the possibility of replacing human readers in examinations. This involved analysing past exam papers for suitability with text-to-speech applications (i.e. TextHelp).
- Some subjects are unproblematic for TextHelp.These subjects are Anthrolopogy, Applied Social Studies, Business & Law, Education, English, Philosophy, Politics and Sociology. A couple have minor issues as a result of formatting (e.g tables reading down instead of across).
- Some are completely unsuitable (e.g. scientific formulae, computer-language syntax, language not English). These subjects are Nua Ghaeilge, Chemistry, German, Electronic Engineering, French, Computer Science, Mathematics, Experimental Physics, Maths Physics and Spanish.
I suggest that with both of these sets of subjects, no further investigation or testing is required. Several subjects show potential and need to be further examined: History, Media Studies, Ancient Classics, Geography, Biology, Finance, Economics and Music. These subjects are ambiguous for several reasons;
- Some comprises of unproblematic papers and problematic papers (e.g. Music – text-based papers OK, papers with musical notation unsuitable).
- Some have a mix of text with significant use of mathematical or other symbols within questions (e.g. Economics, Biology).
- Some have heavy use of Latin words both within questions and as part of other parts of questions (Ancient Classics).
Our next move is to work out what risks are associated with using technology to read the papers? We will need to consult both the departments and student-users on this. For example, if a candidate uses TextHelp to read a question with, say, the symbol for infinity, the software won’t read that symbol correctly. We need to establish whether this is a serious issue for the candidate on two fronts:
1. Exam integrity: Could a misunderstanding occur which would result in a candidate making a avoidable error in the exam. How would it be different if a (non-specialist) human reader was present? (Academic Dept/Exams Office)
2. Effect of SLD/disability. Can we reasonably say that a candidate with an SLD would be aware that he/she would need to pay special attention to items such as these, but yet benefit from the technology with the remaining text in the paper which is unproblematic? Could awareness of the limitations of the software be part of the training they receive from us? Are there issues around exam stress/anxiety that need to be considered? (Yes, I’d imagine)
Interesting times! As we have a site licence for TextHelp, we need to go with that (and PDFAloud as it helpfully pops itself into Acrobat) until we are able to consider others software combinations.
by dullhunk on flickr
It’s exam time at NUIM, and so thoughts go to supporting candidates with disabilities in their examinations.
We currently have about 40 different configurations of examination accommodations in our system. These include
- Extra Time: The vast majority of candidates require additional time only, generally 10 minutes per hour. These students are generally those with specific learning difficulties such as dyslexia.
- Smaller Venue: Many candidates benefit from taking their examinations in smaller rooms as opposed to the large exam halls. This group can include those who are easily distracted, require rest breaks or use computers.
- Readers and Scribes: A reader is someone who reads the paper and script to the candidate. A scribe writes down what a candidate dictates. I’ll come back to these later.
- Use of a PC: Candidates with visual difficulties, specific learning difficulties (e.g. dyspraxia) or physical conditions that make writing by hand impractical can use a computer to write their answers.
- Other supports include enlarged papers, coloured papers, different furniture, separate venues.
With almost three hundred candidates to accommodate, you can appreciate that there needs to be a good relationship between the Disability Office, Computer Services and most importantly, the Examinations Office. This solid relationship is in place now, but some years ago, it wasn’t the case. Some years ago, NUIM was faced with rising numbers of students with disabilities who needed to be supported in their examinations. The need to have additional rooms, invigilators, computers and support for these candidates puts strain on existing services and the three offices found they needed to communicate more much earlier.
While it seems very obvious in retrospect, having the three offices talking, sharing expertise and solutions has rsulted in a successful and suatainable model of supporting students in examinations.
- The Disability Office with the student identifies what accomodations are required. An accomodation (or more likely a combination of accomodations) is represented by a code, and the candidate is tagged with this code on our University system. The student signs off on this accommodation.
- The Examinations Office can read this code in the system and they run off a report by an agreed date. The Disability Office and the student need to have signed off on the supports by that date. This allows the Examinations Office to begin thinking about the resources (buildings, staff etc) that will be needed. Note that the Disability Office does not provide or organise resources for the examinations; this is squarely the remit of the Examinations Office. After all, the Examinations Office are best-placed to ensure the integrity of the examinations.
- If computer equipment is needed, the Computer Centre makes arrangements for this. They configure, test and support candidates using standard equipment and software throughout the examinations. If a candidate needs assistive technology such as JAWS or ZoomText, the Disability Office works with the Computer Centre to ensure that this is available and functioning on the exam setup. The Disability Office are on call when assistive technology is being used.
All of this is achieved through early and open conversation. Now in our third year, I’m pleased to report that we have had no ‘crisis calls’ on the BatPhone (so far) this January. The system is working.
Our next step is to see if we can replace human readers and scribes with technology. But that’s a whole other post…
And let’s not get started on promoting alternative forms of assessment (yet).
Increasingly, I’m finding it difficult to say what it is I do*. Just this weekend I was at a party and watched another human being’s eyes glaze over completely as I attempted to explain my post in the university. Sometimes people ask me what I lecture in and I correct them and explain that I don’t lecture. Maybe I should just say ‘technology’ and be done with it!
Either way it’s probably time to change the subject.
* For the record, I’m Assistive Technology Advisor with a communications and research brief. It’s FAR more fun than that sounds though! (I swear… Hey, come back…)
My colleague Martha alerted me to the presence of pediaphon, a site which will create an mp3 of a wikipedia entry. The voice is a little annoying, but it could be a useful way to get your hands on some listening material if you are away from the ATC or Assistive Technology Room in the Library.
Another very useful site is Scribd. On this site, you can upload virtually any text and have it converted to a range of different formats for download, storage or sharing. So, you could upload your say in Word format and have it converted to pdf. Or you could upload a PowerPoint presentation and convert it to mp3 for listening in the site or for download.
The Norwegian government are to make it mandatory from 2009 for all documents published on their websites to be in open standard formats, reports The Inquirer.
From then, all documents will have to available in HTML, PDF (if layout must be preserved) or ODF (if the document is to be edited by the user e.g. a form). “Everybody should have equal access to public information. From 2009 the citizens will be able to chose which software to use in order to gain access to public information” said IT mister Heidi Grande Røys.
This is a good example of good practice in universal design. By making material available in these open standards, it is less likely that users will be unable to access the information. Similarly, this makes the information more easily accessible to those who use assistive technology such as screenreaders. The Access Office at NUIM strives to make all critical information available in HTML or PDF. We currently use rtf for editable text documents, but we are looking into ODF as a replacement.