The Development of Deaf CAMERA
Introduction
This history describes the adaptation of CAMERA (Communication and Math Employment Readiness Assessment) for use in Deaf-stream literacy. It begins with the first inspiration to consider adapting the assessment and follows through to the point of implementation.
Initial Interest in the Assessment
Peggy Anne Gordon of Deaf Literacy Initiative (DLI) was very interested in helping Deaf literacy learners build the essentials skills needed to succeed in the workplace. She was looking for an assessment that would support programs with a workforce focus. Peggy Anne learned about CAMERA and decided to meet with PTP Adult Learning and Employment Programs to find out more about it.
In 2006, there were a few meetings between DLI and PTP, and Peggy Anne learned more about the assessment. She liked that CAMERA assessed reading, document use, writing and numeracy using real tasks found in the workplace. She felt that CAMERA would be a good assessment for Deaf learners who were preparing for employment, and she decided to find out what the learners and practitioners thought about the test. Peggy Anne asked PTP if there would be an opportunity to find out if CAMERA could be adapted for Deaf learners.
This led to a decision to carry out a feasibility study.
Feasibility Study
In the summer of 2007, DLI conducted a small field test. Deaf learners tried out the CAMERA test components and gave their feedback. This was an important part of the feasibility study because the purpose of the study was to see if CAMERA could work for Deaf learners.
The field test took several days because a lot of Deaf learners wanted to try out the assessment. The learners were told that they would be helping researchers decide if CAMERA could be adapted for the Deaf literacy community. They were asked to share their thoughts and impressions of the test. Because this was an information gathering process, learners were invited to ask questions, seek clarification, offer feedback and comment on the content and procedures. Following each assessment, the learner told a member of the research team what he or she thought about the assessment experience.
There were 27 learners in the field test, and they all completed the assessment. They enjoyed interacting with the assessor and gave very good feedback on how the content and procedures could be made more appropriate for a Deaf population. It was a great experience for everyone who was involved.
These are some of the comments that the Deaf learners made about CAMERA:
“This test seems to me much more like the real workplace.”
“The test was progressively more difficult for me. I like the challenge. This is a hearing framework, so I have to think in ASL to frame the questions so that I understand.”
“The writing gave us a chance to articulate our thoughts. We like the opportunity to write our ideas.”
“I really enjoyed the challenge and I did learn something. The test got more difficult than easier. It was a good test.”
“Why not go with something that is challenging? We want to progress. We like a challenge, the opportunity to stretch, advance, carry on.”
Overall, the feasibility study showed that learners had a good experience taking the assessment and that CAMERA would provide a good foundation for Deaf stream literacy assessment. All learners were able to complete the assessment and most of them felt that the test presented a relevant and positive challenge for them.
The field test also confirmed that many parts of the CAMERA administration and content would need to be changed to suit Deaf learners. It would be necessary to carefully adapt each test component to make it compatible with the needs and interests of the Deaf community.
Based on the feasibility study, an adaptation team was assembled with a strong commitment to create an assessment that would be fair for members of a diverse Deaf community. This was very exciting because it was the beginning of a collaboration that ultimately produced Deaf CAMERA test forms for Stages 1, 2 and 3.
The adaptation team included DLI and PTP staff and consultants with expertise in the following areas:
-
Literacy and Essential Skills
-
Deaf literacy programming, curriculum, and instruction
-
CAMERA content, procedures and scoring
-
Test development and adaptation
-
Workplace and workforce preparation
-
Employment-focused programming
-
Deaf culture and Deaf literacy
There was also an advisory committee of experts in the Deaf literacy field who reviewed the assessment content and gave their feedback. The experts said that CAMERA included the kinds of authentic work-related tasks that would be suitable for Deaf-stream literacy. They had confidence in CAMERA because it had been standardized, validated and used successfully in Anglophone literacy programs for almost a decade. They thought CAMERA could be useful for assessing Deaf learners.
The experts were:
Bruce Belcher: Mohawk College DEP
Deanne Bradley-Coelho: George Brown College
Yvonne Brown: Durham Deaf Services
Jessica Cano-Jauregui: Canadian National Institute for the Blind, Toronto
Simone Edwards-Forde: Alternative Learning Styles and Outlooks, Ottawa
Peggy-Anne Gordon: Deaf Literacy Initiative
David Hamen: Durham Deaf Services
Shirley Henley: Niagara Adult Literacy Program for the Deaf
Judith Julien: Canadian National Institute for the Blind, Toronto
Joyce McHugh: Canadian Hearing Society, Toronto
Pat Morano: Canadian Hearing Society, Toronto
Janet Morden: Niagara Adult Literacy Program for the Deaf
Christine Nelson: Bob Rumball Centre for the Deaf
Patrizio Presenza: Canadian Hearing Society, Sudbury
Susan Toews: George Brown College
This advisory group selected the tasks that would be included in Deaf CAMERA Stage 1, and they made suggestions for making the tasks more suitable for Deaf learners.
Adaptation of Stage 1
CAMERA Stage 1 was the first instrument adapted. Tasks for this test were chosen by the advisory committee based on their suitability. If a task was not considered suitable, changes were made, or a new task was developed. Procedures and instructions were also adapted to make them accessible for Deaf learners.
The adaptation team used feedback gathered during field-testing and recommendations of the advisory committee to adapt the test. The team kept the cultural needs of the Deaf learners and the test objectives in mind as they modified the content and procedures.
The protocol for administering Deaf CAMERA Stage 1 was drafted in English print, documenting all of the instructions that would need to be communicated and all of the demonstrations, examples and explanations that would be required. This draft script was reviewed by advisory committee members to determine the most appropriate and accessible ASL terminology to convey information to test takers. After much discussion, the script was revised, and then it was presented in ASL on a DVD to support the next phase of test development, which was pilot testing.
The Deaf literacy community was very supportive of the pilot testing activities. Many programs were willing to take part. Here is the list of sites where piloting was done:
Mohawk College, Hamilton
George Brown College, Toronto
Bob Rumball Centre for the Deaf, Toronto
Canadian National Institute for the Blind (CNIB), Toronto
Canadian Hearing Society, Sudbury, Thunder Bay and Toronto
Alternative Learning Styles and Outlooks, Ottawa
The pilot results were promising both in terms of administration and scoring. Results indicated that Deaf CAMERA could be administered in a way that is fair and accessible to a wide range of Deaf learners. Responses to the test tasks showed that CAMERA could provide meaningful results for reading, writing, document use and numeracy.
First Introduction to the Field
DLI and PTP prepared a joint presentation that was delivered at a conference in the fall of 2011. The purpose of this presentation was to introduce Deaf CAMERA to the field and explain the features and benefits of the assessment. There was a large audience and the presentation was well received. Stakeholders and practitioners were interested in CAMERA and asked insightful questions about its administration, results, and implementation.
In early 2012, the Deaf CAMERA Stage 1 test package was completed and ready for use.
Adaptation of Stage 2 and 3
Because the adaptation of CAMERA Stage 1 was so successful, and because there were many learners with stronger literacy skills who would benefit from being assessed with Deaf CAMERA, DLI decided to adapt Stage 2 and 3 of PTP’s CAMERA assessment.
The adaptation of Deaf CAMERA Stage 2 and 3 was carried out in 2013-2014. The goal of this project was to adapt one form for Stage 2 and one form for Stage 3.
The team began by selecting the forms that would be most suitable and would have the most relevance and significance for the Deaf community. To do this, Deaf literacy experts were consulted in two meetings.
The first feedback meeting was with Deaf literacy expert Linda Wall. Linda’s feedback was facilitated by the project manager and the test developer. During this in-depth process, Linda was invited to review all of the tasks and items in each test form and to discuss the content with Anselmo DeSousa, Master Assessor and test development team member. Linda pointed out which topics and tasks would be most relevant and accessible for Deaf learners, and she also suggested helpful revisions to improve the content.
A second feedback meeting was held with Deaf literacy expert Bruce Belcher. Bruce selected the same forms of the test that were preferred by Linda and Anselmo. This added assurance that the most appropriate content had been chosen. Bruce also made good suggestions for further refinements, which were incorporated into the adaptation.
Based on the feasibility study and the expert feedback, the content and procedures of Deaf CAMERA Stage 2 and Stage 3 were modified and adapted for Deaf learners. Instructions were prepared in ASL for administration of the tests.
A small field test was conducted at George Brown College and Mohawk College with small groups of learners. Learners were very willing to give their time to try out the test.
The results showed a good range and distribution of overall scores. Writing was the most challenging skill for these groups.
Debriefing Process
Debriefing is an important part of the use of an assessment because it helps the learner know what they can do well and what they need to work on. After a Deaf CAMERA assessment, the assessor scores the test and compiles an overview of the learner’s strengths and areas where improvement is needed. The assessor then meets with the learner and instructor to explain the test results and talk about how the learner can make progress toward his or her goals. A summary sheet is left with the learner.
During the Deaf CAMERA field testing, the debriefing procedures were tried out and refined to ensure that they would meet the needs and expectations of learners and their instructors.
In early 2014, the Deaf CAMERA Stage 2 and Stage 3 test package was completed and ready for the Deaf literacy field.
Summary
Deaf CAMERA was adapted for the Deaf Literacy field because it assesses the essentials skills literacy learners need to succeed in the workplace. Deaf CAMERA results can help learners know what they need to learn next in their workforce-oriented programs as they prepare for the workplace.
Deaf CAMERA was designed and adapted through extensive collaboration with experts and learners in the Deaf community. Content and procedures were tailored to suit this unique literacy stream. Deaf CAMERA Stage 1 follows a face-to-face administration. Each learner is assessed individually by a trained Deaf assessor who provides standardized instructions, clarification and support. This helps to create a very safe and pleasant experience for the learner.
Deaf CAMERA Stages 2 and 3 administration procedures are not face-to-face. Because learners have sufficient levels of skill to work on their own, learners work independently on their test papers for Stage 2 and 3 and hand them in for scoring when they are finished. For this reason, the role of the assessor is to give introductory instructions and to answer questions that arise during the administration.
Deaf CAMERA assessors have an important role in the literacy community. As trained experts, they bring objective feedback to learners and instructors to inform the learning process. Deaf CAMERA assessors must be knowledgeable, experienced, flexible and sensitive to the needs of learners. Currently, there is one trained Deaf CAMERA assessor, Anselmo DeSousa. As the Master Assessor, Anselmo will train and mentor one selected apprentice assessor.
Deaf CAMERA is now ready for a successful roll-out. Now that the intensive period of development and adaptation is over, it is time for the Deaf literacy field to experience the benefits of Deaf CAMERA.
The test will be offered by DLI. Programs can get more information about the test, the cost of assessment and how to make arrangements for testing by contacting the DLI Master Assessor or Executive Director. DLI will also be available for meetings, consultations and presentations regarding Deaf CAMERA.