Personnel Technological Proficiency

CDE’s Personnel Technological Proficiency Surveys: Assessing the Technological

Proficiency of Teachers, Librarians/Media Specialists, and School Administrators

Nazanin Mohajeri-Nelson, DeLilah Collins, George Thornton, and Uma Kedharnath

Colorado Department of EducationJuly 29, 2010

Personnel Technological Proficiency

•Title II, Part D of ESEA•Program performance reporting

•Education Data Exchange Network (EDEN) - N071•Personnel Skilled in Technology

•The unduplicated headcount of school personnel assessed against the standards-based performance profiles of technology user skills as defined by the state.

Citation:U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, N071 – Personnel Skilled in Technology (Headcount) File Specifications, Washington, D.C., 2010

U.S. Department of Education – EDFacts Requirement


•Who must report data?•LEAs that receive Title II, Part D funds

•Who has to be assessed?•All full-time school-based instructional personnel and school administrators

•What methods of assessment are accepted?•Certifications•Endorsements•College Credits/Course Completion•Observations•Self-report checklist, test, or assessment (CDE option)

Federal Requirement

•All teachers = Full-time professional school staff members who provide instruction to students and maintain daily student attendance records.

•Librarians/Media Specialists = Professional staff members and supervisors assigned specific duties and school time for professional library and media service activities including selecting, acquiring, preparing, cataloging, and circulating books and other printed materials; planning the use of library and medial services by students, teachers and other members of the instructional staff; and guiding individuals in their use of media services and library materials.

•School administrators = Staff members whose activities are concerned with directing and managing the operation of a particular school including principals, assistant principals, and other assistants; and persons who supervise school operations, assign duties to staff members, supervise and maintain records of the school, and coordinate school instructional activities with those of the LEA, including department chairpersons.

USDE’s definition of personnel



What CDE did to prepare the state•Collaboration with Technology Leadership Forum

•Subcommittee – started on 5/9/08•May 2009 learned that report due by 9/30/10

•Decision to include in HR/Sped•Many decisions already addressed in HR/Sped

- Job classification codes - Full time as defined by the state (.85)

•One year of data collection with option of “not yet assessed”

•Feedback from TLF, CEA, and CoP•EDAC approval – September 2009•HR/SPED data collection

•Four Trainings + Webinar

Historical Context


•Using District measures (if any available)•New Fields in the HR/Sped collection:

•Technology Proficiency Level•Proficient, Not Proficient, Not assessed

•Type of Assessment Used•Self assessment, performance-based assessment, peer assessment, supervisor assessment, multiple, and not assessed

•Date of Technology Assessment

https://cdeapps.cde.state.co.us/appendix_hrdefs.htm

Data Collection from Year 1


Year 1 Data ~ still being cleaned and prepared for analyses


•International Society of Technology in Education (ISTE) Standards

•National Education Technology Standards for Teachers (NETS·T)•National Education Technology Standards for Administrators (NETS·A)

•Assessment Expert •Converted Evaluation Rubrics into Self-Assessment Surveys•Conducted Pilot Study to test reliability and validity of the surveys

Creation of the Assessment

Methods for Creating and Testing the Self-Assessments

• Used ISTE standards to create 2 self-assessments– Teacher Survey [33 22]– Administrator Survey [48 27]– 5 cluster of questions aligned with the technology standards

• Included demographics, list of definitions for terms within the assessments, and essential conditions

• Pilot data collected online using SurveyMonkey• In addition to self-ratings on the survey, qualitative

data on the survey (feedback) was collected from pilot study participants


Methods for Analyzing the Self-Assessments

• Demographic data (e.g., years of experience, exposure to technology, school size and setting)

• Reliability analyses• Relationships between self assessments and selected

demographics• Qualitative analyses of the write-in comments



Demographics of Respondents (Teacher Survey)•Job title (N=210)

•Teachers = 172•Librarians/Media Specialists = 18•Technology Staff = 10•Other (e.g., PD director) = 10

•Full time = 96% •Had some tech courses in past 5 years = 80%•Master’s degree = 66%

Pilot Study


Demographics of Respondents (Administrator Survey)•Job title (N=72)

•Superintendents = 2•Principals= 27•Asst Principals= 3•Manager/Directors (e.g., Sped, Instruction, Tech) = 15•Program Coordinators = 8•Others = 17

•Full time = 97%•Had taken tech courses in past 5 years = 86%•Master’s degree = 93%

Pilot Study


Reliability was moderate to high based on •Correlations between items within each cluster•Internal consistency (reliability coefficients above .86)•Change in coefficient alpha for a cluster if an item was removed

Means and Standard Deviations were used to determine if there was sufficient variability in responses (no ceiling effects, etc.).

Pilot Study Results


Relationship between survey questions and demographic information was tested by regressing responses on demographic information.

Low correlation

Pilot Study Results


Survey items were revised based on

•Frequency counts (how often a response was selected)•Range of responses (variability of responses to each item)•Results of reliability analyses•Qualitative feedback from survey participants

After Pilot Study


Revised survey questions were reviewed and critiqued by Subject Matter Experts to determine validity

•Relevance to measuring tech proficiency•Relevance to tech standards•Importance of each item•Clarity and understandability of the items•Whether the item clusters sufficiently cover competency on that standard (additional questions needed?)

Subject Matter Expert Review


•Available on CDE website•Will be available as online survey in Fall

Questions in final form


Online System for Data Collection

Personnel Technological Proficiency Self Assessment Survey•Production of the PTP Self Assessment Survey is underway•Survey will consists of Self assessment questions, definitions, and essential conditions questions•If LEA chooses to use this survey, this will eliminate the need to report proficiency data into the HR/SPED collection system•Survey tool should be available in the fall of 2010•Survey tool will provide proficiency information at the end of the survey•Districts will have the ability to request data during the HR/SPED collection period to determine the number of staff members who have not taken the survey


•Meet federal reporting requirements•Needs assessment for future PD plans

Plans for the Data


•Districts can use many other options•District-made assessment tool•Technology endorsements•College transcripts•Performance on technology tasks•Supervisor evaluation (e.g., using rubrics from ISTE)•Peer evaluation

Alternatives to CDE Assessments

Additional information regarding this reporting requirement, including the self-assessment survey questions, can be found at:•www.cde.state.co.us/edtech/PersonnelTechnologicalProficiency.htm


Contacts:

•Nazanin (Nazie) Mohajeri-Nelson ~ •[email protected]•(303) 866-6205

•DeLilah Collins ~ •[email protected] •(303) 866-6850



Personnel Technological Proficiency

Documents

Transcript of Personnel Technological Proficiency