A Deep Dive Into the Online Reporting Suite...

Measuring the Power of Learning.®

A Deep Dive Into the Online Reporting Suite

(ORS) for Interim

Assessments

August 21, 2018

1

Measuring the Power of Learning."'

Housekeeping

If you have problems hearing the presentation, dial in on a telephone. Dial-in information:

• in the Event Info tab at the top-left of the screen

• in your registration email ([email protected])

Dialing in for audio is recommended for best sound quality.

All attendees’ lines have been muted due to the high number of participants.

For questions or comments, use the “Q&A” function.

2

mailto:[email protected]


Objectives Today’s session will cover the following topics:

• Reporting Overview

• Online Reporting Suite (ORS)

• Probability Scores

• Interim Data File

• Interpreting Results

3


Reporting Overview


Reporting

• New Online Reporting Suite (ORS)

• Dynamic, real-time reporting system

• District, campus, and student level

• User-friendly graphs and customization

• Filterable results

• Probability results reported in the STAAR Assessment Management

System

• District data file

5

or Result Description Where Provided

Primary testing results and analysis

The results of the primary (fi rst) testing attempt are recorded here within 48 hours of testing. Includes additional reports and statistical analysis functionality.

• Online Reporting Suite (via Reports > OnHne Reporting or Manage Online Testing > Manage Session page via the

View Report icon).

Grade level tested Testing grade level. • Online Reporting Suite • Interim Data File • Student Attempt History pop-up

Student responses Student responses to each item (A, B, C, or D). Student scores for each item indicating Incorrect (0), Correct (1 ), or Not Answered (X).

• Online Reporting Suite • Interim Data File

Probability of achieving "Approaches," "Meets," or "Masters."

The probability, expressed as a percentage, of a student achieving "Approaches," "Meets," and "Masters." Estimate based on student performance on the Interim test.

• Online Reporting Suite • Interim Data File • Student Attempt History pop-up

Measuring the Power of Learning.'"

Reporting

Results Key

6

or Result Description Where Provided

N/A Score Not Available - indicates that the system has not yet received a completed test or was otherwise unable to provide a score.

• Interim Data File • Student Attempt History pop-up

* Did Not Complete - the student opened, but did not complete, Section 1 of the test.

• Interim Data File • Student Attempt History pop-up


Reporting

Results Key

7

he Assessment Management System records raw scores and calcu'lates the probabilrty of the student meeting grade level in summative ST M R testing_

Section 1* Section 2t Results§

5 ~ Hard Test------ ] Probabilities -c ::s ~E (I) (,/) 0 (/) (]) C)

ro~ -Section 1 T est------l- co .,... ---- Medium T est-------<C ----· Probabilities

~L C)

-~ 0 .§ s~ o 0 U O a: ~ Easy Test ______ (.{) Probabilities

* Students whose Section 1 tests are not scorable in real-time are automatically routed to ~he on grade "medium" Section 2

test. t students that never begin a Section 2 form only receive item ll'evel data in the dist rict data file.

§ Results are reported as raw scores for Sed:iiorn 1 & 2 tests {in campus and district data files on'ly), and as a percentage represent ing the probability of achie,nnig "approaches/ " meets," or " masters."


Reporting

How Results are Calculated

8


Accountability

• Optional assessments

• NOT tied to accountability

9


Online Reporting Suite (ORS)



• Dynamic real-time reporting

• Immediately after test submission, ORS will display the following results

• District

• Campus

• Student

• Results filterable by student demographics

• First release of results available beginning August 31, 2018

11



• Results available from the primary (first) completed attempt only

• Available immediately after test is completed

• Allow five minutes for processing

12


Expanded Online Test Administrator & Teacher Roles

• For the interim assessment, the online test administrator and teacher roles have been expanded.

• The intent of expanding these roles is to give districts an option to allow educators to view their students’ results in ORS.

• Educators who are assigned these roles by the district in the Assessment Management System will be able to view interim results for all students within their campus in the Assessment Management System and ORS.

• See the User Roles & Permissions Matrix for details on each user role.

13



• Educators will have access to demographic information for students registered in their campus via Students > View & Edit on the Profile tab.

• Educators will be able to access the attempts history pop-up under the Manage Online Testing tab in the Assessment Management System.

• Allows user to view probability scores for all attempts

14



• Educators will NOT be able to view any STAAR data or view any student information beyond the campus level.

• No access to summative standard reports under Reports > Results in the Assessment Management System

• No access to the Scores tab in the individual student profile where STAAR Report Cards are posted

15



• Pursuant to the Family Educational Rights and Privacy Act (FERPA), the disclosure of personally identifiable information from education records is permissible without written consent of the parent or eligible student if the disclosure is to other school officials, including teachers, within the local education agency (LEA) whom the LEA has determined to have legitimate educational interests. Please note FERPA compliance is required when assigning roles to access the online reporting system.

16

lt'5tin9 +

Dashboard: Welcome, Whitney Williams

News & Announcements

Click to ,xpand lar~r announcements..

Tr•iningAnnounnmmt2 Jul 17,2018 I ~~tulf.12All 0 ThlS Is J le ,ttralni "l em 2

0 Training Announ~tmfflf I Jul 16. 2013 I ~.8)';.I.8AININ~ED. Th s Is a test announcement for training purposes

SlrllC'ture +

Olrccto,y

OrganIza11ons

N- & AnnouncemenH Student Transfers

Online Tewng

Students Crot1 AnnCMA~11ment I

Oroos

Onl,n~ Reponing CE

Texas Data Portal r?f'

Showing 1 of 1


Navigation

• From the Assessment Management System, select Reports > Online Reporting from the top dropdown menu.

17

em I 1t~l lr-H lnten·m _ ..... ~ •-' ·.:..i....;o Ao:.,..:.~<= ... i~'"· ,,.,,,. .... ~r,•i:,r• ,5•.:.-.1 -, .. "'1 • .-. Reporting System _,_ · · · .,.,_ - · u ,,. - • - --,-,

Interim Reports

Find Interim Report

Fmd Organization Report

Ofganilabon •

• Select Organization•

Grade •

• Select Ora.de •

( Orpanlz.atlon Reports I Studat Report

• denotes requ red field!.

Sl.lbtect .

Select Subiect • . -


Navigation

• From the Online Reporting Suite, select the Organization Reports tab to view organizational reports (district or campus).

18

ED I 1tJI 1r,p lnten·m ... ,.,~~ t- ·:..J.- Ac.se5-::77i:,r• ,,,..,.,..,.:::n-&:.""'' 5•.-,7 -1-,1-1 • .-. Rc:porting System _,_ · · · · · · · ·' ,,. · •· · --,--

lnlenm Reports

Find Interim Report Organlz.atton Reports Student Report

Find Organization Report · denotes requ red fields

Organizabon • Grade • Subject .

Select Organizauon • • Sell!cl Grade • • S1!11!cl SubJl!Cl •

' -


Navigation

• From the Online Reporting Suite, select the Student Reports tab to view student level reports.

19

Reports

Find Interim Report Organization Reports Student RllpOrt

Fmd Organization Report • denotes requ red fields

Ofganinl!Qn • Grade · SYb-;ect •

• Select Organization• • Select Grade • • Select SubJecl.


Navigation

• From the Online Reporting Suite, select Back to STAAR Assessment Management System from the top dropdown menu.

20


Find Interim Report

• Opens at the user’s organizational level

• Users can then navigate to campus-level and student-level reports as permitted by their user role

• Users with multiple user roles can view all their organizations

• Use the Organization Reports and Student Report tabs to toggle between organization level and student level

21

Measuring the Power of learning.•

Find Interim Report

1. From the Assessment Management System, navigate to Reports > Online Reporting.

OR

2. Click the View Report icon on the Manage Session page.

The Find Interim Report appears, displaying the summary report at the user’s organizational level.

3. Select an Organization (if different from the organization displayed), Grade, and Subject.

4. Click the Find button.

5. Click a report link under Report Name to view.

22

IZ:J , , e I t I ' . . . ' . .-. Reporfing Sy.otem " er m

- Access via Reports > Online Reporting or click

lnlor,m R8!)0•1s

Fmd lntenm Report

Orgamzallon Reports

51.qoct •

Mathematics

Interim Tests Fourid: 2

, Reportt,...,.

2018-Hl.,t""m GS MalhemallCS, Opp1

....

Grade .

G@dc05

Org1n1utlon

Ill

Grade

05

organllatlon RCj)Ol15 Stut1,.,,1 R• 1 ,

the View Report icon from Manage Sessions.

Click the Student Report tab to search for an individual student report.

+-----------------+-Select a Grade, and Subject and then click the Find button to search for test results.

Subject I Students

Mal~•

D1te lbnge

ian 29 2C18-Mar23 2018

Mean Score

16AJ

2 Sl,o8:z'9 .,1""mG!i ~:reoro!IC5 • ___ P_ea_san_ 11i_•Jley..:...I_SO _____ o_~ ____ M•_l""'8alJ<.S _____ 2._•1_s ___ A_ug;;..3_1_20_ 1_S_-____ 163_ 6 ____ -+-CliCk a link under

TEA logll Pr,,acy & S.CIMlty I CTS r.-,k,

Cai,yrq, 07.llb Edu allo alT• tw, St • AJl1,gtl.,ft/l,J trM9mHi ,.. t. , pr pi1r.y of H ,.,ptctw• owrntr1

"Report Name" to view.


Find Interim Report

23


Find Interim Report

• Available at district, campus, and student levels

• Summary view and expanded list views at each level

• Includes clickable graphics and other elements that expand on information

• View individual student reports at any time by using the Student Report tab

24


District-Level Reports

• Campus results in a bar chart (lowest to highest mean score)

• District results in pie chart with aggregate results

• Segmented into campuses by the following criteria

• Top 25%

• Bottom 25%

• Middle 50%

• Click Students Tested button to view a list of all campuses with data.

• Organized lowest to highest mean score

• Number of students tested in grade and subject listed

25

Overview Report ..

--· -- ----1644 2,204

Oaincfs C;irrp,s Sccre::s 1

•-... " ....------ --- -----1- Click Students Tested to view a list of all campuses with data.

~----- - - ----- - --+ Hover over status bar to display

--

students tested and registered.

Click Find to search for other organization or student reports.

-...------t.-Aggregate results are segmented into top and bottom quarters and middle half. Clicking a segment displays a list of schools inoluded in lhe range and the mean soore for each at the bottom of the page.

<•- ~-========::.... ___________ -J- Campus results are shown from

-~- --· lowest to highest mean score. Scroll over the bars to identify oampuses. Click a bar to view the oampus report.

..-----------••_.,_. --------""--'------------ campuses are listed from --- "'" "" lowest to highest mean soore.

-- ,,,. • •

Number of students who completed testing is afso

, -------- shown. ~ ~--------.:::·":.:'' ______ .:.._.:..__-=:=:===~::::::::~- + c1;ck a campus link lo view the

campus report.

�~------------=----~--....... ~~~~~----------~~~} Click an Info icon to view ~ ,,,, additional informa lion about the

• Composite image from multiple views. Click the PDF icon to download a PDF file of the displayed report.

data displayed.


District-Level Reports

26


Campus-Level Reports

• Open at summary report displaying:

• Student results in bar chart (lowest to highest mean)

• Campus pie chart with aggregate results

• Segmented into groups of students by the following criteria

Top 25%

Bottom 25%

Middle 50%

• Click Students Tested button to view a list of all students (listed alphabetically) with test results including the following:

• Score

• Probabilities

• Reporting Category Scores

27

,pus Overview Report ...

C11npan t1.iowt1 IDotff 1

-

1672 17

0

....-. --------- ---------1- Click Students Tested fo view

the campus's student

....

Performance report.

Click Find to search for other organization or student reports.

~ ----+-Aggregate results are segment into top and bottom ,quarters a middle haft. Cricking a segment displays a list of students and mean score for ,each at the bottom of the page_

-----------------------+- student results are displayed fro lowest to highest mean sccre. Scroll over bars to view student

- scores. Click to vie-w a student repott. .--~ .... , ... 'I Student results include

, •· "'l!, , • ,... •-• aggregate score, probabilities,

:~----::--•-~~-~~=~~=~aL~;-1~-fr- __.;;;__ _ _:: __ ;::__~~J~and performance by category ' - - •• ,... •• - tor the selected segment. • lla.-...11-- 0. 1,J

Clicl< a student link to, view

" �---~-----------~;::::::.._::....,=-=::=:~ ________ J::~~'Xti individual student results. [-..-!'a~ Clicl< an tnfo icon to view additional information about the data displayed. TEA

.. Composite image from multipfe views. Cfick the PDF icon to download a PDF tile of the displayed report



28

s's Student Scores Report~

---..-.111.. •

.. '' 1672 17

• I._.Ml..,_

,a,, • ' .. • i ,, J

• '". , .. • l1t4

I •

'FEA

" Composite image from multiple views.

� ""'

-... -... ... . ... ... ,,_

"'" " 211

.... .., ~ .. ~ ... l•U 1dJ

Click Mean Score to return to the Campus Overview report_

Raw scores for each reporting category are included for all students.

Student results are available in a PDF or a CSV fife format.



29


Student-Level Reports

• Allow educators to perform item analysis based on individual student performance

• Display the following:

• Overall score as a number and plotted on a scale

• Performance in each reporting category

• Approaches, Meets, and Masters probabilities

• TEKS student expectation for each test question

• Student’s response to each test question

• A pop-up window that shows the test question

30

�

I

Strength . lhe response category is a strength for the student

Neither. The response category is not a strength o r needs improvement category for the student.

0 Needs Improvement The student needs improvement in the category.

Correct IReponse. The student answered the item correctly.

X Incorrect !Response. The student answered the item tnoorrectly.

No Response. Tlhe student did not answer the item.



• Click an Info icon in “Reporting Category Performance” section or a link in the “Student Expectation” column to review student expectation information and reporting categories for each question.

• Click the View icon to see the item, student response, and correct response.

31

Level Report*

.. II

.... _ 1650

........... --· ............... ~---~ ---

"" -A ------------·------------'-,

- -----------,--,-=-----+- Scores and strength

...... .-..i.. , .. _.. .,. ... .. ............. ....... .. ~ ~ ..... .. ............... _ ....... ,.. ... u.--........

...

,.,

........ i.,--... , .. ...

............ ..,. ....-11 .......... i...-. • ..,.... ..-~----· ....., .... __,, ....,,.._,. ...... ... ''""-

----· -------

)(

)(

)(

indications are shown for Reporting categories.

-

- -----L Click the Info icons lo review Student Expectations.

- - -1- Probability results are displayed in bar graphs.

student item responses are grouped by reporting category. Click the links under ·student Expectation· for more information about expectation for each reporting category.

st~ - ---1- Click the View icon to review q_ the assessment item, student

and correct responses. q_

__ _ ____ _____ :...._.:_=_=-~;;;=~~;:=;;-~;:=; =-=-=_=_=_=_=_::::s _ _ ___ _ ____ -:T Click the Info icon to view a .f - --•- report key.

--~ - as, ~--•~""- A PDF report with the data on

this page is available for download.

TEA

* Composite image from multiple views.



32


Probability Scores

Measuring the Power of Learning.•

Probability Scores

• Assessment Management System will display probability results

• Available immediately after testing

• Located in the Manage Online Sessions page in the View Attempts icon

• Show percentage probability of achieving the following categories on STAAR

• Approaches

• Meets

• Masters

• Includes results from all testing attempts

Copyright © 2018 by Educational Testing Service. All rights reserved. ETS, the ETS logo and MEASURING THE POWER OF LEARNING are registered trademarks of Educational Testing Service (ETS). 34728 34


Probability Scores

1. From the top navigation bar in the Assessment Management System, select Online Testing, then Manage Online Testing.

2. Select an interim administration (“INT”).

3. Select a Subject, Testing Grade, and District or Campus.

4. Click the SEARCH button.

5. In the Groups in Session list, click the View icon next to the appropriate testing group.

6. Results will appear on the Student Attempt History pop-up.

35

Student Attempt History

Student Name

CHILOERS, JOHN

Attempt o.te

SUbJect

Math

P'NP

10/121.2019 T

Close

Tesoog Gr.de

OS

Score

56

Probibilrty Probability of Achieving A of Achlu_lng Appro..:hes v Meets U

0

Probability ._ _____________ ..., Probability results of Achlev!Jlg Testing Clmpus appear under Masten O

27% AUSTIN H S(101911001j "Probability of Achieving Approaches," "-Meets," and "-Masters" columns.


0

Probability Scores

Student Attempt History

36


Interim Data File


Interim Data File

• A preliminary data file will be available in early January.

• A final data file will be available two weeks after the test window closes.

• Available in the Assessment Management System from Reports > Results

• Process for downloading similar to summative

• Choose interim test administration (“INT”)

38


Interim Data File

• Fixed-width data file

• Refer to Appendix B: Interim Data File Form for how to find and interpret information

• Includes the following:

• Student responses to test items

• Which Section 2 the student took (higher difficulty, medium difficulty, lower difficulty)

39


Interpreting Results


Interpreting Results How should districts interpret results?

• Interim Assessments Results Guide will be posted this fall.

• https://tea.texas.gov/student.assessment/IA

• Provides information on interpreting each type of reporting information including:

1. Student’s Scale Score

2. Student’s Probability of Achieving Each Performance Level

3. Student’s Reporting Category Results

4. Student’s Performance on Each Item

• Provides sample remediation strategies

41

https://tea.texas.gov/student.assessment/IA


Training


Interim Assessments Upcoming Webinar Trainings

Starting early, get ready for online testing (interim and summative)

• New secure browser needed

• 7/23 Bi-Weekly Email

• Online Testing > Secure Browsers

• Thursday, August 23, 2018

• 1:00 p.m.–2:00 p.m. CT

Registration links posted to https://www.texasassessment.com/administrators/training

43

https://www.texasassessment.com/administrators/training

Measuring the Power of Learning.-

Questions

4 4

r - -. the Power of Learning.'" Measunng

Questions ?

45


Customer Support

Texas Assessment Support Center

• Monday–Friday

• 8:00 a.m.–5:00 p.m. CT

• 855-333-7770

• [email protected] or [email protected]

46



A Deep Dive Into the Online Reporting Suite...

Documents

Transcript of A Deep Dive Into the Online Reporting Suite...