1st Annual Conference of the Florida Association of College Test Administrators
description
Transcript of 1st Annual Conference of the Florida Association of College Test Administrators
1st Annual Conference
of the
Florida Association of College Test Administrators
February 3-4, 2010
Lakeland Florida
Dr. Louis Woodruff
The oldest continually inhabited city in the
USA is St Augustine, founded in 1565.
The world’s first scheduled passenger
airline flight was on January 1, 1914
from St. Petersburg's to Tampa.
Ybor City was once known as
“The Cigar Capital of the World”
producing 700,000,000 cigars annually
Fort Lauderdale is known as the
Venice of America because it has
185 miles of local waterways.
Venice Florida is known as the
Shark Tooth Capital of the World.
The Moonstone is the official
Florida State Gem.
Moonstones are not
found in Florida . . .
They are also not
found on the moon !
Clearwater has more lightening strikes
per-capita than any city in the U.S.
The Alligator is the official
Florida State Reptile.
Gatorade was named for the
University of Florida Gators
where the drink was first developed.
The first mechanical refrigeration was
invented in Apalachicola in 1851.
FLORIDA LAWS
THE TOP 10 ACTS
FORBIDDEN BY LAW
IN THE STATE OF FLORIDA
THE TOP 10 ACTS
FORBIDDEN BY LAW
IN THE STATE OF FLORIDA
Showering naked
Singing in a public place
wearing a swimsuit
while in Sarasota
Harming a chicken in any way
while in Key West
Imitating an animal
while in Miami
Leaving a ship
while docked in Tampa Bay
if you are a rat
Parachuting on Sunday
if you are an
unmarried woman
Topless running or walking
within a 150 foot zone
between the beach and the street
Tying an elephant
to a parking meter
without paying the parking fee
Fishing from one’s car
while driving across a bridge
Having sexual relations
with a porcupine !
Shared Vision – Shared Purpose Shared Vision – Shared Purpose
Connecting with Other State Organizations
Strategies for Making Money
Current and Future Trends in Testing
NCTA RESOURCE LINKS
NCTA WEB SITE
Connecting with Other State Organizations
Strategies for Making Money
Take credit for your productivity increases
(doing more for the same money or with less).
Saving money is the same as making money.
Strategies For Making MoneyStrategies For Making Money
Team with your ConEd or Workforce
Development programs to provide
employment related assessment services.
Look for testing services you
provide for which you could
introduce or increase a fee.
Use the NCTA Mailing List to
locate opportunities for vendor
testing, with FACTA serving
as the initial point of contact.
Look for testing services you
provide for which you could
introduce or increase a fee.
Use the NCTA Mailing List to
locate opportunities for vender
testing, with FACTA serving
as the initial point of contact.
Take credit for your productivity increases
(doing more for the same money or with less).
Saving money is the same as making money.
Team with your ConEd or Workforce
Development programs to provide
employment related assessment services.
NCTA Grant ProgramNCTA Grant Program
Current and Future Trends in Testing
A very early lesson for
the future testing industry . . .
learned from the #2 pencil ! !
Slates and chalk continued to beused after the pencil was inventedin the early 19th Century due to the high costs for the manualproduction of pencils.
In 1866 a patent was grantedfor a wood-plane machineenabling mass production . . .
$ $ $
. . . the “new technology” (the pencil) was soon embraced in the marketplace.
Within 4 years over20 million pencilswere sold.
This enabled the unitcost of the pencil tobe lowered . . .
By 1903 the eraserwas addedand the cost dropped toa penny.
$ $ $
$ $ $
The testing industry has repeatedly confronted this same lesson . . .
$ $ $
$ $ $
Having the Technology is not enough.
Through mass
production and
robust distribution
channels, Manufacturing
has to provide the Market
the “right” unit cost !
Having what the
market needs is
not enough.
$ $ $
Lesson from the past – still relevant:
A BUSINESS MANDATE
FundingSources
ProductionCapability
MarketNeed
EnablingTechnology
DistributionChannels
ViableProductLow Unit
Price
ABusiness
Mandate
Universities and the CollegeBoard were operating their own commercial testing business
The Stanford Achievement Tests to assess subject knowledge
The Scholastic Aptitude Test operated by the College Board
The California Achievement Test used at the secondary level
The Iowa Test of Basic Skillsoperated at the University of Iowa
FundingSources
ProductionCapability
MarketNeed
EnablingTechnology
DistributionChannels
ViableProduct
Low UnitPrice
Increasing Demandfor Assessment
Private and Public Funding
Valid and Reliable Tests Manual Scoring and Reporting
In 1938 IBM creates the first electronicgraphite/optical test scoring machine.
Technology
Needed an Enabling Technology
$ $
$ $
FundingSources
ProductionCapability
MarketNeed
EnablingTechnology
DistributionChannels
ViableProduct
Low UnitPrice
Automated Scanning Scoring and Reporting
Measures performance relative to a
representative national sample of
students called a “norming sample”
Test Item DatabaseTest Item Database
Norm-Referenced Test
Test ItemStatistics
Test FormEquating
Test ItemRotation
Test Design
Validity &Reliability
% of Reusable Test Items
Reconstructed Test Forms
Large Test Volume
FundingSources
ProductionCapability
MarketNeed
EnablingTechnology
DistributionChannels
ViableProduct
Low UnitPrice
DistributionChannels
Labor forTest Delivery
InternationalPartners
Staff, Services and Facilities for Very Low Cost
FundingSources
ProductionCapability
MarketNeed
EnablingTechnology
DistributionChannels
ViableProduct
Low UnitPrice
National Norm Referenced
Paper & Pencil Testing
How did the industry get into this situation?
Is there a way out of it?
Service Issues
Delayed Reporting
Shipping Problems
Processing Delays
Delivery Problems
Scoring Errors
CBT significantly improvedthe test product . . . but it seriouslylimited production capabilities, required a new distribution infrastructure and raised operational and development costs
The delivery advantage of using schools(inexpensive classrooms for large group testing)
lost its business value for CBT.
CBT altered the business dynamics in the testing industry by reducing profit margins, expandingcompetition, and raising client expectationsbeyond what the vendors could profitably deliver.
Store-front commercial outlets: the new distribution channel.
-$
FundingSources
ProductionCapability
MarketNeed
EnablingTechnology
DistributionChannels
ViableProductLow Unit
Price
CBT disrupted the
dynamics of the P&P business model
CBT
http://www
Even internet delivery did not resolve things
Internet Internet Delivery Delivery
Additional competitors emerged
Cost savings were minimal
Test development and overall operating costs remained higher than traditional P&P
THEN CAME
NO CHILD SHALL BE LEFT BEHIND
WHICH MEANT
NO CHILD SHALL MOVE FORWARD
WITHOUT STANDARDIZED TESTING
At the outset, NCLB was seen by the
testing industry as a potential P&P
cash cow that could support the
marginal CBT side of the business
Low cost P&P delivery Less costly test development High test volumes Economies of scale
What turned the NCLB golden goose
into a turkey for the testing industry?
Individual State specs
Lower per test volumes
State owned test items
Short reporting windows
Delivery by the client
Low economies of scale
Increased competition
The Testing
Industry needs
a new Enabling
Technology
Measure performance relative to a
representative national sample of
students called a “norming sample”
Norm-Referenced Tests
Criterion-Referenced Tests
Measure performance relative
to mastery of content at certain
pre-defined levels of proficiency
NCLB Brought:
Mass Production Capability
Distribution Channels
Low Unit Cost
Complete the
“Business Mandate”
STILL NEEDS:
Future
Hand Held
Mini-PC Designs
Test Delivery Unit #4
Laptop Server + IDD’s
Test Delivery Unit #3
Laptop Server + IDD’s
Test Delivery Unit #2
Laptop Server + IDD’s
Test Delivery Unit #1
Laptop Server + MPC’s
Laptop Server Hub
Host Server
Laptop #1
Laptop #2
Laptop #3
Laptop #4
Test Server #1 (laptop)
Testing Room #4
Test Server #4 (laptop)
Testing Room #3
Test Server #1 (laptop)
Testing Room #2
Test Server #1 (laptop)
Testing Room #1
Test Server #1 (laptop)
A Future Model Data Source (Vendor)
Host and Laptop Serversstored in “wired” cabinet
Laptop Servers & hand heldMini-PCs transported inmobile Test Delivery Units
After testing in classroomsLaptop Servers returned to Host for test data upload.
Test data download throughSecure Internet Technology
No Costly PC Desktops NeededNo Costly PC Desktops Needed
No Internet Needed for DeliveryNo Internet Needed for Delivery
No Costly Test Servers NeededNo Costly Test Servers Needed
Large Capacity – Low Cost – High QualityLarge Capacity – Low Cost – High Quality
Test Delivery Business Paradigm ShiftTest Delivery Business Paradigm Shift
The Old Model in a New DimensionThe Old Model in a New Dimension
Returning the Test DeliveryReturning the Test Delivery
Business AdvantageBusiness Advantage
to the Schoolsto the Schools
FundingSources
MassProduction
MarketNeed
EnablingTechnology
DistributionChannels
ViableProduct
Low UnitPrice
Meeting The Business
Mandate
What other future technologies
may impact our testing profession?
What other future technologies
may impact our testing profession?
Confirming Examinee Identity Confirming Examinee Identity
Continuously During Testing Continuously During Testing
Confirming Examinee Identity Confirming Examinee Identity
Continuously During Testing Continuously During Testing
ComputerizedComputerized PaperPaper
Immediate Immediate
Answer SheetAnswer Sheet
andand
Essay ScoringEssay Scoring
ComputerizedComputerized
“ “Test Coat” Test Coat”
for electronic for electronic
test monitoringtest monitoring
Projection of a laser generated3-D image suspended in space
A future Test Center
with individualized virtual
testing environments and
projected Holographic images
as interactive test stimuli
In the future we may be able to
reach total standardization through
the management of each examinee’s
perceptions during the testing event.
In the future we may be able to
reach total standardization through
the management of each examinee’s
perceptions during the testing event.
Will future advances in brain mapping
and Psychopharmacology enable
selected modifications of the brain’s
perceptions to achieve absolute virtual
standardization - from the inside out?
Will future advances in brain mapping
and Psychopharmacology enable
selected modifications of the brain’s
perceptions to achieve absolute virtual
standardization - from the inside out?
Virtual Testing EnvironmentVirtual Testing EnvironmentVirtual Testing EnvironmentVirtual Testing Environment
Transmitting an Ideal, Standardized,
Virtual, Personal Testing Space,
Integrated within the Test
StandardizationStandardization
as an as an
Altered State ofAltered State of
Consciousness !Consciousness !
“It’s always better with a friend !”
“It’s always better with a friend !”
Dr. Louis Woodruff
NCTA President
Director of Testing
Bucks County Community College
Newtown, PA
215-968-8467