Printed by Jouve, 75001 PARIS (FR)

(19)E

P2

790

157

A1

TEPZZ 79Z_57A_T(11) EP 2 790 157 A1

(12) EUROPEAN PATENT APPLICATION

(43) Date of publication: 15.10.2014 Bulletin 2014/42

(21) Application number: 14171855.1

(22) Date of filing: 12.06.2006

(51) Int Cl.:G07B 15/06 (2011.01) G08G 1/017 (2006.01)

G08G 1/00 (2006.01)

(84) Designated Contracting States: AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

(30) Priority: 10.06.2005 US 689050 P

(62) Document number(s) of the earlier application(s) in accordance with Art. 76 EPC: 12161598.3 / 2 472 47606808926.7 / 1 897 065

(71) Applicant: Accenture Global Services LimitedDublin 4 (IE)

(72) Inventors: • Hedley, Jay E.

Arlington, VA 22209 (US)• Thornburg, Neal Patrick

Charlotte, NC 28202 (US)

(74) Representative: Noble, Nicholas et alKilburn & Strode LLP 20 Red Lion StreetLondon WC1R 4PJ (GB)

Remarks: This application was filed on 10-06-2014 as a divisional application to the application mentioned under INID code 62.

(54) Electronic vehicle identification

(57) Identifying a vehicle in a toll system includes ac-cessing image data for a first vehicle and obtaining li-cense plate data from the accessed image data for thefirst vehicle. A set of records is accessed. Each recordincludes license plate data for a vehicle. The license platedata for the first vehicle is compared with the license platedata for vehicles in the set of records. Based on the re-sults of the comparison of the license plate data, a set ofvehicles is identified from the vehicles having records inthe set of records. Vehicle fingerprint data is accessedfor the first vehicle. The vehicle fingerprint data for thefirst vehicle is based on the image data for the first vehicle.Vehicle fingerprint data for a vehicle in the set of vehiclesis accessed. Using a processing device, the vehicle fin-gerprint data for the first vehicle is compared with thevehicle fingerprint data for the vehicle in the set of vehi-cles. The vehicle in the set of vehicles is identified as thefirst vehicle based on results of the comparison of vehiclefingerprint data.

EP 2 790 157 A1

2

5

10

15

20

25

30

35

40

45

50

55

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to United StatesProvisional Patent Application Number 60/689,050, filedon June 10, 2005, and titled ELECTRONIC TOLL MAN-AGEMENT, hereby incorporated by reference in its en-tirety for all purposes.

TECHNICAL FIELD

[0002] This disclosure relates to electronic vehicleidentification.

BACKGROUND

[0003] Transportation facilities such as roads, bridges,and tunnels produce tolls often representing a majorsource of income for many states and municipalities. Thelarge number of automobiles, trucks, and buses stoppingat tollbooths to pay a toll daily can cause significant prob-lems. For example, such facilities may restrict the flowof traffic causing traffic backups and lane changing, oftenincreasing the likelihood of accidents and even more bot-tlenecks. In addition, many people may be delayed fromreaching their destinations, and goods may be delayedfrom getting to market and millions of gallons of fuel maybe wasted as vehicles idle. Environments may experi-ence an increase in pollution as idling and slow movingvehicles emit pollutants (particularly carbon dioxide andcarbon monoxide), which may pose a significant healthhazard to motorists as well as to tollbooth operators.[0004] Some tollbooth systems may have a programrequiring that a motorist rent and then attach to the wind-shield of the vehicle a radio transponder that communi-cates via radio frequency with receiver units at tollboothplazas. However, such programs require drivers to seekout the program and to register for the program. Theseprograms may make it mandatory for a motorist to makea credit card deposit and create an automatic debit ac-count arrangement, which may effectively eliminate driv-ers with credit problems. These programs also may billparticipants based on a minimum amount of travel re-gardless of the actual amount of travel. Thus, many mo-torists who travel infrequently travel through the toll roadmay receive little benefit after investing time and moneyto participate in the program.[0005] One problem with existing technology for iden-tifying vehicles is therefore that a transponder unit is re-quired in each vehicle that is to be identified.

SUMMARY

[0006] The present disclosure describes at least onemethod of identifying a vehicle that enables automaticand electronic handling of payment of tolls by vehiclespassing a toll facility, without requiring the vehicles to

slow down or to have a transponder. The method mayconstitute at least part of a toll system. Such a systemautomatically identifies all or substantially all of the vehi-cles that pass the toll facility, and bills the owner of eachidentified vehicle for the incurred toll fee.[0007] An existing technology for identifying vehicleswithout a transporter is license plate reading (LPR).[0008] A problem with existing LPR technology foridentifying vehicles in a toll system however is that, dueto the high number if vehicles passing through a typicaltoll facility, such technology typically has too high an errorrate for effective use.[0009] For example, the error rate for a typical LPRsystem may be approximately 1%. While such an errorrate may be acceptable for toll systems that only identifyvehicles that are violators, this error rate is typically toohigh for a toll system that attempts to identify every pass-ing vehicle, not just the violators, for collection of toll fees.In such a system, a 1% error rate can result in a significantloss of revenue (e.g., the loss of 1000 or more toll feesa day).[0010] Additionally, typical LPR systems often exhibita tradeoff between the number of vehicles identified (i.e.,those vehicles for which the read result exceeds a readconfidence threshold for presumption of correct ID) andthe error rate. In an ideal world, this tradeoff would bereflected in a binary confidence continuum, where thesystem always produces a read confidence level of onewhen the read result is correct and a read confidencelevel of zero when the read result is incorrect. In reality,however, the read results are usually at least partiallycorrect, and the system generates a confidence contin-uum having a broad range of confidence levels ranging,for example, from a level of one or near one (very likelycorrect) to a level of zero or near zero (very likely incor-rect). The system, therefore, is often required to set anarbitrary read confidence threshold for determining whichread results will be deemed correct. Once the read con-fidence threshold is set, any read results having confi-dence levels above the threshold are deemed correctand any read results having confidence levels below thethreshold are deemed incorrect. Setting the read confi-dence threshold too high (e.g., at .95 or higher) signifi-cantly decreases the possibility of an error but also ex-cludes many correct read results, thereby reducing rev-enue. Conversely, setting the read confidence thresholdtoo low (e.g., .3 or higher)increases the number of readsdeemed correct but also significantly increases thenumber of errors, thereby increasing costs by introducingerrors into a large number of accounts/bills which requiremuch time and effort to audit and correct. In a toll systemthat identifies every passing vehicle, this tradeoff is par-ticularly problematic since it may result in a significantloss of profits.[0011] Moreover, a toll system that identifies everypassing vehicle is identifying a much larger number ofvehicles than a conventional toll system, which typicallyonly identifies violators. Accordingly, such a toll system

1 2

EP 2 790 157 A1

3

5

10

15

20

25

30

35

40

45

50

55

attempts to identify every passing vehicle and is designedto both maximize revenue by identifying vehicles veryaccurately and limit personnel costs by minimizing theneed for manual identification of vehicles and account/billerror processing.[0012] In one particular implementation, to obtain alower vehicle identification error rate (and obtain a higherautomated identification rate), the toll system uses twovehicle identifiers to identify a target vehicle. Specifically,the toll system collects image and/or sensor data for thetarget vehicle and extracts two vehicle identifiers fromthe collected data. The vehicle identifiers extracted fromthe collected data may include, for example, license plateinformation, a vehicle fingerprint, a laser signature, andan inductive signature for the target vehicle. In one par-ticular implementation, the first vehicle identifier is licenseplate information and the second vehicle identifier is avehicle fingerprint.[0013] The toll system uses the first vehicle identifierto determine a set of one or more matching vehicle can-didates by searching a vehicle record database and in-cluding in the set only those vehicles associated withrecords having data that match or nearly match the firstvehicle identifier of the target vehicle. The toll systemuses the second vehicle identifier of the target vehicle toidentify the target vehicle from among the set of matchingvehicle candidates.[0014] When the first vehicle identifier is license plateinformation and the second vehicle identifier is a vehiclefingerprint, the toll system may eliminate the problematictrade-off between the number of vehicles identified andthe error rate typical of LPR systems by using the LPRidentification for identification of the group of vehicle can-didates, rather than for the final identification of the ve-hicle, and then using the much more accurate vehiclefingerprint matching for the final identification of the ve-hicle. Thus, incorrect reads by the LPR system are elim-inated during the final and more accurate fingerprintmatching identification. This toll system may thereby beable to obtain extremely accurate identification resultsfor a larger proportion of vehicles than would be obtainedthrough license plate reading alone.[0015] In particular, the toll system accesses therecords of the matching vehicle candidates and searchesfor one or more records that have data sufficiently similarto the second vehicle identifier of the target vehicle soas to indicate a possible match. If no possible matchesare found for the target vehicle among the set of matchingvehicle candidates, the toll system may increase the sizeof the set by changing the matching criteria and may onceagain attempt to identify one or more possible matchesfor the target vehicle from among the larger set of match-ing vehicle candidates. If still no possible matches arefound, the toll system may enable a user to manuallyidentify the target vehicle by providing the user with ac-cess to the collected data for the target vehicle and ac-cess to databases internal and/or external to the toll sys-tem.

[0016] If one or more possible matches are found, aconfidence level is determined for each possible match.If the confidence level of a possible match surpasses anautomated confidence threshold, the toll system auto-matically identifies the target vehicle without human in-tervention as the vehicle corresponding to the possiblematch. If the confidence level of a possible match sur-passes a probable match threshold, the toll systempresents the probable match to a human operator andenables the human operator to confirm or reject the prob-able match. If no automatic match or confirmed probablematch is found, the toll system enables a user to manuallyidentify the target vehicle by providing the user with ac-cess to the collected data for the target vehicle and thepossible matches identified by the toll system, and withaccess to databases internal and/or external to the tollsystem.[0017] In this manner, the toll system typically obtainsgreater vehicle identification accuracy by requiring thattwo vehicle identifiers be successfully matched for suc-cessful vehicle identification. Moreover, the identificationprocess may be faster because the matching of the sec-ond identifier is limited to only those vehicle candidateshaving records that successfully match the first vehicleidentifier. Human operator intervention is also kept to aminimum through use of multiple confidence level thresh-olds.[0018] In one general aspect, identifying a vehicle in atoll system includes accessing image data for a first ve-hicle and obtaining license plate data from the accessedimage data for the first vehicle. A set of records is ac-cessed. Each record includes license plate data for avehicle. The license plate data for the first vehicle is com-pared with the license plate data for vehicles in the setof records. Based on the results of the comparison of thelicense plate data, a set of vehicles is identified from thevehicles having records in the set of records. Vehiclefingerprint data is accessed for the first vehicle. The ve-hicle fingerprint data for the first vehicle is based on theimage data for the first vehicle. Vehicle fingerprint datafor a vehicle in the set of vehicles is accessed. Using aprocessing device, the vehicle fingerprint data for the firstvehicle is compared with the vehicle fingerprint data forthe vehicle in the set of vehicles. The vehicle in the setof vehicles is identified as the first vehicle based on re-sults of the comparison of vehicle fingerprint data.[0019] Implementations may include one or more ofthe following features. For example, comparing licenseplate data for the first vehicle with license plate data forvehicles in the set of records may include searching avehicle record database for records that include licenseplate data that exactly match the license plate data ob-tained for the first vehicle. Comparing license plate datafor the first vehicle may further include performing anextended search of the vehicle record database forrecords that include license plate data that nearly matchthe license plate data obtained for the first vehicle. Theextended search may be conditioned on no vehicle iden-

3 4

EP 2 790 157 A1

4

5

10

15

20

25

30

35

40

45

50

55

tification records being found that include license platedata that exactly match the license plate data obtainedfor the first vehicle.[0020] Comparing the license plate data for the firstvehicle with the license plate data for vehicles in the setof records may include comparing the license plate datausing predetermined matching criteria. The predeter-mined matching criteria may be changed to increase thenumber of vehicles in the identified set of vehicles.Changing the predetermined matching criteria to in-crease the number of vehicles in the identified set of ve-hicles may be conditioned on a failure to identify any ve-hicles in the set of vehicles as the first vehicle based onresults of the comparison of vehicle fingerprint data.[0021] Identifying a vehicle in a toll system may furtherinclude capturing laser signature data or inductive sig-nature data for the first vehicle. The laser signature datamay include data obtained by using a laser to scan thefirst vehicle. The laser signature data may include oneor more of an overhead electronic profile of the first ve-hicle, an axle count of the first vehicle, and a 3D imageof the first vehicle.[0022] The inductive signature data may include dataobtained through use of a loop array over which the firstvehicle passes. The inductive signature data may includeone or more of an axle count of the first vehicle, a typeof engine of the first vehicle, and a vehicle type or classfor the first vehicle.[0023] Each record in the set of records includes lasersignature data or inductive signature data for a vehicle.Identifying a vehicle in a toll system may further includecomparing laser signature data or inductive signature da-ta for the first vehicle with laser signature data or inductivesignature data for vehicles in the set of records. Identi-fying a set of vehicles from the vehicles having recordsin the set of records may include identifying the set ofvehicles based on the results of the comparison of thelicense plate data and the results of the comparison ofthe laser signature data or the inductive signature data.[0024] Identifying the set of vehicles based on the re-sults of the comparison of license plate data and the re-sults of the comparison of the laser signature data orinductive signature data may include determining a com-bined equivalent matching score for each vehicle havinga record in the set of records and identifying the set ofvehicles as a set of vehicles having combined equivalentmatching scores above a predetermined threshold. Eachcombined equivalent matching score may include aweighted combination of a laser or inductive signaturematching score and a license plate matching score.[0025] Identifying the vehicle in the set of vehicles asthe first vehicle may include identifying the vehicle as thefirst vehicle based on the results of the comparison ofthe vehicle fingerprint data and the results of the com-parison of the laser signature data or inductive signaturedata. Identifying the vehicle in the set of vehicles as thefirst vehicle based on the results of the comparison ofthe vehicle fingerprint data and the results of the com-

parison of the laser signature data or inductive signaturedata may include determining a combined equivalentmatching score for the vehicle in the set of vehicles anddetermining that the combined equivalent matchingscore is above a predetermined threshold. The combinedequivalent matching score may include a weighted com-bination of a laser or inductive signature matching scoreand a vehicle fingerprint matching score.[0026] Identifying the vehicle in the set of vehicles asthe first vehicle may include identifying the vehicle as thefirst vehicle if the comparison of the vehicle fingerprintdata for the first vehicle with the vehicle fingerprint datafor the vehicle in the set of vehicles indicates a matchhaving a confidence level that exceeds a confidencethreshold. Identifying the vehicle in the set of vehicles asthe first vehicle may include identifying the vehicle as thefirst vehicle without human intervention if the confidencelevel of the match exceeds a first confidence thresholdand/or may include identifying the vehicle as the first ve-hicle if the confidence level of the match is less than thefirst confidence level but greater than a second confi-dence threshold and a human operator confirms thematch. The human operator may confirm or reject thematch by enabling the operator to perceive the imagedata for the first vehicle and enabling the human operatorto interact with a user interface to indicate rejection orconfirmation of the match. Identifying the vehicle in theset of vehicles as the first vehicle may include identifyingthe vehicle as the first vehicle if the confidence level ofthe match is less than the first and second confidencethresholds and a human operator manually identifies thevehicle as the first vehicle by accessing the image datafor the first vehicle and the record for the vehicle in theset of records. The human operator may manually iden-tify the vehicle in the set of vehicles as the first vehicleby enabling the human operator to access the image datafor the first vehicle, enabling the human operator to ac-cess the record for the vehicle in the set of records, andenabling the human operator to interact with a user in-terface to indicate positive identification of the first vehicleas the vehicle in the set of vehicles. The human operatormay be enabled to manually identify the vehicle in theset of vehicles as the first vehicle by enabling the humanoperator to access data stored in databases of externalsystems.[0027] Identifying the vehicle in the set of vehicles asthe first vehicle may include identifying the vehicle bycombining vehicle identification number (VIN), laser sig-nature, inductive signature, and image data.[0028] In another general aspect, an apparatus foridentifying a vehicle in a toll system includes an imagecapture device configured to capture image data for afirst vehicle. The apparatus further includes one or moreprocessing devices communicatively coupled to eachother and to the image capture device. The one or moreprocessing devices are configured to obtain license platedata from the captured image data for the first vehicleand access a set of records. Each record in the set of

5 6

EP 2 790 157 A1

5

5

10

15

20

25

30

35

40

45

50

55

records includes license plate data for a vehicle. The oneor more processing devices are further configured tocompare the license plate data for the first vehicle withthe license plate data for vehicles in the set of recordsand identify a set of vehicles from the vehicles havingrecords in the set of records. The set of vehicles is iden-tified based on results of the comparison of the licenseplate data. The one or more processing devices are fur-ther configured to access vehicle fingerprint data for thefirst vehicle. The vehicle fingerprint data for the first ve-hicle is based on the captured image data for the firstvehicle. The one or more processing devices are alsoconfigured to access vehicle fingerprint data for a vehiclein the set of vehicles, compare the vehicle fingerprint datafor the first vehicle with the vehicle fingerprint data forthe vehicle in the set of vehicles, and identify the vehiclein the set of vehicles as the first vehicle based on resultsof the comparison of vehicle fingerprint data.[0029] The above and other implementations and fea-tures are described in detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

[0030]

FIG. 1 is a block diagram of an implementation of anelectronic toll management system.FIG. 2 is a flow chart of an implementation of anelectronic toll management system related to high-lighted vehicle identifier management.FIG. 3 is a flow chart of an implementation of anelectronic toll management system related to pay-ment management.FIG. 4 is a flow chart of an implementation of anelectronic toll management system related to pay-ment management.FIG. 5 is a flow chart of an implementation of anelectronic toll management system related to mailingaddress verification.FIG. 6 is a block diagram of an implementation of anelectronic toll management system.FIG. 7 is a flow chart of an implementation of anelectronic toll management system related to vehicleidentification.FIG 8. is a flow chart of an implementation of anelectronic toll management system related to vehicleidentification.FIGs 9A-9C are a flow chart of an implementation ofan electronic toll management system related to ve-hicle identification.

[0031] Like reference symbols in the various drawingsindicate like elements.

DETAILED DESCRIPTION

[0032] FIG. 1 is a block diagram of an implementationof an electronic toll management system 10. The system

10 is configured to capture a vehicle identifier 31 of ve-hicle 30 interacting with a facility 28 and to notify externalsystems 34 of such interaction. For example, the system10 may allow a toll road authority to capture a vehicleidentifier 31, such as license plate information, from avehicle 30 traveling through the toll road and then to notifylaw enforcement whether the captured vehicle identifiermatches a license plate previously highlighted by law en-forcement.[0033] The toll management system 10 also can man-age payment from a party associated with the vehicle 32based on the interaction between the vehicle 30 and thefacility 28. For example, the system 10 can capture li-cense plate information from a vehicle 30 and identifythe registered owner of the vehicle. The system wouldthen provide to the owner, over a communications chan-nel such as the Internet, an account for making paymentor disputing payment. The toll management system 10can send a bill requesting payment from the party 32using a mailing address that has been verified againstone or more mailing address sources. The system 10 iscapable of automatically capturing an image of the vehi-cle 30 triggered by the vehicle interacting with the facility.Such image capturing can be accomplished using image-processing technology without having to install a radiotransponder (e.g., RFID device) in a vehicle.[0034] The electronic toll management system 10 in-cludes a toll management computer 12 which can beconfigured in a distributed or a centralized manner. Al-though one computer 12 is shown, one or more comput-ers can be configured to implement the disclosed tech-niques. The computer 12 is coupled to a facility 28 thatmay charge a fee for interacting with the facility. Exam-ples of a facility 28 include a toll facility (managed by tollauthorities) such as toll road, a toll bridge, a tunnel, park-ing facility, or other facility. The fee may be based on theinteraction between the vehicle 30 and the facility 28.Examples of interactions that may involve a fee includea distance traveled by the vehicle through the facility, atime period the vehicle is present in a facility, the type ofvehicle interacting with the facility, the speed at whichthe vehicle passes through the facility, and the type ofinteraction between the vehicle and the facility.[0035] The facility 28 can process vehicles includingautomobiles, a truck, buses, or other vehicles. For easeof explanation, the system 10 shows a single facility 28interacting with a single vehicle 30 and a party associatedwith the vehicle 32. However, in other implementations,the disclosed techniques could be configured to operatewith one or more vehicles interacting with one or morefacilities spanning different geographic locations.[0036] The toll management computer 12 includes animage acquisition module 24 configured to detect thepresence of a vehicle, acquire one or more images of thevehicle, and forward the image(s) to an image-process-ing module 25 for further processing. The module 24 mayinclude image acquisition equipment based on the phys-ical environment in which it is used. For example, for

7 8

EP 2 790 157 A1

6

5

10

15

20

25

30

35

40

45

50

55

open-road applications, image acquisition equipmentmay be mounted above the roadway, on existing struc-tures or on purpose-built gantries. Some open-road ap-plications may use equipment mounted in or beside theroadway as well. Lane-based (or tollbooth-style) appli-cations may use equipment mounted on physical struc-tures beside each lane, instead of or in addition to equip-ment mounted overhead or in the roadway.[0037] The image acquisition module 24 may includeimaging components such as vehicle sensors, cameras,digitizing systems, or other components. Vehicle sensorscan detect the presence of a vehicle and provide a signalthat triggers a camera to capture one or more images ofthe vehicle. Vehicle sensors may include one or more ofthe following:

(1) Laser/sonic/microwave devices - these devices,commonly used in Intelligent Transportation Sys-tems (ITS) applications, can recognize the presenceof a vehicle and provide information regarding thevehicle’s size, classification, and/or speed. Thesesensors may be configured to provide additional in-formation about the vehicle which can be used inidentify the vehicle and its use of the toll facility, in-cluding trip time and compliance with traffic laws.(2) Loops - these sensors can detect the presenceand the vehicle type by recognizing the presence ofmetal masses using a wire loop embedded in theroad. Loops can be used as a backup to more so-phisticated sensors. Loops can also be used as aprimary source of data to detect vehicles, classifyvehicles, trigger cameras, and provide vehicle sig-nature data (e.g., based on use of an array of loopswith a smart loop control program such as DiamondConsulting’s IDRIS® system of Buckinghamshire,United Kingdom).(3) Through-beam sensors - these sensors may emita continuous beam across the roadway, and detectthe presence of a vehicle based upon interruptionsin the beam. This type of sensor may be used ininstallations where traffic is channeled into tollbooth-style lanes.(4) Optical sensors - vehicle may be recognized us-ing cameras to continuously monitor images of theroadway for changes indicating the presence of avehicle. These cameras also can be used to recordimages for vehicle identification.

[0038] Cameras can be used to capture images of ve-hicles and their identifying characteristics. For example,they can be used to generate a vehicle identifier such asa vehicle license number based on an image of a licenseplate. Cameras may be analog or digital, and may captureone or more images of each vehicle.[0039] Digitizing systems convert images into digitalform. If analog cameras are used, the cameras can beconnected to separate digitizing hardware. This hard-ware may include a dedicated processing device for an-

alog-to-digital conversion or may be based on an inputdevice installed in a general-purpose computer, whichmay perform additional functions such as image process-ing. Lighting can be employed to provide adequate andconsistent conditions for image acquisition. The lightingmay include strobes or continuous illumination, and mayemit light of light in the visible spectrum or in the infraredspectrum. If strobes are used, they may be triggered byinputs from the vehicle sensor(s). Other sensors such aslight sensors may be required to control the image ac-quisition module 24 and provide consistent results.[0040] Once the image acquisition module 24 has cap-tured images of the vehicles, the images may be forward-ed to an image-processing module 25. The image-processing module 25 may be located in the same loca-tion as the image acquisition module 24 and the imagecomputer 12, in a remote location, or a combination ofthese locations. The module 25 can process a single im-age for each vehicle or multiple images of each vehicle,depending on the functionality of the image acquisitionmodule 24 and/or business requirements (e.g., accuracy,jurisdictional requirements). If multiple images are used,each image may be processed, and the results may becompared or combined to enhance the accuracy of theprocess. For example, more than one image of a rearlicense plate, or images of both front and rear licenseplates, may be processed and the results compared todetermine the most likely registration number and/or con-fidence level. Image processing may include identifyingthe distinguishing features of a vehicle (e.g., the licenseplate of a vehicle) within the image, and analyzing thosefeatures. Analysis may include optical character recog-nition (OCR), template matching, or other analysis tech-niques.[0041] The toll management system 10 may includeother systems capable of substantially real-time process-ing located at the site where images are acquired to re-duce data communication requirements. In an implemen-tation of local image processing, the results may be com-pared to a list of authorized vehicles. If a vehicle is rec-ognized as authorized, images and/or data may be dis-carded rather than forwarded for further processing.[0042] Images and data can be forwarded to a centralprocessing facility such as the image database 14 oper-ating in conjunction with the billing engine 22. This proc-ess may involve a computer network, but may also in-clude physical media from another computer located atthe image acquisition site (i.e., facility 28). Generally, in-formation can be temporarily stored on a computer at theimage acquisition site in the event the network is una-vailable.[0043] Images received at the central site may not havebeen processed. Any unprocessed images can be han-dled as described above. The data resulting from imageprocessing (remote or central) may be separated into twocategories. Data that meets application-specific or juris-diction-specific criteria for confidence may be sent direct-ly to the billing engine 22. On the other hand, data results

9 10

EP 2 790 157 A1

7

5

10

15

20

25

30

35

40

45

50

55

not meeting required confidence levels may be flaggedfor additional processing. Additional processing may in-clude, for example, determining whether multiple imagesof a vehicle are available and independently processingthe images and comparing the results. This may includecharacter-by-character comparisons of the results of op-tical character recognition (OCR) on the license plate im-age. In another example, the image(s) may be processedby one or more specialized algorithms for recognizinglicense plates of certain types or styles (such as platesfrom a particular jurisdiction). These algorithms may con-sider the validity of characters for each position on thelicense plate, the anticipated effect of certain design fea-tures (such as background images), or other style-spe-cific criteria. The processed image may be forwardedbased on preliminary processing results, or may includeprocessing by all available algorithms to determine thehighest confidence level.[0044] Preliminary data may be compared to other dataavailable to increase the confidence level. Such tech-niques include:

(1) Comparing OCR processed license plate dataagainst lists of valid license plate numbers within thebilling system or at the appropriate jurisdiction’s mo-tor vehicle registration authority.(2) Comparing other data obtained from sensors atthe imaging location (such as vehicle size) to knowncharacteristics of the vehicle registered under theregistration number recognized by the system, in therecognized jurisdiction or in multiple jurisdictions.(3) Comparing the registration and other data torecords from other sites (e.g., records of the sameor similar vehicle using other facilities on the sameday, or using the same facility at other times).(4) Comparing vehicle fingerprint data against storedlists of vehicle fingerprint data. The use of vehiclefingerprint data for vehicle identification is describedin more detail below.(5) Manually viewing the images or data to confirmor override the results of automated processing.

[0045] If additional processing provides a result with aparticular confidence level, the resulting data then canbe forwarded to the billing engine 22. If the required con-fidence level cannot be attained, the data may be keptfor future reference or discarded.[0046] The billing engine 22 processes the informationcaptured during the interaction between the vehicle andthe toll facility, including the vehicle identifier as deter-mined by the image processing module 25 to create atransaction event corresponding to an interaction be-tween the vehicle and the facility. The engine 22 canstore the transaction event in a billing database 16 forsubsequent payment processing. For example, the bill-ing engine 22, alone or in combination with a customermanagement module 26 (described below), producespayment requests based on the transaction events. The

transaction event data may include individual chargesbased on a vehicle’s presence at specific points or facil-ities, or trip charges based on a vehicle’s origin and des-tination involving a facility. These transaction events canbe compiled and billed, for example, by one or more ofthe following methods:

(1) Deducting payment from an account establishedby the vehicle owner or operator. For example, thebilling database 20 can be used to store an accountrecord for each vehicle owner. In turn, each accountrecord can include a reference to one more transac-tion events. A paper or electronic payment statementmay be issued and sent to the registered owner ofthe vehicle.(2) Generating a paper bill and sending it to the ownerof the vehicle using a mailing address derived froma vehicle registration record.(3) Presenting an electronic bill to a predefined ac-count for the vehicle owner, hosted either by the com-puter 12 or a third party.(4) Submitting a bill to the appropriate vehicle regis-tration authority or tax authority, permitting paymentto be collected during the vehicle registration renew-al process or during the tax collection process.

[0047] Billing may occur at regular intervals, or whentransactions meet a certain threshold, such as maximuminterval of time or maximum dollar amount of outstandingtoll charges and other fees. Owners may be able to ag-gregate billing for multiple vehicles by establishing anaccount with the computer 12.[0048] The customer management module 26 can al-low a user to interact with the toll management computer12 over a communications channel such as a computernetwork (e.g., Internet, wired, wireless, etc.), a telephoneconnection, or other channel. The user can include a par-ty associated with a vehicle 22 (e.g., owner of the vehi-cle), a public or private authority responsible for manage-ment of the facility 28, or other user. The customer man-agement module 26 includes a combination of hardwareand software module configured to handle customer in-teractions such as an account management module 26a,a dispute management module 26b and a paymentprocessing module 26c. The module 26 employs secureaccess techniques such as encryption, firewalls, pass-word or other techniques.[0049] The account management module 26a allowsusers such as motorists to create an account with thesystem 10, associate multiple vehicles with that account,view transactions for the account, view images associ-ated with those transactions, and make payments on theaccount. In one implementation, a user responsible forthe facility can access billing and collection informationassociated with motorists that have used the facility.[0050] The dispute management module 26b may per-mit customers to dispute specific transactions on theiraccounts and to resolve disputes using the computer 12

11 12

EP 2 790 157 A1

8

5

10

15

20

25

30

35

40

45

50

55

or third parties. Disputes may arise during billing situa-tions. The module 26b may help resolve such disputesin an automated fashion. The module 26b can provide acustomer to access an "eResolution" section of a con-trolling/billing authority website. Customers can file a dis-pute and download an image of their transaction, the onein question. If there is no match (i.e., the customers au-tomobile is not the automobile in the photo frame), thebill can be forwarded for a third party evaluation such asarbitration. In the far more likely case, the photo will showthat the customer’s automobile was indeed billed correct-ly. Dispute management can use encrypted security inwhich all text and images are sent over a computer net-work (e.g., the Internet) using high strength encryption.Proof of presence images can be embedded into the dis-pute resolution communication as an electronic water-mark.[0051] The payment processing module 26c providesfunctionality for processing payments manually or elec-tronically, depending on the remittance received. For ex-ample, if payment remittance is in the form of a papercheck, then scanning devices could be used to convertthe paper information into electronic format for furtherprocessing. On the other hand if electronic payment isemployed, then standard electronic payment techniquescan be used. The payment processing module 26c cansupport billing methods such as traditional mailing, elec-tronic payment (e.g. using a credit card, debit card, smartcard, or Automated Clearing House transaction),periodicbilling (e.g., send the bill monthly, quarterly, upon reach-ing a threshold, or other). The payment processing mod-ule 26c can support discounts and surcharges based onfrequency of usage, method of payment, or time of facilityusage. The payment processing module 26c also cansupport payment collection methods such as traditionalcheck processing, processing payment during renewalof a vehicle registration (with interest accrued), electronicpayment, direct debit bank, credit cards, pre-payment,customer-initiated payments(as often as the customerdesires), or provide discounts for different purposes.[0052] The toll management computer 12 communi-cates with external systems 34 using one or more com-munications techniques compatible with the communi-cations interfaces of the systems. For example, commu-nications interfaces can include computer networks suchas the Internet, electronic data interchange (EDI), batchdata file transfers, messaging systems, or other interfac-es. In one implementation, external systems 34 includelaw enforcement agencies 36, postal authorities 38, ve-hicle registration authorities 40, insurance companies 42,service providers 44, financial systems 46 and a home-land security agency 48. The external systems 34 caninvolve private or public organizations that span one ormore geographic locations such as states, regions, coun-tries, or other geographic locations.[0053] The toll management computer 12 can interfaceand exchange information with law enforcement agen-cies 36. For example, as vehicles are identified, the com-

puter can submit substantially real-time transactions tolaw enforcement systems, in formats defined by the lawenforcement agencies. Transactions also can be submit-ted for vehicles carrying hazardous materials or violatingtraffic regulations (e.g. speeding, weight violations, miss-ing plates), if the appropriate sensors are in place(e.g.laser/sonic/microwave detectors as described above,weight sensors, radiation detectors). Alternatively, vehi-cle records can be compiled and forwarded in batches,based on lists provided by law enforcement agencies.[0054] The highlighted vehicle identifier database 20can be used to store the lists provided by the law en-forcement agencies. The term "highlighted" refers to thenotion that the law enforcement agencies have provideda list of vehicle identifiers that the agencies have indicat-ed (highlighted) they wish the toll facility to monitor. Forexample, when a motor vehicle is stolen and reported topolice, the police can send a list of highlighted vehicleidentifiers to the database 20. When the vehicle high-lighted by the police travels through facility, the imagingprocessing module 24 determines a vehicle identifier as-sociated with the vehicle and determines through certaininterfaces that the particular vehicle is being sought bylaw enforcement. The law enforcement authorities maywish to be instantly notified of the location of the vehicle(and driver), the time it was detected at the location, andthe direction it was headed. The computer 12 can notifyin substantially real-time mobile units associated with lawenforcement. In addition, law enforcement can automat-ically highlight vehicles based upon the expiration of alicense, occurrence of a traffic court date, or other event.This could, in turn, help keep illegal drivers off the roadand increase revenue to the state.[0055] The toll management computer 12 can interfaceand exchange information with postal authorities 38.Since the disclosed techniques would require toll author-ities to convert from receiving payment by drivers at thetime of travel to receiving paying in arrears, it is importantthat bills be sent to the correct driver/vehicle owner. Tominimize the possibility of sending the bill to the wrongperson, the computer 12 supports address reconciliation.For example, before a bill is mailed, the computer 12-verifies that the address provided by a motor vehicle de-partment matches the address provided by the postalauthority. The motor vehicle database can then be up-dated with the most accurate address information relatedto the vehicle owner. Since this occurs before the bill ismailed, billing errors can be reduced.[0056] The toll management computer 12 can interfac-es and exchange information with vehicle registration au-thorities 40. The registration authorities 40 provide aninterface to exchange information related to the ownersof vehicles, the owners’ addresses, characteristics of thevehicles, or other information. Alternatively, this informa-tion can be accessed through third-party data providersrather than through an interface to public motor vehiclerecords. The accuracy of records in the various databas-es used by the computer 12, including vehicle ownership

13 14

EP 2 790 157 A1

9

5

10

15

20

25

30

35

40

45

50

55

and owner addresses, may be verified periodicallyagainst third-party databases or government records, in-cluding motor vehicle records and address records. Thismay help ensure the quality of ownership and addressrecords, and reduce billing errors and returned corre-spondence.[0057] The toll management computer 12 can interfaceand exchange information with insurance companies 42.Insurance companies could highlight vehicle identifiersin a manner similar to law enforcement authorities 36.For example, the highlighted vehicle identifiers database20 can include license plate numbers of vehicles with anexpired insurance indicating that such drives would bedriving illegally. The computer could notify law enforce-ment as well as insurance companies whether the high-lighted vehicle has been detected using a particular fa-cility.[0058] The toll management computer 12 can interfaceand exchange service providers 44. For example, thecomputer 12 can support batch or real-time interfacesfor forwarding billing and payment collection functions tobilling service providers or collection agencies.[0059] The toll management computer 12 can interfaceand exchange information with financial systems 46. Forexample, to handle bill payment and collection, the com-puter 12 can interface to credit card processors, banks,and third-party electronic bill presentment systems. Thecomputer 12 can also exchange information with ac-counting systems.[0060] The toll management computer 12 can interfaceand exchange information with the homeland securityagency 48. The office of homeland security can automat-ically provide a list of individuals for use in the highlightedvehicle identifier database 20. For example, registereddrivers that are on a visa to this country can be automat-ically highlighted when that visa expires. The computer12 would then notify the office of homeland security 48that the highlighted vehicle identifier associated with theperson has been detected driving in the country includingthe time and location information about the vehicle.[0061] As described above, data captured from the tollsite flows into the image database, and is retrieved fromthe image database by the billing engine. In another im-plementation, the toll computer detects, for each vehicle,an interaction between the vehicle and a toll facility, cap-tures images and generates a data record. The datarecord can include date, time, and location of transaction,a reference to the image file, and any other data availablefrom the sensors at the facility (e.g., speed, size). Theimage can be passed to the image-processing module25, which can generate a vehicle identifier, a state, anda confidence factor for each vehicle.[0062] This information can be added to the datarecord. (This process my occur after transmission to thecentral facility.) The data record and image file can besent to the central facility. The image can be stored inthe image database, and referenced if (a) additionalprocessing is required to identify the vehicle, or (b) some-

one wishes to verify the transaction. If the confidencelevel is adequate, the data record can be submitted tothe billing engine, which can associate it with an accountand store it in the billing database for later billing. If noaccount exists, the vehicle identifier is submitted to theappropriate state registration authority or a third-partyservice provider to determine the owner and establish anaccount. This process may be delayed until enoughtransactions are collected for the vehicle to justify issuinga bill. If confidence level is not adequate, additionalprocessing may be performed as described elsewhere.[0063] The techniques described above describe theflow of data based on a single transaction end-to-end,then looping back to the beginning. In another implemen-tation, some of the functions described may be event-driven or scheduled, and may operate independently ofone another. For example, there may be no flow of controlfrom back-end processes to vehicle imaging. The imag-ing process may be initiated by an event, including thepresence of a vehicle at the toll site.[0064] In another implementation, the system may beused to monitor traffic and manage incidents. For exam-ple, if a drop in average vehicle speed is detected, thecomputer can send a message to a highway control fa-cility alerting controllers to the possibility of an incident.Authorized controllers may communicate with the equip-ment at the toll site to view images from the cameras anddetermine if a response is required.[0065] The operation of the toll management system10 is explained with reference to FIGS. 2-5.[0066] FIG. 2 is a flow chart of an implementation ofelectronic toll management system related, particularlya process 100 for managing highlighted vehicle identifi-ers 20 provided by external systems 34. To illustrate, inone example, it is assumed that law enforcement agen-cies 36 generate a list of highlighted vehicle identifiers(e.g., license plate numbers) of drivers being sought bythe agencies and that the agencies 36 wish to be notifiedwhen such vehicles have been identified using a toll fa-cility 28.[0067] The computer 12 obtains (block 102) highlight-ed vehicle identifiers from a party such as law enforce-ment agencies 36. In one implementation, these vehicleidentifiers can be stored in the vehicle identifier database20 for subsequent processing. The database 20 can beupdated by the agencies with new as well as additionalinformation in real-time and/or in batch mode. The lawenforcement agencies accessed by the computer spanacross multiple jurisdictions such as cities, municipali-ties, states, regions, countries or other geographic des-ignations. As a result, the computer 12 can process ve-hicle information across multiple jurisdictions and on anational scale.[0068] The computer 12 captures (block 104) an imageof a vehicle triggered by a transaction event based on aninteraction between the vehicle 30 and the facility 28. Forexample, the image acquisition module 24 can be usedto acquire one or more images of a vehicle as it travels

15 16

EP 2 790 157 A1

10

5

10

15

20

25

30

35

40

45

50

55

through a facility such as a toll road. These images canbe stored in the image database 14 for further processingby the image-processing module 25. Compression tech-niques can be applied to the captured images to helpreduce the size of the database 14.[0069] The computer 12 determines (block 106) a ve-hicle identifier based on the captured image. For exam-ple, as discussed previously, the image-processing mod-ule 25 can apply image analysis techniques to the rawimages in the image database 14. These analysis tech-niques can extract a license number from one or moreimages of a license plate of the vehicle. The extractedvehicle identifiers can be stored in the vehicle identifierdatabase 18 for further processing.[0070] The computer 12 compares (block 108) a cap-tured vehicle identifier with the highlighted vehicle iden-tifier. For example, the computer 12 can compare a cap-tured license plate number from the vehicle identifier da-tabase 18 with a license number from the highlightedvehicle identifier database 20. As discussed above, au-tomatic as well as manual techniques can be applied tocheck for a match.[0071] If the computer 12 detects a match (block 110)between the license numbers, then it checks (block 112)how the party associated with the highlighted vehicleidentifiers wishes to be notified. This information can bestored in the vehicle identifier database 20 or other stor-age mechanism. On the other hand, if there is no match,the computer 12 resumes executing the process 100 be-ginning at block 102.[0072] If the party indicates that it wishes to be notifiedimmediately (block 114), then the computer notifies(block 118) the party upon the occurrence of a match. Inthis example, the computer can notify law enforcementof the match in substantially real-time using wirelesscommunications techniques or over a computer network.[0073] On the other hand, if the party does not wish tobe notified immediately (block 114), then the computer12 stores (block 116) the match for later notification uponsatisfaction of predefined criteria. In one implementation,predefined criteria can include gathering a predefinednumber of matches and then sending the matches to lawenforcement in batch mode.[0074] Once the party has been notified (block 118) ofa match or the match has been stored for later notification(block 116), the computer 12 resumes executing process100 beginning at block 102.[0075] FIG. 3 is a flow chart of an implementation ofelectronic toll management system 10, particularly aprocess 200 for managing payment from a party associ-ated with a vehicle that has interacted with a facility. Toillustrate, in one example, it is assumed that a toll roadauthority decides to employ the disclosed techniques tohandle payment processing including billing and collect-ing tolls from vehicles using its toll road.[0076] The computer 12 captures (block 202) an imageof a vehicle triggered by a transaction event based on aninteraction between the vehicle and a facility. This func-

tion is similar to the process discussed above in referenceto block 104 of FIG. 2. For example, the image acquisitionmodule 24 can be used to acquire one or more imagesof a vehicle 30 as it travels through the toll road 28. Theseimages can be stored in the image database 14 for furtherprocessing by the image-processing module 25.[0077] The computer 12 determines (block 204) a ve-hicle identifier based on the captured image. This func-tion is also similar to the process discussed above inreference to block 106 of FIG. 2. For example, the image-processing module 25 can be used to extract a licensenumber from one or more images of a license plate ofthe vehicle. These vehicle identifiers can be stored in thevehicle identifier database 18 for further processing.[0078] The computer 12 determines (block 206) a partyassociated with the vehicle identifier by searching a reg-istration authority databases. For example, the computer12 can use the vehicle identifier from the vehicle identifierdatabase 18 to search a database of a vehicle registrationauthority 40 to determine the registered owner of the ve-hicle associated with the vehicle identifier. The computer12 is capable of accessing vehicle information from oneor more vehicle registration databases across multiplejurisdictions such as cities, municipalities, states, re-gions, countries or other geographic locations. In oneimplementation, the computer 12 can maintain a copy ofregistration information from multiple registration author-ities for subsequent processing. Alternatively, the com-puter 12 can access multiple registration authorities andobtain registration information on a demand basis. In ei-ther case, these techniques allow the computer 12 toprocess vehicle information across multiple jurisdictions,and thus process vehicles on a national scale.[0079] The computer 12 checks (block 208) whetherto request payment from the party associated with thevehicle identifier. The request for payment can dependon payment processing information associated with theregistered owner. For example, the registered owner maybe sent a bill based on a periodic basis (e.g., monthlybasis), when a predefined amount has been reached, orother arrangement.[0080] If the computer 12 determines that payment isrequired (block 210), then it requests (block 214) pay-ment from the party associated with the vehicle identifierbased on the transaction event. As discussed above, arequest for payment can be generated using traditionalmail service techniques or electronic techniques such aselectronic payment. The amount of the bill can dependon information from the transaction event such as thenature of the interaction between the vehicle and the fa-cility. For example, the transaction event can indicatethat the vehicle traveled a particular distance defined asa distance between a starting and ending point on thetoll road. Accordingly, the amount of the payment re-quested from the registered owner can be based on thedistance traveled.[0081] On the other hand, if the computer 12 deter-mines that payment is not required (block 210), then it

17 18

EP 2 790 157 A1

11

5

10

15

20

25

30

35

40

45

50

55

forwards (block 212) the transaction event to another par-ty to handle the payment request. For example, the tollauthority may have decided that the computer 12 canhandle image processing functions and that toll billingand collection should be handled by a third party suchas external systems 34. In one implementation, the com-puter 12 can interface with service providers 44 and fi-nancial systems 48 to handle all or part of the billing andpayment-processing functionality. Once the transactionevent has been forwarded to a third party, the computer12 resumes executing the functions of process 200 be-ginning at block 202.[0082] If the computer handles payment processing,the computer 12 processes (block 216) a payment re-sponse from the party associated with the vehicle iden-tifier. In one implementation, the billing database 16, inconjunction with the billing engine 22 and the customermanagement module 26, can be used to handle billingand collection functions. As discussed above, the pay-ment processing module 26c can support electronic ormanual payment processing depending on the remit-tance received. For example, the computer 12 can pro-vide an account for handling electronic payment process-ing over a computer network such as the Internet. Thecomputer can also handle traditional payment receiptsuch as a check.[0083] Once a payment has been processed (block216), the computer 12 resumes executing process 200beginning at block 202.[0084] FIG. 4 is a flow chart of an implementation ofelectronic toll management system 10, particularly proc-ess 300 for managing payment over a communicationschannel from a party associated with a vehicle that hasinteracted with a facility. To illustrate, assume a toll au-thority responsible for a toll road employs the disclosedtechniques and that a registered owner wishes to effi-ciently and automatically make payments for using thetoll road.[0085] The computer 12 provides (block 302) an ac-count for a party associated with the vehicle identifier. Inone embodiment, the computer 12 in conjunction withthe account management module 26a can provide a web-site for customers to open an account for making elec-tronic payment over a computer network such as the In-ternet. The website also can permit the customer to ac-cess and update account information such as paymenthistory, payment amount due, preferred payment meth-od, or other information.[0086] The computer 12 receives (block 304) a requestover a communications channel from the party to reviewa transaction event. For example, the account paymentmodule 26a can handle this request by retrieving trans-action event information associated with the customer’saccount from the billing database 16. The retrieved in-formation can include image data of a particular transac-tion involving the customer’s vehicle and the tollbooth.[0087] The computer 12 sends (block 306) the trans-action event to the party 32 over the communications

channel. Information related to the transaction event caninclude images of the vehicle and the vehicle identifier(i.e., license plate). Such data can be encrypted to permitsecure transmission over the Internet. Standard commu-nications protocols such as hypertext markup language(HTML) can be used to transmit the information over theInternet.[0088] The computer 12 determines (block 308)whether the party agrees to make payment. For example,once the customer receives the information related to thetransaction event, the customer can review the informa-tion to determine whether to make payment based onwhether the vehicle shown in the images is the custom-er’s vehicle.[0089] If the computer 12 determines (block 310) thatthe party agrees to pay, then it processes (block 314)payment from the party by deducting an amount from theaccount based on the transaction event. For example, ifthe image information indicates that the transaction eventdata is accurate, then the customer can authorize pay-ment such as by submitting an electronic payment trans-action.[0090] On the other hand, if the computer 12 deter-mines (block 310) that the party does not agrees to pay,then the computer 12 processes (block 312) a paymentdispute request from the party. In one implementation,the dispute management module 26b can handle a dis-pute request submitted by the customer using onlinetechniques. The module 26b can handle specific trans-actions related to the customer’s account including in-volving a third party to resolve the dispute.[0091] Once a payment has been processed (block314) or a dispute resolved (block 312), the computer 12resumes executing process 300 beginning at block 304.[0092] FIG. 5 is a flow chart of an implementation ofelectronic toll management system, particularly a proc-ess 400 for reconciling mailing addresses from differentsources. To illustrate, it is assumed that a toll authorityhas decided to employ the disclosed techniques forprocessing payment related to the use of toll facility.Since the disclosed techniques involve processing pay-ment some time after the vehicle has traveled throughthe toll authority, these techniques help ensure that pay-ment is sent to the correct address of the registered own-er of the vehicle.[0093] The computer 12 determines (block 402) that apayment request is to be sent to a party associated witha vehicle identifier. As explained above, for example,payment requests may be generated based on a periodicbasis or on an amount threshold basis.[0094] The computer 12 accesses (block 404) a vehi-cle registration authority for a mailing address of a partyassociated with the vehicle identifier. For example, thecomputer 12 may access one or more databases asso-ciated with vehicle registration authorities 40 to retrieveinformation such as the mailing address of the registeredowner of the vehicle.[0095] The computer 12 accesses (block 406) a postal

19 20

EP 2 790 157 A1

12

5

10

15

20

25

30

35

40

45

50

55

authority for a mailing address of the party associatedwith the vehicle identifier. For example, the computer 12may access one or more databases associated with post-al authorities 38 to retrieve information such as the mail-ing address of the registered owner of the vehicle.[0096] The computer 12 compares (block 408) themailing address from the vehicle registration authoritywith the mailing address from the postal authority. Forexample, the computer compares the mailing addressesfrom the two authorities to determine if there is a discrep-ancy between the database information.[0097] If the computer 12 determines (block 410) thatthe addresses match, then it requests (block 414) pay-ment from the party associated with the vehicle identifierusing the mailing address accessed from the postal au-thority. For example, the computer 12 can use the tech-niques discussed above to handle payment processingincluding billing and collecting payment from the regis-tered owner.[0098] On the other hand, if the computer 12 deter-mines (block 410) that the addresses do not match, itthen updates (block 412) the vehicle registration authoritywith the mailing address from the postal authority. Forexample, the computer 12 can update databases asso-ciated with vehicle registration authorities 40 with the cor-rect mailing address retrieved from the postal authorities38. Such techniques may help reduce the likelihood ofmailing a bill to an incorrect mailing address resulting inan reducing time for payment remittance.[0099] Once the vehicle registration authority has beenupdated (block 412) or payment requested (block 414),the computer 12 executes process 400 beginning atblock 402 as explained above.[0100] FIG. 6 is a block diagram of an implementationof an electronic toll management system 600 that pro-vides vehicle identification by extracting multiple vehicleidentifiers for each vehicle that interacts with the toll fa-cility. The toll management system 600 includes a tollmanagement computer 612. The toll management com-puter includes an image database 614, a billing database616, a vehicle identification database 618, a highlightedvehicle identifier database 620, a billing engine 622, animage acquisition module 624, an image processingmodule 625, and a customer management module 626.The toll management computer 612 communicates withor is integrated with a toll facility 628, which interacts witha vehicle 630 and a party associated with the vehicle632. The toll management computer 612 also communi-cates with external systems 634.[0101] Examples of each element within the toll man-agement system 600 of FIG. 6 are described broadlyabove with respect to FIG. 1. In particular, the toll man-agement computer 612, the image database 614, thebilling database 616, the vehicle identification database618, the highlighted vehicle identifier database 620, thebilling engine 622, the image acquisition module 624, theimage processing module 625, the customer manage-ment module 626, and the toll facility 628 typically have

attributes comparable to and illustrate one possible im-plementation of the toll management computer 12, theimage database 14, the billing database 16, the vehicleidentification database 18, the highlighted vehicle iden-tifier database 20, the billing engine 22, the image acqui-sition module 24, the image processing module 25, thecustomer management module 26, and the toll facility 28of FIG. 1, respectively. Likewise, the vehicle 630, theparty associated with the vehicle 632, and the externalsystems 634 typically have attributes comparable to thevehicle 30, the party associated with the vehicle 32, andthe external systems 34 of FIG. 1.[0102] The vehicle identification database 618 in-cludes an extracted identifier database 6181, a vehiclerecord database 6182, and a read errors database 6183.The functions of the databases 6181-6183 are describedin more detail below.[0103] The system 600 is similar to system 10 and isconfigured to provide, for example, reduced vehicle iden-tification error rates by identifying each vehicle throughuse of multiple vehicle identifiers. Two such identifiersare designated as 631A and 631B. A vehicle identifier ispreferably an identifier that uniquely or substantiallyuniquely identifies the vehicle but may be an identifierthat helps in the identification process by distinguishingthe vehicle from other vehicles without necessarilyuniquely identifying the vehicle. Identifiers 631A and631B may be part of vehicle 630, as suggested by FIG.6, but need not be. For example, identifiers 631A and/or631B may be produced by image processing module 625based on characteristics of the vehicle 630.[0104] As described previously, one example of a ve-hicle identifier is license plate information of a vehicle,such as a license plate number and state. The imageprocessing module 625 may determine the license plateinformation of a vehicle from an image of the license plateby using OCR, template matching, and other analysistechniques. A license plate number may include anycharacter but is typically restricted to alphanumeric char-acters. License plate information typically may be usedto uniquely identify the vehicle.[0105] Another example of a vehicle identifier is a ve-hicle detection tag as described in U.S. Patent No.6,747,687, hereby incorporated by reference in its en-tirety for all purposes. The vehicle detection tag, herein-after referred to as a vehicle fingerprint, is a distilled setof data artifacts that represent the visual signature of thevehicle. The image processing module 625 may generatea vehicle fingerprint by processing an image of the vehi-cle. To save on processing time and storage needs how-ever, the generated vehicle fingerprint typically does notinclude the normal "picture" information that a humanwould recognize. Accordingly, it is usually not possibleprocess the vehicle fingerprint to obtain the original ve-hicle image. Some vehicle fingerprints, however, mayinclude normal picture information. A vehicle fingerprinttypically may be used to uniquely identify the vehicle.[0106] In one implementation, a camera in the image

21 22

EP 2 790 157 A1

13

5

10

15

20

25

30

35

40

45

50

55

acquisition module 624 captures a single "still" image ofthe back of each vehicle that passes the toll facility 628.For each vehicle, the image processing module 625 rec-ognizes the visual cues that are unique to the vehicle andreduces them into a vehicle fingerprint. Because a li-cense plate is a very unique feature, the image process-ing module 625 typically maximizes the use of the licenseplate in creating the vehicle fingerprint. Notably, the ve-hicle fingerprint also includes other parts of the vehiclein addition to the license plate and, therefore, vehicleidentification through matching of vehicle fingerprints isgenerally considered more accurate than vehicle identi-fication through license plate information matching. Thevehicle fingerprint may include, for example, portions ofthe vehicle around the license plate and/or parts of thebumper and the wheelbase.[0107] Another example of a vehicle identifier is a ve-hicle signature generated using a laser scan (hereinafterreferred to as a laser signature). The laser signature in-formation that may be captured using a laser scan mayinclude one or more of an overhead electronic profile ofthe vehicle, including the length, width, and height of thevehicle, an axle count of the vehicle, and a 3D image ofthe vehicle. In one implementation, the image acquisitionmodule 624 includes two lasers for a given lane, one thatis mounted over the lane and another that is mountedalongside of the lane. The laser mounted above the lanetypically scans the vehicle to capture the overhead profileof the vehicle, and the laser mounted alongside or aboveof the lane typically scans the vehicle to capture the axlecount of the vehicle. Together, both lasers are also ableto generate a 3D image of the vehicle. A laser signaturemay be used to uniquely identify some vehicles. For ex-ample, vehicles that have been modified to have a dis-tinctive shape may be uniquely identified by a laser sig-nature.[0108] Another example of a vehicle identifier is a ve-hicle signature generated using a magnetic scan (here-inafter referred to as an inductive signature). The induc-tive signature of a vehicle is a parameter that reflects themetal distribution across the vehicle and, therefore, maybe used to classify the vehicle and, in some circumstanc-es, to uniquely identify the vehicle (e.g., if the metal dis-tribution of a particular vehicle is unique to that vehiclebecause of unique modifications to that vehicle). The in-ductive signature may include information that may beused to determine one or more of the axle count (andlikely the number of tires) of the vehicle, the type of engineused in the vehicle, and the type or class of vehicle. Inone implementation, the image acquisition module 624includes a a pair of vehicle detection loops, an axle de-tection loop, and a camera trigger loop in each lane..[0109] Once the two or more vehicle identifiers are ex-tracted by the image processing module 625, the imageprocessing module 625 stores the extracted vehicle iden-tifiers in the extracted vehicle identifier database 6181.Ideally, the computer 612 would then be able to uniquelyidentify the owner of the vehicle by choosing a vehicle

identifier that uniquely identifies the vehicle (e.g., licenseplate information or vehicle fingerprint) and searchingone or more internal or external vehicle record databasesfor a record containing a matching vehicle identifier. Un-fortunately, extracting a vehicle identifier is an imperfectprocess. The extracted vehicle identifier may not corre-spond to the actual vehicle identifier, and therefore, maynot uniquely identify the vehicle. An incorrectly or partiallyextracted vehicle identifier may not match the vehicleidentifier of any vehicle, may match the vehicle identifierof the wrong vehicle, or may match the vehicle identifiersof more than one vehicle. To increase identification ac-curacy, the computer 612 of the system 600 implementsa multi-tier identification process using two or more ve-hicle identifiers.[0110] FIG. 7 is a flow chart of an exemplary two-tieridentification process 700 that may be implemented toincrease the accuracy of vehicle identification. Imageand/or sensor data is captured for a vehicle that interactswith a toll facility (hereinafter referred to as the "targetvehicle") and two vehicle identifiers are extracted fromthe captured data (block 710). In one implementation,only image data is collected and the two vehicle identifiersextracted are a license plate number and a vehicle fin-gerprint. In another implementation, image data and in-ductive sensor data are collected and the vehicle identi-fiers extracted are the vehicle fingerprint and the induc-tive signature.[0111] One of the two extracted vehicle identifiers isdesignated as the first vehicle identifier and used to iden-tify a set of one or more matching vehicle candidates(block 720). Typically, the vehicle identifier that isdeemed to be the least able to accurately and/or uniquelyidentify the target vehicle is designated as the first vehicleidentifier. For example, if the two extracted vehicle iden-tifiers were license plate number and vehicle fingerprint,the license plate number would be designated as the firstvehicle identifier because of the lower expected accuracyof vehicle identification through license plate matchingas compared to fingerprint matching. The one or morematching vehicle candidates may be determined, for ex-ample, by accessing a vehicle record database and find-ing records that contain vehicle identifiers that match ornearly match the first vehicle identifier.[0112] Once the set of one or more matching vehiclecandidates is determined, the target vehicle is identifiedfrom the set based on the second vehicle identifier (block730). For example, if 12 vehicle candidates were identi-fied as matching a partially extracted license platenumber, the target vehicle is identified by accessing thevehicle fingerprints for each of the 12 vehicle candidatesand determining which of the 12 vehicle fingerprintsmatches the extracted vehicle fingerprint. If no match isfound within a predetermined confidence threshold, man-ual identification of the vehicle may be used. In anotherimplementation, one or more larger sets (e.g., supersets)of matching vehicle candidates are determined succes-sively or concurrently by changing (e.g., loosening) the

23 24

EP 2 790 157 A1

14

5

10

15

20

25

30

35

40

45

50

55

criteria for matching and additional attempts are madeto identify the target vehicle from each of the one or morelarger sets prior to resorting to manual identification.[0113] In some implementations, the toll managementsystem may be purposefully designed to identify a largerset of matching vehicle candidates during operation 720to, for example, ensure that the expected lesser accuracyof vehicle identification through the first identifier doesnot erroneously result in exclusion of the target vehiclefrom the set of matching vehicle candidates. For exam-ple, if the first vehicle identifier is a license plate number,the license plate reading algorithm may be intentionallymodified in, for example, two ways: (1) the matching cri-teria of the license plate reading algorithm may be loos-ened to enable the algorithm to generate a larger set ofmatching vehicle candidates and (2) the license platereading algorithm may be "detuned" by lowering the readconfidence threshold used to determine whether a readresult is included in the matching candidate set. For in-stance, the license plate reading algorithm may be loos-ened to only require a matching vehicle candidate tomatch a subset or lesser number of the characters in thelicense plate number extracted for the target vehicle. Ad-ditionally or alternatively, the read confidence thresholdmay be lowered to enable previously suspected incorrectreads (i.e., partial or low confidence reads) to be includedin the matching vehicle candidate set.[0114] The two-tier identification process 700 providesgreater identification accuracy over a single-tier/singleidentifier identification system by requiring that two vehi-cle identifiers be successfully matched for successful ve-hicle identification. Moreover, the process 700 may pro-vide greater identification speed by limiting the matchingof the second vehicle identifier to only those vehicle can-didates having records that successfully match the firstvehicle identifier. This can provide increased speed if,for example, the extracted second vehicle identifier istime-consuming to match against other such identifiersor if a large number of other such identifiers exists (e.g.,millions of identifiers for millions of vehicles in a vehicledatabase).[0115] In another implementation, two or more secondidentifiers are used to identify the target vehicle fromamong the set of matching vehicle candidates. Each ofthe second identifiers must match the same candidatevehicle to within a predetermined confidence level forsuccessful vehicle identification. Alternatively, the de-gree of matching of each of the two or more second iden-tifiers may be weighted and a combined equivalentmatching score may be generated. If the combined equiv-alent matching score is above a predetermined thresh-old, the identification is deemed successful.[0116] In one implementation, each second vehicleidentifier is assigned a match confidence level numberthat ranges from 1 to 10, where 1 corresponds to nomatch and 10 corresponds to an exact match. Each ve-hicle identifier is also assigned a weight value from 1 to10, with greater weight values being assigned to vehicle

identifiers that are considered more accurate in uniquelyidentifying vehicles. If, for example, the second vehicleidentifiers are a laser signature and license plate infor-mation, a weighting of 6 may be assigned to the lasersignature and a greater weighting of 9 may be assignedto the license plate information. If a combined equivalentmatching score of 100 is necessary for an identificationto be deemed successful and the license plate informa-tion matches to a confidence level of 7 and the lasersignature also matches to a confidence level of 7, thecombined equivalent matching score would be7*6+7*9=105 and the identification would be consideredsuccessful.[0117] In another implementation, two or more first ve-hicle identifiers are used to identify vehicles in the set ofmatching vehicle candidates. Each of the first vehicleidentifiers for a possible candidate vehicle must matchthe target vehicle to within a predetermined confidencelevel for the possible candidate vehicle to be included inthe set of matching vehicle candidates. Alternatively, thedegree of matching of each of the two or more first iden-tifiers may be weighted and a combined equivalentmatching score may be generated. If the combined equiv-alent matching score is above a predetermined thresh-old, the possible candidate vehicle is included in the setof matching vehicle candidates.[0118] In another implementation, the second identifieris not used to uniquely identify the target vehicle fromamong the vehicles in the set of matching vehicle candi-dates. Rather, the second identifier is used to generatea new and smaller set of matching vehicle candidates asa subset of the set determined using the first identifier,and a third identifier is then used to uniquely identify thetarget vehicle from this subset of matching vehicle can-didates. In yet another implementation, multiple vehicleidentifiers are used to successively reduce the set ofmatching vehicle candidates and the target vehicle isuniquely identified from the successively reduced subsetthrough use of one or more final vehicle identifiers. In yetanother implementation, each of the multiple vehicleidentifiers is used to generate its own set of matchingvehicle candidates through matching and near matchingtechniques and the reduced set is the intersection of allof the determined sets. In yet another implementation,the reduced set is determined using a combination of theabove-described techniques.[0119] FIG. 8 is a flow chart of an exemplary two-tieridentification process 800 that may be implemented toincrease the accuracy and/or automation of vehicle iden-tification. Process 800 is an implementation of process700 wherein the first identifier is a license plate numberand the second identifier is a vehicle fingerprint. In par-ticular, process 800 includes operations 810-830, andassociated sub-operations, that correspond to and illus-trate one possible implementation of operations 710-730,respectively. For convenience, particular componentsdescribed with respect to FIG. 6 are referenced as per-forming the process 800. However, similar methodolo-

25 26

EP 2 790 157 A1

15

5

10

15

20

25

30

35

40

45

50

55

gies may be applied in other implementations where dif-ferent components are used to define the structure of thesystem, or where the functionality is distributed differentlyamong the components shown by FIG. 6.[0120] The image acquisition module 624 captures im-age data for the target vehicle based on an interactionbetween the target vehicle and the toll facility 628 (block812). In another implementation, the image acquisitionmodule 624 additionally or alternatively captures sensordata including, for example, laser scanning and/or loopsensor data. The image processing module 625 obtainslicense plate data, including, for example, a complete orpartial license plate number and state, for the target ve-hicle from the captured image data (block 814). Option-ally, the image processing module 625 also may deter-mine a vehicle fingerprint for the target vehicle from theimage data. In another implementation, the imageprocessing module 625 may determine other vehicle sig-nature data, such as, for example, laser and/or inductivesignature data, from the image data and/or sensor data.[0121] The computer 612 stores the captured imagedata in the image database 614 and stores the extractedlicense plate data in the extracted identifier database6181. If applicable, the toll management computer 612also stores the extracted vehicle fingerprint and othersignature data, such as, for example, the inductive sig-nature and/or laser signature, in the extracted identifierdatabase 6181.[0122] The computer 612 accesses a set of vehicleidentification records from the vehicle record database6182 (block 822). Each of the vehicle identificationrecords associates an owner/driver of a vehicle with ve-hicle identifier data. The computer 612 compares the ex-tracted license plate data with the license plate data inthe set of vehicle identification records (block 824) andidentifies a set of candidate vehicles from the vehicleshaving records in the set of records (block 826). The com-parison may be done using matching or near matchingtechniques.[0123] The computer 612 accesses extracted vehiclefingerprint data for the target vehicle (block 832). If thevehicle fingerprint has not already been determined/ex-tracted from the captured image data, the computer 612calculates the vehicle fingerprint and stores the vehiclefingerprint in the extracted vehicle identifier database6181.[0124] The computer 612 accesses vehicle fingerprintdata for a vehicle in the set of candidate vehicles by ac-cessing the corresponding vehicle identification record(block 834) and compares the vehicle fingerprint data forthe target vehicle to the vehicle fingerprint data for thecandidate vehicle (block 836). The computer 612 identi-fies the candidate vehicle as the target vehicle based onthe results of the comparison of the vehicle fingerprintdata (block 838). If the vehicle fingerprint data matcheswithin a predetermined confidence threshold, the candi-date vehicle is deemed to be the target vehicle, and theowner/driver of the candidate vehicle is deemed to be

the owner/driver of the target vehicle.[0125] FIGs. 9A-9C are a flow chart of an exemplarytwo-tier identification process 900 that may be imple-mented to increase the accuracy of vehicle identificationwhile minimizing the need for manual identification of ve-hicles. Process 900 is another implementation of process700 wherein the first identifier is a license plate numberand the second identifier is a vehicle fingerprint. In par-ticular, process 900 includes operations 910-930, andassociated sub-operations, that correspond to and illus-trate one possible implementation of operations 710-730,respectively. For convenience, particular componentsdescribed with respect to FIG. 6 are referenced as per-forming the process 800. However, similar methodolo-gies may be applied in other implementations where dif-ferent components are used to define the structure of thesystem, or where the functionality is distributed differentlyamong the components shown by FIG. 6.[0126] The image acquisition module 624 captures im-age and sensor data for the target vehicle (block 911).Roadside sensors, for example, trigger cameras thatcapture front and rear images of the target vehicle. Othersensors may capture additional data used for classifica-tion/identification of the vehicle. For example, a laserscan may be used to determine laser signature data in-cluding the height, width, length, axle count, and vehicledimensional profile. Sensors also may be used to deter-mine data related to the transaction between the targetvehicle and the toll facility 628 such as, for example, theweight of the vehicle, the speed of the vehicle, and trans-ponder data associated with the vehicle.[0127] The image processing module 625 performs alicense plate read on the captured image data, createsa vehicle fingerprint from the captured image data, andoptionally determines other vehicle signature/classifica-tion data from the captured sensor data (block 912). Forexample, the image processing module 625 may use anautomated license plate read algorithm to read one ormore of the captured images. The license plate read al-gorithm may read the captured images, for example, ina prioritized order based on visibility of the plate and itslocation in the image. The license plate read results mayinclude one or more of a license plate number, a licenseplate state, a license plate style, a read confidence score,a plate location in the image, and a plate size. The imageprocessing module 625 also may apply a visual signatureextraction algorithm to generate the vehicle fingerprintfor the target vehicle. The visual signature extraction al-gorithm may be similar to that developed by JAI-PULNiXInc. of San Jose, California and described in U.S. PatentNo. 6,747,687. The computer 612 stores the capturedimages in the image database 614 and stores the licenseplate read results, vehicle fingerprint, and other vehiclesignature/classification data in the extracted vehicleidentifier database 6181.[0128] The image processing module 625 determineswhether the captured images have provided any partialor complete read results for the license plate number and

27 28

EP 2 790 157 A1

16

5

10

15

20

25

30

35

40

45

50

55

state of the target vehicle (block 913). If no partial orcomplete read results were provided by the captured im-ages, process 900 proceeds to operation 941 of the man-ual identification process 940.[0129] If partial or complete read results for the licenseplate number and state of the target vehicle were provid-ed by the captured images, computer 612 searches thevehicle record database 6182 and read errors database6183 for the exact (either partial or complete) licenseplate number (as read by the license plate reader) (block921).[0130] The vehicle record database 6182 includesrecords for all vehicles previously recognized and poten-tially includes records for vehicles that are anticipated tobe seen. The vehicle record database 6182 is typicallypopulated through a registration process during which adriver/owner of a vehicle signs the vehicle up for auto-mated toll payment handling. The driver/owner of a ve-hicle may sign a vehicle up for automated toll paymenthandling by driving the vehicle through a special regis-tration lane in the toll facility 628 and providing a customerservice representative at the facility 628 with his or heridentity and other contact information. The image acqui-sition module 624 and the image processing module 625capture the license plate number, the fingerprint, and oth-er identification/classification data (e.g., the vehicle di-mensions) of the user’s vehicle while the vehicle travers-es the facility 628. The vehicle and owner identificationdata is stored in a new vehicle identification record as-sociated with the newly registered vehicle and own-er/driver.[0131] Alternatively, a driver/owner may register a ve-hicle for automatic toll payment handling by simply drivingthrough the facility 628, without stopping. The computer612 captures image data and sensor data for the vehicleand attempts to identify the driver/owner by reading thelicense plate image and looking up the read results in adatabase of an external system 634 (e.g., vehicle regis-tration authorities). It an owner/driver is identified, thecomputer 612 bills the owner/driver. Once a billing rela-tionship has been successfully setup, the computer 612officially registers the vehicle, generates as necessarythe vehicle fingerprint data and other signature/classifi-cation data from the captured image and sensor data,and stores these in a vehicle identification record asso-ciated with the identified owner/driver.[0132] In another implementation, the computer 612 isconfigured to obtain greater accuracy in identifying anunregistered driver/owner by looking up the license plateread results in a database of a vehicle registration au-thority (or other external system) and requesting a cor-responding vehicle identification number (VIN) from thevehicle registration authority (or other external system).The computer 612 uses the VIN to determine the make,model, and year of the vehicle. The make, model, andyear of the vehicle may be used to determine the length,width, and height of the vehicle. The computer 612 maythen determine a successful match of the target vehicle

with a vehicle registered with the vehicle registration au-thority not only by comparing license plate data but alsoby comparing vehicle dimensions (as captured, for ex-ample, in a laser signature and/or an inductive signature).Typically, the computer 612 will consider a match suc-cessful if the license plate read results for the target ve-hicle match the license plate data for the vehicle regis-tered with the vehicle registration authority to within apredetermined threshold and the vehicle dimensions ofboth vehicles match within a given tolerance.[0133] The make, model, and year of a vehicle may beused, for example, to determine the length, width, andheight of the vehicle by either accessing this informationfrom a public database or from a 3rd party database or,additionally or alternatively, by accessing the vehiclerecords database 6182 to retrieve the length, width, andheight data from one or more vehicle identificationrecords corresponding to vehicles having the samemake, model, and year as the target vehicle. Given thata vehicle’s dimensions may change if the vehicle hasbeen modified, the length, width, and height accessedfrom the vehicle identification records may vary by vehi-cle. Accordingly, the computer 612 may need to statisti-cally determine the appropriate dimensions for compar-ison by, for example, taking the average or medianlength, width, and height dimensions.[0134] In one implementation, the computer 612 iden-tifies a vehicle in part through use of an electronic signa-ture that includes a laser signature and/or an inductive(i.e., magnetic) signature. When a vehicle transacts withthe toll system, an electronic signature is captured forthe vehicle. The image and measurements of the vehiclecreated by the laser (i.e., the laser signature) and/or themagnetic scan (i.e., the inductive signature) are com-pared against known dimensions and images of vehiclesbased on vehicle identification number (VIN) that were,for example, previously captured by the toll system or byan external system. By comparing the electronic signa-ture image and dimensions to known dimensions of ve-hicles based on VIN, the search for a matching vehicleand associated VIN may be narrowed. If, for example,an LPR for the vehicle has a low confidence level, butthe electronic signature of the vehicle has been captured,the toll system may access a database, as describedabove, of known dimensions and images for vehicles andassociated VINs and cross reference the electronic sig-nature dimensions and images against the database toidentify the matching vehicle VIN or identify potentialmatching vehicle candidates/VINs. The read errors da-tabase 6183 links previous incorrect read results to cor-rect vehicle identification records. For example, when au-tomated vehicle identification fails but manual vehicleidentification succeeds, the captured vehicle identifica-tion data (e.g., the license plate read result) that led toan "error" (i.e., an identification failure) by the automatedsystem is stored in an error record in the read errorsdatabase 6183 that is linked to the vehicle identificationrecord that was manually identified for the vehicle. Thus,

29 30

EP 2 790 157 A1

17

5

10

15

20

25

30

35

40

45

50

55

when the same vehicle identification data is capturedagain at a later date, the computer 612 may successfullyidentify the vehicle automatically by accessing the errorrecord in the read errors database 6183, which identifiesthe correct vehicle identification record for the vehicle,without requiring another manual identification of the ve-hicle.[0135] An error record also may be generated andstored in the read errors database 6183 when automatedidentification of the vehicle succeeds based on a nearmatch of an incorrect license plate read result. For ex-ample, if the license plate number "ABC123" is read as"ABC128" and the identified candidate match set is"ABC128," "ABC123," "ABG128" and "ABC128" whichin turn yields the correct match of "ABC123," an errorrecord may be created that automatically links a licenseplate read result of "ABC128" to the vehicle having thelicense plate number "ABC123."[0136] The computer 612 determines whether any ve-hicle identification records correspond to the license plateread results for the target vehicle (block 922). If no vehicleidentification records correspond to the read results, thecomputer 612 performs an extended search (block 923).[0137] The computer 612 performs an extendedsearch by changing or loosening the criteria for a suc-cessful match or detuning the license plate read algo-rithm. For example, the computer 612 may perform anextended search by one or more of the following: (1) com-paring a subset of the license plate number read resultwith the characters of the license plate numbers storedin the vehicle record database 6182 (e.g., the last twocharacters of the license plate number may be omittedsuch that if the license plate number is "ABC123," anyvehicles having license plate numbers "ABC1**" aredeemed matching candidates, wherein "*" is a variable);(2) comparing a subset of the license plate number readresult in reverse order with the characters of the licenseplate numbers stored in the vehicle record database 6182in reverse order (e.g., the last two characters of the li-cense plate number in reverse order may be omitted suchthat if the license plate number is "ABC123", which is"321CBA" in reverse order, any vehicles having licenseplate numbers in reverse order of "321C**" are deemedmatching candidates, wherein "*" is a variable); and (3)other near match techniques including comparing mod-ified versions of the license plate read result and licenseplate numbers stored in the vehicle record database 6182in which some of either or both are substituted and/orremoved to reduce the impact of misread characters. Forexample, if the OCR algorithm does not indicate a con-fidence level above a predetermined threshold in a readresult of a character on the license plate, that charactermay be ignored. Additionally or alternatively, if the OCRalgorithm indicates that a character on the license platemay be one of two possible different characters, bothalternative characters may be used in the extendedsearch.[0138] The computer 612 determines whether any ve-

hicle identification records correspond to the read resultsfor the target vehicle after performing the extendedsearch (block 924). If no vehicle identification records arefound, process 900 proceeds to operation 941 of themanual identification process 940 (block 924).[0139] Referring to Fig. 9B, if either the search or theextended search lead to identification of one or more ve-hicle identification records, the computer 612 retrievesvehicle fingerprint and optionally other vehicle signa-ture/classification data from the identified vehicle identi-fication records (block 931). The computer 612 comparesthe retrieved vehicle fingerprint and optionally other ve-hicle signature/classification data for each matching ve-hicle candidate with the corresponding data associatedwith the target vehicle to identify one or more possiblematches(block 932). The vehicle fingerprint comparisonmay be performed with a comparison algorithm identicalor similar to the one developed by JAI-PULNiX Inc. ofSan Jose, California and described in U.S. Patent No.6,747,687.[0140] A possible match may be defined, for example,as a vehicle fingerprint match with a confidence scoregreater than or equal to a predefined threshold and all orsome of the other classification/signature data fallingwithin tolerances defined for each data type. For exam-ple, if the fingerprint matching algorithm generates ascore of 1 to 1000, where 1 is no match and 1000 is aperfect match, then a score greater than or equal to 900may be required for a successful match. Additionally, ifthe other classification/signature data includes target ve-hicle height, width, and length, then the height, width,and length of the vehicle candidate may be required tobe within plus or minus four inches of the extracted height,width, and length of the target vehicle for a successfulmatch. One or more vehicle identification records maybe deemed to correspond to vehicles that possibly matchthe target vehicle.[0141] The computer 612 determines whether a pos-sible match is sufficient to automatically identify the ve-hicle without human intervention by determining a com-bined equivalent matching score for each possible matchand comparing the result to a predetermined automatedconfidence threshold (block 933). The computer 612may, for example, determine a combined equivalentmatching score for each possible match in a manner sim-ilar to that described previously with respect to process700. Specifically, the computer 612 may assign a matchconfidence level number to the fingerprint matching and,optionally, to the classification/signature data matching,assign a weight to each data type, and calculate a com-bined equivalent matching score by combining theweighted match confidence level numbers. If the com-bined equivalent matching score exceeds a predeter-mined automated confidence threshold, the computer612 deems the target vehicle successfully identified andprocess 900 proceeds to operation 937 for recording thetransaction event between the identified vehicle and thefacility 628. If more than one possible match exceeds the

31 32

EP 2 790 157 A1

18

5

10

15

20

25

30

35

40

45

50

55

automated confidence threshold, the automated identifi-cation process may be faulty, and process 900 may op-tionally proceed (not shown) to operation 941 of the man-ual identification process 940.[0142] If no possible match is deemed sufficient to au-tomatically identify the vehicle without human interven-tion, the computer 612 determines whether one or morepossible matches satisfy a lower probable match thresh-old (block 934). The computer 612 may, for example,determine that a possible match satisfies the probablematch threshold if the combined equivalent matchingscore of the possible match is higher than the probablematch threshold but lower than the automated confi-dence threshold.[0143] If at least one possible match satisfies the prob-able match threshold, the computer 612 enables an op-erator to perform visual match truthing (block 935). Visualmatch truthing is a process in which the computer 612presents one or more of the images of the target vehicleto the operator along with one or more of the referenceimages associated with the vehicle or vehicles that prob-ably match the target vehicle. The operator quickly con-firms or rejects each probable match with a simple yesor no indication by, for example, selecting the appropriatebuttons on a user interface (block 936). The operator alsomay optionally provide a detailed explanation to supporthis or her response.[0144] If the match exceeds the automated confidencethreshold or is visually confirmed by the operator throughvisual match truthing, the computer 612 creates a recordof the event (i.e., a record of the interaction between thepositively identified target vehicle and the facility 628) as,for example, a billable or non-revenue transaction (block937). If the match was confirmed through visual matchtruthing, the computer 612 may optionally update theread errors database 6183 to include the extracted ve-hicle identification data and a link that associates the ex-tracted vehicle identification data with the correct vehicleidentification record (block 938).[0145] Referring also to Fig. 9C, the computer 612 isconfigured to enable an operator to manually identify thetarget vehicle (block 941) under the following circum-stances: (1) the captured images of the target vehicle donot provide any partial or complete read results for thelicense plate number and state of the target vehicle (block913); (2) no vehicle identification records are found thatcorrespond to the license plate read results for the targetvehicle after performing an extended search (block 924);(3) one or more possible matches are found but the con-fidence level in the one or more possible matches, asreflected by combined equivalent matching scores, fallbelow both the automated confidence threshold and theprobable match threshold (block 934); and (4) one ormore probable matches are found but a human operatorrejects the one or more probable matches through visualmatch truthing (block 936).[0146] The human operator attempts to manually iden-tify the vehicle by (1) reading the license plate(s), and (2)

observing vehicle details captured by the image acquisi-tion module 624, and (3) comparing the license plate dataand vehicle details with data available from the vehiclerecords database 6182, read errors database 6183,and/or databases of external systems 634. Licenseplates read by a human operator may be confirmed bycomparison with automated license plate reader resultsand/or multiple entry by multiple human operators.[0147] The manual identification may be deemed suc-cessful if the manually collected data, weighed againstdefinable criteria for a positive vehicle match, exceeds apredetermined identification confidence threshold (block942). This determination may be done by the computer612, the operator that provided the manual data, and/ora more qualified operator.[0148] In one implementation, if a vehicle cannot bepositively identified automatically and no near matchesare found, one or more images of the vehicle are dis-played to a first human reviewer. The first human review-er inspects the images and manually specifies the licenseplate number that the first reviewer believes correspondsto the vehicle based on the images. Because this manualreview by the first human reviewer is also subject to error(e.g., perceptual or typographical error), the license plateread by the first human reviewer is compared to an LPRdatabase to determine whether the license plate numberspecified by the first human reviewer exists. Additionally,if a database record having fingerprint data correspond-ing to the license plate read exists, a fingerprint compar-ison also may be performed. If the first human reviewerread result does not match any known LPR result or ve-hicle, the one or more images of the vehicle may be dis-played to a second human reviewer. The second humanreviewer inspects the images and manually specifies thelicense plate number that the second human reviewerbelieves corresponds to the vehicle based on the images.If the read result by the second human reviewer is differ-ent than the read result by the first human reviewer, aread by a third human reviewer, who is typically a morequalified reviewer, may be necessary. In sum, the firsthuman reviewer read is effectively a jumping off point tore-attempt an automated match. If the automated matchstill fails, multiple human reviewers must show agree-ment in reading the license plate for the read to bedeemed accurate.[0149] If the vehicle is not successfully identified, thecomputer 612 creates a record of the event as an uni-dentified or unassigned transaction (block 943). If thevehicle is successfully identified, the computer 612 cre-ates a record of the event as, for example, a billable ornon-revenue transaction (block 937). If the vehicle hadnever been previously identified, the computer 612 maycreate a new vehicle identification record for the vehicleand its owner/driver in the vehicle record database 6182.The computer 612 also may update the read errors da-tabase 6183 to include the extracted vehicle identificationdata and a link that associates the extracted vehicle iden-tification data with the correct vehicle identification record

33 34

EP 2 790 157 A1

19

5

10

15

20

25

30

35

40

45

50

55

(block 938).[0150] The above applications represent illustrativeexamples and the disclosed techniques disclosed canbe employed in other applications. Further, the variousaspects and disclosed techniques (including systemsand processes) can be modified, combined in whole orin part with each other, supplemented, or deleted to pro-duce additional implementations.[0151] The systems and techniques described herecan be implemented in digital electronic circuitry, or incomputer hardware, firmware, software, or in combina-tions of them. The systems and techniques describedhere can be implemented as a computer program prod-uct, i.e., a computer program tangibly embodied in aninformation carrier, e.g., in a machine-readable storagedevice or in a propagated signal, for execution by, or tocontrol the operation of, data processing apparatus, e.g.,a programmable processor, a computer, or multiple com-puters. A computer program can be written in any formof programming language, including compiled or inter-preted languages, and it can be deployed in any form,including as a stand-alone program or as a module, com-ponent, subroutine, or other unit suitable for use in a com-puting environment. A computer program can be de-ployed to be executed on one computer or on multiplecomputers at one site or distributed across multiple sitesand interconnected by a communication network.[0152] Method steps of the systems and techniquesdescribed here can be performed by one or more pro-grammable processors executing a computer programto perform functions of the invention by operating on inputdata and generating output. Method steps can also beperformed by, and apparatus of the invention can be im-plemented as, special purpose logic circuitry, e.g., anFPGA (field programmable gate array) or an ASIC (ap-plication-specific integrated circuit).[0153] Processors suitable for the execution of a com-puter program include, by way of example, both generaland special purpose microprocessors, and any one ormore processors of any kind of digital computer. Gener-ally, a processor will receive instructions and data froma read-only memory or a random access memory or both.The typical elements of a computer are a processor forexecuting instructions and one or more memory devicesfor storing instructions and data. Generally, a computerwill also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass stor-age devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. Information carriers suita-ble for embodying computer program instructions anddata include all forms of nonvolatile memory, includingby way of example semiconductor memory devices, e.g.,EPROM, EEPROM, and flash memory devices; magnet-ic disks such as internal hard disks and removable disks;magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supple-mented by, or incorporated in special purpose logic cir-cuitry.

[0154] To provide for interaction with a user, the sys-tems and techniques described here can be implementedon a computer having a display device such as a CRT(cathode ray tube) or LCD (liquid crystal display) monitorfor displaying information to the user and a keyboard anda pointing device such as a mouse or a trackball by whichthe user can provide input to the computer. Other kindsof devices can be used to provide for interaction with auser as well; for example, feedback provided to the usercan be any form of sensory feedback, such as visualfeedback, auditory feedback, or tactile feedback; and in-put from the user can be received in any form, includingacoustic, speech, or tactile input.[0155] The systems and techniques described herecan be implemented in a computing system that includesa back-end component, e.g., as a data server, or thatincludes a middleware component, e.g., an applicationserver, or that includes a front-end component, e.g., aclient computer having a graphical user interface or anWeb browser through which a user can interact with animplementation of the invention, or any combination ofsuch back-end, middleware, or front-end components.The components of the system can be interconnectedby any form or medium of digital data communication,e.g., a communication network. Examples of communi-cation networks include a local area network ("LAN"), awide area network ("WAN"), and the Internet.[0156] The computing system can include clients andservers. A client and server are generally remote fromeach other and typically interact through a communica-tion network. The relationship of client and server arisesby virtue of computer programs running on the respectivecomputers and having a client-server relationship to eachother.[0157] Examples are set out in the following list of num-bered clauses.

1. A method of identifying a vehicle in a toll system,the method comprising:

accessing image data for a first vehicle;obtaining license plate data from the accessedimage data for the first vehicle;accessing a set of records, each record includ-ing license plate data for a vehicle;comparing the license plate data for the first ve-hicle with the license plate data for vehicles inthe set of records;identifying a set of vehicles from the vehicleshaving records in the set of records, the set ofvehicles being identified based on results of thecomparison of the license plate data;accessing vehicle fingerprint data for the first ve-hicle, the vehicle fingerprint data for the first ve-hicle being based on the accessed image datafor the first vehicle;accessing vehicle fingerprint data for a vehiclein the set of vehicles;

35 36

EP 2 790 157 A1

20

5

10

15

20

25

30

35

40

45

50

55

comparing, using a processing device, the ve-hicle fingerprint data for the first vehicle with thevehicle fingerprint data for the vehicle in the setof vehicles; andidentifying the vehicle in the set of vehicles asthe first vehicle based on results of the compar-ison of vehicle fingerprint data.

2. The method of clause 1, wherein comparing li-cense plate data for the first vehicle with the licenseplate data for vehicles in the set of records includessearching a vehicle record database for records thatinclude license plate data that exactly match the li-cense plate data obtained for the first vehicle.

3. The method of clause 2, wherein comparing li-cense plate data for the first vehicle with the licenseplate data for vehicles in the set of records includesperforming an extended search of the vehicle recorddatabase for records that include license plate datathat nearly match the license plate data obtained forthe first vehicle, the extended search being condi-tioned on no vehicle identification records beingfound that include license plate data that exactlymatch the license plate data obtained for the firstvehicle.

4. The method of clause 2, wherein comparing thelicense plate data for the first vehicle with the licenseplate data for vehicles in the set of records includescomparing the license plate data using predeter-mined matching criteria.

5. The method of clause 4, further comprising chang-ing the predetermined matching criteria to increasethe number of vehicles in the identified set of vehi-cles.

6. The method of clause 5, wherein changing thepredetermined matching criteria to increase thenumber of vehicles in the identified set of vehicles isconditioned on a failure to identify any vehicles inthe set of vehicles as the first vehicle based on resultsof the comparison of vehicle fingerprint data.

7. The method of clause 1, further comprising ac-cessing laser signature data or inductive signaturedata for the first vehicle.

8. The method of clause 7, wherein the laser signa-ture data comprises data obtained by using a laserto scan the first vehicle.

9. The method of clause 7, wherein the laser signa-ture data includes one or more of an overhead elec-tronic profile of the first vehicle, an axle count of thefirst vehicle, and a 3D image of the first vehicle.

10. The method of clause 7, wherein the inductivesignature data comprises data obtained through useof a loop array over which the first vehicle passes.

11. The method of clause 7, wherein the inductivesignature data includes one or more of an axle countof the first vehicle, a type of engine of the first vehicle,and a vehicle type or class for the first vehicle.

12. The method of clause 7, wherein each record inthe set of records includes laser signature data orinductive signature data for a vehicle.

13. The method of clause 12, further comprisingcomparing laser signature data or inductive signa-ture data for the first vehicle with laser signature dataor inductive signature data for vehicles in the set ofrecords.

14. The method of clause 13, wherein identifying aset of vehicles from the vehicles having records inthe set of records includes identifying the set of ve-hicles based on the results of the comparison of thelicense plate data and the results of the comparisonof the laser signature data or the inductive signaturedata.

15. The method of clause 14, wherein identifying theset of vehicles based on the results of the compari-son of the license plate data and the results of thecomparison of the laser signature data or inductivesignature data includes determining a combinedequivalent matching score for each vehicle having arecord in the set of records and identifying the set ofvehicles as a set of vehicles having combined equiv-alent matching scores above a predeterminedthreshold.

16. The method of clause 15, wherein each com-bined equivalent matching score comprises aweighted combination of a laser or inductive signa-ture matching score and a license plate matchingscore.

17. The method of clause 13, wherein identifying thevehicle in the set of vehicles as the first vehicle in-cludes identifying the vehicle as the first vehiclebased on the results of the comparison of the vehiclefingerprint data and the results of the comparison ofthe laser signature data or inductive signature data.

18. The method of clause 17, wherein identifying thevehicle in the set of vehicles as the first vehicle basedon the results of the comparison of the vehicle fin-gerprint data and the results of the comparison ofthe laser signature data or inductive signature dataincludes determining a combined equivalent match-ing score for the vehicle in the set of vehicles and

37 38

EP 2 790 157 A1

21

5

10

15

20

25

30

35

40

45

50

55

determining that the combined equivalent matchingscore is higher than a predetermined threshold.

19. The method of clause 18, wherein the combinedequivalent matching score comprises a weightedcombination of a laser or inductive signature match-ing score and a vehicle fingerprint matching score.

20. The method of clause 1, wherein identifying thevehicle in the set of vehicles as the first vehicle in-cludes identifying the vehicle as the first vehicle ifthe comparison of the vehicle fingerprint data for thefirst vehicle with the vehicle fingerprint data for thevehicle in the set of vehicles indicates a match havinga confidence level that exceeds a confidence thresh-old.

21. The method of clause 20, wherein identifying thevehicle in the set of vehicles as the first vehicle in-cludes identifying the vehicle in the set of vehiclesas the first vehicle without human intervention if theconfidence level of the match exceeds a first confi-dence threshold.

22. The method of clause 21, wherein identifying thevehicle in the set of vehicles as the first vehicle in-cludes identifying the vehicle in the set of vehiclesas the first vehicle if the confidence level of the matchis less than the first confidence threshold but greaterthan a second confidence threshold and a humanoperator confirms the match.

23. The method of clause 22, further comprising en-abling the human operator to confirm or reject thematch by:

enabling the human operator to perceive the ac-cessed image data for the first vehicle, andenabling the human operator to interact with auser interface to indicate rejection or confirma-tion of the match.

24. The method of clause 22, wherein identifying thevehicle in the set of vehicles as the first vehicle in-cludes identifying the vehicle as the first vehicle ifthe confidence level of the match is less than the firstand second confidence thresholds and a human op-erator manually identifies the vehicle as the first ve-hicle by accessing the image data for the first vehicleand the record for the vehicle in the set of records.

25. The method of clause 24, further comprising en-abling the human operator to manually identify thevehicle in the set of vehicles as the first vehicle by:

enabling the human operator to access the im-age data for the first vehicle,enabling the human operator to access the

record for the vehicle in the set of records,andenabling the human operator to interact with auser interface to indicate positive identificationof the first vehicle as the vehicle in the set ofvehicles.

26. The method of clause 25, further comprising en-abling the human operator to manually identify thevehicle in the set of vehicles as the first vehicle byenabling the human operator to access data storedin databases of external systems.

27. The method of clause 1, wherein identifying thevehicle in the set of vehicles as the first vehicle in-cludes identifying the vehicle by combining vehicleidentification number (VIN), laser signature, induc-tive signature, and image data.

28. An article comprising a machine-readable medi-um storing machine-executable instructions that,when applied to a machine, cause the machine toperform operations comprising:

accessing image data for a first vehicle;obtaining license plate data from the accessedimage data for the first vehicle;accessing a set of records, each record includ-ing license plate data for a vehicle;comparing the license plate data for the first ve-hicle with the license plate data for vehicles inthe set of records;identifying a set of vehicles from the vehicleshaving records in the set of records, the set ofvehicles being identified based on results of thecomparison of the license plate data;accessing vehicle fingerprint data for the first ve-hicle, the vehicle fingerprint data for the first ve-hicle being based on the accessed image datafor the first vehicle;accessing vehicle fingerprint data for a vehiclein the set of vehicles;comparing the vehicle fingerprint data for thefirst vehicle with the vehicle fingerprint data forthe vehicle in the set of vehicles; andidentifying the vehicle in the set of vehicles asthe first vehicle based on results of the compar-ison of vehicle fingerprint data.

29. An apparatus for identifying a vehicle in a tollsystem, the apparatus comprising:

an image capture device configured to captureimage data for a first vehicle; andone or more processing devices communica-tively coupled to each other and to the imagecapture device and configured to:obtain license plate data from the captured im-age data for the first vehicle;

39 40

EP 2 790 157 A1

22

5

10

15

20

25

30

35

40

45

50

55

access a set of records, each record includinglicense plate data for a vehicle;compare the license plate data for the first ve-hicle with the license plate data for vehicles inthe set of records;identify a set of vehicles from the vehicles havingrecords in the set of records, the set of vehiclesbeing identified based on results of the compar-ison of the license plate data;access vehicle fingerprint data for the first vehi-cle, the vehicle fingerprint data for the first vehi-cle being based on the captured image data forthe first vehicle;access vehicle fingerprint data for a vehicle inthe set of vehicles;compare the vehicle fingerprint data for the firstvehicle with the vehicle fingerprint data for thevehicle in the set of vehicles; andidentify the vehicle in the set of vehicles as thefirst vehicle based on results of the comparisonof vehicle fingerprint data.

30. Computing apparatus programmed and opera-ble to carry out a method according to any one ofclause 1 to clause 27.

31. A computer program including code portions ex-ecutable by computing apparatus to cause the com-puting apparatus to carry out a method according toany one of clause 1 to clause 27.

32. An information carrier carrying information indic-ative of computer code portions that are executableby computing apparatus to cause the computing ap-paratus to carry out a method according to any oneof clause 1 to clause 27.

33. An information carrier according to clause 32,wherein the information carrier is, for example, anelectrical signal, a wireless radio-frequency signal,or a recording medium such as, for example, an op-tical recording medium, a magnetic recording medi-um or solid-state memory.

34. A computer-implemented method of identifyinga vehicle in a toll system, the method comprising:

acquiring image data for a first vehicle;obtaining license plate data from the acquiredimage data for the first vehicle;accessing a set of records that include licenseplate data for vehicles;executing a detuned license plate reading algo-rithm to:

compare the license plate data for the firstvehicle with the license plate data for vehi-cles in the set of records, and

identify a set of vehicle candidates from thevehicles having records in the set ofrecords, the set of vehicle candidates beingidentified based on results of the compari-son of the license plate data, wherein thedetuned license plate reading algorithm in-cludes loosened license plate matching cri-teria or a lowered license plate read confi-dence threshold to enable generation of alarger set of matching vehicle candidatesrelative to a license plate reading algorithmdesigned to identify a single and best vehi-cle candidate match; andselecting, from the set of vehicle candi-dates, a vehicle candidate as correspond-ing to the first vehicle by:

accessing second vehicle identifier data for thefirst vehicle, the second vehicle identifier databeing data for identifying a vehicle that is distinctfrom license plate data;accessing second vehicle identifier data for avehicle candidate in the set of vehicle candi-dates,comparing, using the at least one processingdevice, the second vehicle identifier data for thefirst vehicle with the second vehicle identifier da-ta for the vehicle candidate in the set of vehiclecandidates, andidentifying the vehicle candidate in the set of ve-hicle candidates as the first vehicle based onresults of the comparison of second vehicleidentifier data.

35. The method of clause 34, wherein identifying thevehicle candidate in the set of vehicle candidates asthe first vehicle includes identifying the vehicle can-didate as the first vehicle if the comparison of thesecond vehicle identifier data for the first vehicle withthe second vehicle identifier data for the vehicle can-didate in the set of vehicle candidates indicates amatch having a confidence level that exceeds a con-fidence threshold.

36. The method of clause 35, wherein identifying thevehicle candidate in the set of vehicle candidates asthe first vehicle includes identifying the vehicle can-didate in the set of vehicle candidates as the firstvehicle without human intervention if the confidencelevel of the match exceeds a first confidence thresh-old.

37. The method of clause 36, wherein identifying thevehicle candidate in the set of vehicle candidates asthe first vehicle includes identifying the vehicle can-didate in the set of vehicle candidates as the firstvehicle if the confidence level of the match is lessthan the first confidence threshold but greater than

41 42

EP 2 790 157 A1

23

5

10

15

20

25

30

35

40

45

50

55

a second confidence threshold and a human oper-ator confirms the match.

38. The method of clause 37, further comprising en-abling the human operator to confirm or reject thematch by:

enabling the human operator to perceive the ac-quired image data for the first vehicle, andenabling the human operator to interact with auser interface to indicate rejection or confirma-tion of the match.

39. The method of clause 37, wherein identifying thevehicle candidate in the set of vehicle candidates asthe first vehicle includes identifying the vehicle can-didate as the first vehicle if the confidence level ofthe match is less than the first and second confidencethresholds and a human operator manually identifiesthe vehicle candidate as the first vehicle by access-ing the image data for the first vehicle and the recordfor the vehicle in the set of records.

40. The method of clause 34, wherein identifying thevehicle candidate in the set of vehicle candidates asthe first vehicle includes identifying the vehicle can-didate based on vehicle identification number (VIN),laser signature, inductive signature, and image data.

41. The method of clause 34, wherein identifying aset of vehicle candidates based on the results of thecomparison of the license plate data comprises iden-tifying multiple vehicle candidates as correspondingto the first vehicle based on the results of the com-parison of the license plate data.

42. The method of clause 34, wherein the licenseplate reading algorithm comprises an algorithm thatreads a license plate number of a target vehicle froman image of the target vehicle and compares the li-cense plate number read from the image to knownlicense plate numbers of vehicles to identify a set ofmatching vehicle candidates for the target vehicle.

43. The method of clause 34, wherein obtaining li-cense plate data from the acquired image data forthe first vehicle comprises obtaining license plate da-ta from the acquired image data using optical char-acter recognition.

44. The method of clause 34, wherein the licenseplate data includes a license plate number.

45. The method of clause 34, wherein at least one of:

the second vehicle identifier data comprises la-ser signature data or inductive signature datafor the first vehicle;

the laser signature data includes one or more ofan overhead electronic profile of the first vehicle,an axle count of the first vehicle, and a 3D imageof the first vehicle; andthe inductive signature data includes one ormore of an axle count of the first vehicle, a typeof engine of the first vehicle, and a vehicle typeor class for the first vehicle.

46. The method of clause 45, wherein the records inthe set of records include laser signature data or in-ductive signature data for vehicles.

47. The method of clause 34, wherein the secondvehicle identifier data comprises vehicle fingerprintdata for the first vehicle, the vehicle fingerprint datafor the first vehicle being based on the acquired im-age data for the first vehicle and the vehicle finger-print data for the first vehicle being a set of data ar-tifacts corresponding to a visual signature of the firstvehicle, optionally wherein the vehicle fingerprint da-ta for the first vehicle is unique to the first vehicle.

48. An apparatus for identifying a vehicle in a tollsystem, the apparatus comprising:

an image capture device configured to captureimage data for a first vehicle; andone or more processing devices communica-tively coupled to each other and to the imagecapture device and configured to:

access a set of records that include licenseplate data for vehicles;acquire image data for the first vehicle;obtain license plate data from the acquiredimage data for the first vehicle; andexecute a detuned license plate reading al-gorithm to:

compare the license plate data for thefirst vehicle with the license plate datafor vehicles in the set of records, andidentify a set of vehicle candidates fromthe vehicles having records in the setof records, the set of vehicle candidatesbeing identified based on results of thecomparison of the license plate data,wherein the detuned license plate read-ing algorithm includes loosened licenseplate matching criteria or a lowered li-cense plate read confidence thresholdto enable generation of a larger set ofmatching vehicle candidates relative toa license plate reading algorithm de-signed to identify a single and best ve-hicle candidate match; andselect, from the set of vehicle candi-

43 44

EP 2 790 157 A1

24

5

10

15

20

25

30

35

40

45

50

55

dates, a vehicle candidate as corre-sponding to the first vehicle by:accessing second vehicle identifier da-ta for the first vehicle, the second vehi-cle identifier data being data for identi-fying a vehicle that is distinct from li-cense plate data;accessing second vehicle identifier da-ta for a vehicle candidate in the set ofvehicle candidates,comparing, using the at least oneprocessing device, the second vehicleidentifier data for the first vehicle withthe second vehicle identifier data for thevehicle candidate in the set of vehiclecandidates, andidentifying the vehicle candidate in theset of vehicle candidates as the first ve-hicle based on results of the compari-son of second vehicle identifier data.

[0158] Other implementations are within the scope ofthe following claims.

Claims

1. A computer-implemented method of identifying a ve-hicle in a toll system, the method comprising:

accessing image data for a vehicle transactingwith a toll system;obtaining first vehicle identifier data from the ac-cessed image data for the transacting vehicle;accessing a set of records that includes first ve-hicle identifier data for vehicles;executing, using at least one processing device,an algorithm to:

compare the first vehicle identifier data forthe transacting vehicle with the first vehicleidentifier data for vehicles in the set ofrecords, andidentify a set of matching vehicle candidatesfrom the vehicles having records in the setof records; and

selecting, from the set of matching vehicle can-didates, a vehicle candidate as correspondingto the transacting vehicle by:

accessing second vehicle identifier data forthe transacting vehicle, the second vehicleidentifier data being data for identifying avehicle that is distinct from the first vehicleidentifier data,accessing second vehicle identifier data foreach vehicle candidate in the set of match-

ing vehicle candidates,comparing the second vehicle identifier da-ta for the transacting vehicle with the secondvehicle identifer data for each vehicle can-didate in the set of matching vehicle candi-dates, andidentifying the vehicle candidate in the setof matching vehicle candidates as the trans-acting vehicle based on results of the com-parison of second vehicle identifier data.

2. The method of claim 1, wherein the first vehicle iden-tifier data comprises license plate data.

3. The method of claim 2,wherein the algorithm comprises a loosened licenseplate reading algorithm, andwherein the loosened license plate reading algorithmincludes loosened license plate matching criteria ora lowered license plate read confidence threshold toenable generation of a larger set of matching vehiclecandidates relative to a license plate reading algo-rithm designed to identify a single and best vehiclecandidate match, optionally wherein at least one of:

the license plate reading algorithm comprisesan algorithm that reads a license plate numberof a target vehicle from an image of the targetvehicle and compares the license plate numberread from the image to known license plate num-bers of vehicles to identify a set of matching ve-hicle candidates for the target vehicle;the license plate reading algorithm has been in-tentionally modified to generate a larger set ofmatching vehicle candidates by lowering theread confidence threshold used by the algorithmto determine whether a read result is designatedas corresponding to a matching vehicle candi-date; andthe license plate reading algorithm has been in-tentionally modified to generate a larger set ofmatching vehicle candidates by requiring a li-cense plate number of a candidate vehicle tomatch only a subset or a lesser number of char-acters in the license plate number of a targetvehicle for the candidate vehicle to be includedin the set of matching vehicle candidates.

4. The method of claim 1, wherein the second vehicleidentifier comprises vehicle fingerprint data for thetransacting vehicle, the vehicle fingerprint data forthe transacting vehicle being based on the accessedimage data for the transacting vehicle and the vehiclefingerprint data for the transacting vehicle being aset of data artifacts corresponding to a visual signa-ture that is deemed unique to the transacting vehicle.

5. The method of claim 1, wherein comparing the first

45 46

EP 2 790 157 A1

25

5

10

15

20

25

30

35

40

45

50

55

vehicle identifier data for the first vehicle with the firstvehicle identifier data for vehicles in the set of recordsincludes searching a vehicle record database forrecords that include first vehicle identifier data thatexactly match the first vehicle identifier data obtainedfor the transacting vehicle.

6. The method of claim 5, wherein comparing the firstvehicle identifier data for the transacting vehicle withthe first vehicle identifier data for vehicles in the setof records includes performing an extended searchof the vehicle record database for records that in-clude first vehicle identifier data that nearly matchthe first vehicle identifier data obtained for the trans-acting vehicle, the extended search being condi-tioned on no vehicle identification records beingfound that include first vehicle identifier data that ex-actly match the first vehicle identifier data obtainedfor the transacting vehicle.

7. The method of claim 5,wherein comparing the first vehicle identifier data forthe transacting vehicle with the first vehicle identifierdata for vehicles in the set of records includes com-paring the first vehicle identifer data using predeter-mined matching criteria, andfurther comprising changing the predeterminedmatching criteria to increase the number of vehiclesin the identified set of vehicles, optionallywherein changing the predetermined matching cri-teria to increase the number of vehicles in the iden-tified set of vehicles is conditioned on a failure toidentify any vehicles in the set of vehicles as thetransacting vehicle based on results of the compar-ison of second vehicle identifier data.

8. The method of claim 1,further comprising accessing laser signature data,wherein the laser signature data comprises data ob-tained by using a laser to scan the transacting vehi-cle, optionally wherein at least one of:

the laser signature data includes one or more ofan overhead electronic profile of the transactingvehicle, an axle count of the transacting vehicle,and a 3D image of the transacting vehicle;andthe method further comprises comparing the la-ser signature data for the transacting vehiclewith laser signature data for vehicles in the setof records, andwherein identifying a set of vehicles from the ve-hicles having records in the set of records in-cludes identifying the set of vehicles based onthe results of the comparison of the first vehicleidentifier data and the results of the comparisonof the laser signature data.

9. The method of claim 1,further comprising accessing inductive signature da-ta, wherein the inductive signature data comprisesdata obtained through use of a loop array over whichthe transacting vehicle passes, optionally wherein atleast one of:

the inductive signature data includes one ormore of an axle count of the transacting vehicle,a type of engine of the transacting vehicle, anda vehicle type or class for the transacting vehi-cle;identifying the vehicle candidate in the set ofmatching vehicle candidates as the transactingvehicle includes identifying the vehicle candi-date based on vehicle identification number(VIN), laser signature, inductive signature, andimage data; andthe method further comprises comparing the in-ductive signature data for the transacting vehiclewith inductive signature data for vehicles in theset of records, andwherein identifying a set of vehicles from the ve-hicles having records in the set of records in-cludes identifying the set of vehicles based onthe results of the comparison of the first vehicleidentifier data and the results of the comparisonof the inductive signature data.

10. The method of claim 1, wherein identifying the vehi-cle candidate in the set of matching vehicle candi-dates as the transacting vehicle includes identifyingthe vehicle candidate as the transacting vehicle ifthe comparison of the second vehicle identifier datafor the transacting vehicle with the second vehicleidentifier data for the vehicle candidate in the set ofmatching vehicle candidates indicates a match hav-ing a confidence level that exceeds a confidencethreshold.

11. The method of claim 10, wherein identifying the ve-hicle candidate in the set of matching vehicle candi-dates as the transacting vehicle includes identifyingthe vehicle candidate in the set of matching vehiclecandidates as the transacting vehicle without humanintervention if the confidence level of the match ex-ceeds a first confidence threshold, optionallywherein identifying the vehicle candidate in the setof matching vehicle candidates as the transactingvehicle includes identifying the vehicle candidate inthe set of matching vehicle candidates as the trans-acting vehicle if the confidence level of the match isless than the first confidence threshold but greaterthan a second confidence threshold and a humanoperator confirms the match, further optionallywherein at least one of:

identifying the vehicle candidate in the set of

47 48

EP 2 790 157 A1

26

5

10

15

20

25

30

35

40

45

50

55

matching vehicle candidates as the transactingvehicle includes identifying the vehicle candi-date as the transacting vehicle if the confidencelevel of the match is less than the first and sec-ond confidence thresholds and a human oper-ator manually identifies the vehicle candidate asthe transacting vehicle by accessing the imagedata for the transacting vehicle and the recordfor the vehicle candidate in the set of records;andthe method further comprises enabling the hu-man operator to confirm or reject the match by:

enabling the human operator to perceivethe accessed image data for the transactingvehicle,enabling the human operator to perceiveone or more reference images associatedwith the vehicle candidate, andenabling the human operator to interact witha user interface to indicate rejection or con-firmation of the match.

12. The method of claim 1, wherein identifying the vehi-cle candidate in the set of matching vehicle candi-dates as the transacting vehicle includes identifyingthe vehicle candidate based on vehicle identificationnumber (VIN), laser signature, inductive signature,and image data.

13. The method of claim 2, wherein identifying a set ofmatching vehicle candidates based on the results ofthe comparison of the license plate data comprisesidentifying multiple vehicle candidates as corre-spond ing to the transacting vehicle based on theresults of the comparison of the first vehicle identifierdata.

14. An apparatus for identifying a vehicle in a toll system,the apparatus comprising:

an image capture device configured to captureimage data for a vehicle transacting with a tollsystem; andone or more processing devices communica-tively coupled to each other and to the imagecapture device and configured to:

access the image data for the transactingveh icle;obtain first vehicle identifier data from theaccessed image data for the transacting ve-hicle;access a set of records that includes firstvehicle identifier data for vehicles;execute, using at least one processing de-vice, an algorithm to:

compare the first vehicle identifier datafor the transacting vehicle with the firstvehicle identifier data for vehicles in theset of records, andidentify a set of matching vehicle can-didates from the vehicles havingrecords in the set of records; and

select, from the set of matching vehicle can-didates, a vehicle candidate as correspond-ing to the transacting vehicle by:

accessing second vehicle identifier da-ta for the transacting vehicle, the sec-ond vehicle identifier data being datafor identifying a vehicle that is distinctfrom the first vehicle identifier data,accessing second vehicle identifier da-ta for each vehicle candidate in the setof matching vehicle candidates,comparing, using the at least oneprocessing device, the second vehicleidentifier data for the transacting vehi-cle with the second vehicle identifier da-ta for each vehicle candidate in the setof matching vehicle candidates, andidentifying the vehicle candidate in theset of matching vehicle candidates asthe transacting vehicle based on resultsof the comparison of second vehicleidentifier data.

15. An article comprising a machine-readable mediumstoring machine-executable instructions that, whenapplied to a machine, cause the machine to performthe method of any of claims 1 to 13.

49 50

EP 2 790 157 A1

27

EP 2 790 157 A1

28

EP 2 790 157 A1

29

EP 2 790 157 A1

30

EP 2 790 157 A1

31

EP 2 790 157 A1

32

EP 2 790 157 A1

33

EP 2 790 157 A1

34

EP 2 790 157 A1

35

EP 2 790 157 A1

36

EP 2 790 157 A1

37

EP 2 790 157 A1

38

5

10

15

20

25

30

35

40

45

50

55

EP 2 790 157 A1

39

5

10

15

20

25

30

35

40

45

50

55

EP 2 790 157 A1

40

5

10

15

20

25

30

35

40

45

50

55

EP 2 790 157 A1

41

REFERENCES CITED IN THE DESCRIPTION

This list of references cited by the applicant is for the reader’s convenience only. It does not form part of the Europeanpatent document. Even though great care has been taken in compiling the references, errors or omissions cannot beexcluded and the EPO disclaims all liability in this regard.

Patent documents cited in the description

• US 60689050 B [0001] • US 6747687 B [0105] [0127] [0139]