Example: Data Mining for the NBA

download Example: Data Mining for the NBA

of 28

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Example: Data Mining for the NBA

  • 1.Data Mining,Security and Privacy Prof.Bhavani Thuraisingham Prof. Murat Kantarcioglu Ms Li Liu (PhD Student completing December 2007) The University of Texas at Dallas August 24, 2008

2. Outline

  • Data Mining for Security Applications
  • Privacy Concerns
  • What is Privacy?
  • Why is data mining a threat to privacy
  • Developments in Privacy
  • Directions for Privacy
  • Confidentiality, Privacy and Trust for Data Mining

3. Data Mining Needs for Counterterrorism:Non-real-time Data Mining

  • Gather data from multiple sources
    • Information on terrorist attacks: who, what, where, when, how
    • Personal and business data: place of birth, ethnic origin, religion, education, work history, finances, criminal record, relatives, friends and associates, travel history, . . .
    • Unstructured data: newspaper articles, video clips, speeches, emails, phone records, . . .
  • Integrate the data, build warehouses and federations
  • Develop profiles of terrorists, activities/threats
  • Mine the data to extract patterns of potential terrorists and predict future activities and targets
  • Find the needle in the haystack - suspicious needles?
  • Data integrity is important
  • Techniques have to SCALE

4. Data Mining Needs for Counterterrorism:Real-time Data Mining

  • Nature of data
    • Data arriving from sensors and other devices
      • Continuous data streams
    • Breaking news, video releases, satellite images
    • Some critical data may also reside in caches
  • Rapidly sift through the data and discard unwanted data for later use and analysis (non-real-time data mining)
  • Data mining techniques need to meet timing constraints
  • Quality of service (QoS) tradeoffs among timeliness, precision and accuracy
  • Presentation of results, visualization, real-time alerts and triggers

5. Data Mining for Real-time Threats Integrate data sources inreal - time Build real - time models Examine Results inReal - time Report final results Data sources with information about terrorists and terrorist activities Mine the data Rapidly sift through data anddiscard irrelevantdata 6. What should be done: Form a Research Agenda

  • Immediate action (0 - 1year)
    • Weve got to know what our current capabilities are
    • Do the commercial tools scale?Do they work only on special data and limited cases? Do they deliver what they promise?
    • Need an unbiased objective study with demonstrations
  • At the same time, work on the big picture
    • What do we want? What are our end results for the foreseeable future? What are the criteria for success? How do we evaluate the data mining algorithms? What testbeds do we build?
  • Near-term (1 - 3years)
    • Leverage current efforts
    • Fill the gaps in a goal-directed way; technology transfer
  • Long-term (3 - 5years and beyond)
    • 5-yearR&D plan for data mining for counterterrorism


  • Data Mining is very useful to solve Security Problems
    • Data mining tools could be used to examine audit data and flag abnormal behavior
    • Much recent work in Intrusion detection (unit #18)
      • e.g., Neural networks to detect abnormal patterns
    • Tools are being examined to determine abnormal patterns for national security
      • Classification techniques, Link analysis
    • Fraud detection
      • Credit cards, calling cards, identity theft etc.

8. What is Privacy

  • Medical Community
    • Privacy is about a patient determining what information the doctor should release about him/her
  • Financial community
    • A bank customer determines what financial information the bank should release about him/her
  • Government community
    • FBI would collect information about US citizens. However FBI determines what information about a US citizen it can release to say the CIA

9. Some Privacy concerns

  • Medical and Healthcare
    • Employers, marketers, or others knowing of private medical concerns
  • Security
    • Allowing access to individuals travel and spending data
    • Allowing access to web surfing behavior
  • Marketing, Sales, and Finance
    • Allowing access to individuals purchases

10. Data Mining as a Threat to Privacy

  • Data mining gives us facts that are not obvious to human analysts of the data
  • Can general trends across individuals be determined without revealing information about individuals?
  • Possible threats:
    • Combine collections of data and infer information that is private
      • Disease information from prescription data
      • Military Action from Pizza delivery to pentagon
  • Need to protect the associations and correlations between the data that are sensitive or private

11. Some Privacy Problems and Potential Solutions

  • Problem: Privacy violations that result due to data mining
    • Potential solution: Privacy-preserving data mining
  • Problem: Privacy violations that result due to the Inference problem
    • Inference is the process of deducing sensitive information from the legitimate responses received to user queries
    • Potential solution: Privacy Constraint Processing
  • Problem: Privacy violations due to un-encrypted data
    • Potential solution: Encryption at different levels
  • Problem: Privacy violation due to poor system design
    • Potential solution: Develop methodology for designing privacy-enhanced systems

12. Privacy as Inference: Privacy Constraint Processing

  • Privacy constraint/policy processing
    • Based on prior research in security constraint processing
    • Simple Constraint: an attribute of a document is private
    • Content-based constraint: If document contains information about X, then it is private
    • Association-based Constraint: Two or more documents taken together is private; individually each document is public
    • Release constraint: After X is released Y becomes private
  • Augment a database system with a privacy controller for constraint processing

13. Architecture for PrivacyConstraint Processing User Interface Manager Constraint Manager Privacy Constraints Query Processor: Constraints during query and release operations Update Processor: Constraints during update operation Database Design Tool Constraints during database design operation Database DBMS 14. Semantic Model for Privacy Control Patient John Cancer Influenza Has disease Travels frequently England address Johnsaddress Dark lines/boxes contain private information 15. Privacy Preserving Data Mining

  • Prevent useful results from mining
    • Introduce cover stories to give false results
    • Only make a sample of data available so that an adversary is unable to come up with useful rules and predictive functions
  • Randomization
    • Introduce random values into the data and/or results
    • Challenge is to introduce random values without significantly affecting the data mining results
    • Give range of values for results instead of exact values
  • Secure Multi-party Computation
    • Each party knows its own inputs; encryption techniques used to compute final results
    • Rules, predictive functions
  • Approach:Only make asampleof data available
    • Limits ability to learn good classifier

16. Cryptographic Approaches forPrivacy Preserving Data Mining

  • Secure Multi-part Computation (SMC) for PPDM
    • Mainly used for distributed data mining.; Provably secure under some assumptions.; Learned models are accurate; Mainly semi-honest