Image Classification pipeline Lecture 2 - Stanford .Fei-Fei Li & Andrej Karpathy & Justin...

download Image Classification pipeline Lecture 2 - Stanford .Fei-Fei Li & Andrej Karpathy & Justin Johnson

of 57

  • date post

    13-Jun-2018
  • Category

    Documents

  • view

    213
  • download

    0

Embed Size (px)

Transcript of Image Classification pipeline Lecture 2 - Stanford .Fei-Fei Li & Andrej Karpathy & Justin...

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20161

    Lecture 2:Image Classification pipeline

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20162

    Administrative

    First assignment will come out tonight (or tomorrow at worst)It is due January 20 (i.e. in two weeks). Handed in through CourseWorkIt includes:

    - Write/train/evaluate a kNN classifier- Write/train/evaluate a Linear Classifier (SVM and Softmax)- Write/train/evaluate a 2-layer Neural Network (backpropagation!)- Requires writing numpy/Python code

    Warning: dont work on assignments from last year!

    Compute: Can use your own laptops, or Terminal.com

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20163

    http://cs231n.github.io/python-numpy-tutorial/

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20164

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20165

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20166

    Image Classification: a core task in Computer Vision

    cat

    (assume given set of discrete labels){dog, cat, truck, plane, ...}

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20167

    The problem:semantic gap

    Images are represented as 3D arrays of numbers, with integers between [0, 255].

    E.g. 300 x 100 x 3

    (3 for 3 color channels RGB)

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20168

    Challenges: Viewpoint Variation

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 20169

    Challenges: Illumination

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201610

    Challenges: Deformation

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201611

    Challenges: Occlusion

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201612

    Challenges: Background clutter

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201613

    Challenges: Intraclass variation

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201614

    An image classifier

    Unlike e.g. sorting a list of numbers, no obvious way to hard-code the algorithm for recognizing a cat, or other classes.

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201615

    Attempts have been made???

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201616

    Data-driven approach:1. Collect a dataset of images and labels2. Use Machine Learning to train an image classifier3. Evaluate the classifier on a withheld set of test images

    Example training set

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201617

    First classifier: Nearest Neighbor Classifier

    Remember all training images and their labels

    Predict the label of the most similar training image

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201618

    Example dataset: CIFAR-1010 labels 50,000 training images, each image is tiny: 32x3210,000 test images.

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201619

    For every test image (first column), examples of nearest neighbors in rows

    Example dataset: CIFAR-1010 labels 50,000 training images10,000 test images.

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201620

    How do we compare the images? What is the distance metric?

    L1 distance:

    add

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201621

    Nearest Neighbor classifier

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201622

    Nearest Neighbor classifier

    remember the training data

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201623

    Nearest Neighbor classifier

    for every test image:- find nearest train image

    with L1 distance- predict the label of

    nearest training image

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201624

    Nearest Neighbor classifier

    Q: how does the classification speed depend on the size of the training data?

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201625

    Nearest Neighbor classifier

    Q: how does the classification speed depend on the size of the training data? linearly :(

    This is backwards:- test time performance is usually much more important in practice. - CNNs flip this: expensive training, cheap test evaluation

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201626

    Aside: Approximate Nearest Neighborfind approximate nearest neighbors quickly

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201627

    The choice of distance is a hyperparametercommon choices:

    L1 (Manhattan) distance L2 (Euclidean) distance

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201628

    k-Nearest Neighborfind the k nearest images, have them vote on the label

    the data NN classifier 5-NN classifier

    http://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201629

    For every test image (first column), examples of nearest neighbors in rows

    Example dataset: CIFAR-1010 labels 50,000 training images10,000 test images.

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201630

    Q: what is the accuracy of the nearest neighbor classifier on the training data, when using the Euclidean distance?

    the data NN classifier 5-NN classifier

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201631

    Q2: what is the accuracy of the k-nearest neighbor classifier on the training data?

    the data NN classifier 5-NN classifier

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201632

    What is the best distance to use?What is the best value of k to use?

    i.e. how do we set the hyperparameters?

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201633

    What is the best distance to use?What is the best value of k to use?

    i.e. how do we set the hyperparameters?

    Very problem-dependent. Must try them all out and see what works best.

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201634

    Try out what hyperparameters work best on test set.

  • Lecture 2 - 6 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 2 - 6 Jan 201635

    Trying out what hyperparameters work best on test set:Very bad idea. The test set is a proxy for the generalization performance!Use only VERY SPARINGLY, a