"
]
},
"metadata": {},
@@ -645,267 +804,18 @@
}
],
"source": [
- "print(\"Actual class of test image:\", test_lbl[test_img_choice])\n",
- "plt.imshow(test_img[test_img_choice].reshape((28,28)))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": true
- },
- "source": [
- "Hurray! We've got it correct. Don't worry if our algorithm predicted a wrong class. With this techinique we have only ~97% accuracy on this dataset. Let's try with a different test image and hope we get it this time.\n",
- "\n",
- "You might have recognized that our algorithm took ~20 seconds to predict a single image. How would we even predict all 10,000 test images? Yeah, the implementations we have in our learning module are not optimized to run on this particular dataset. We will have an optimised version below in NumPy which is nearly ~50-100 times faster than our native implementation."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Faster implementation using NumPy\n",
- "\n",
- "Here we calculate manhattan distance between two images faster than our native implementation. Which in turn make predicting labels for test images far efficient."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 18,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "class kNN_learner:\n",
- " \"Simple kNN learner with manhattan distance\"\n",
- " def __init__(self):\n",
- " pass\n",
- " \n",
- " def train(self, train_img, train_lbl):\n",
- " self.train_img = train_img\n",
- " self.train_lbl = train_lbl\n",
- "\n",
- " def predict_labels(self, test_img, k=1, distance=\"manhattan\"):\n",
- " if distance == \"manhattan\": \n",
- " distances = self.compute_manhattan_distances(test_img)\n",
- " num_test = distances.shape[0]\n",
- " predictions = np.zeros(num_test, dtype=np.uint8)\n",
- " \n",
- " for i in range(num_test):\n",
- " k_best_labels = self.train_lbl[np.argsort(distances[i])].flatten()[:k]\n",
- " predictions[i] = mode(k_best_labels)\n",
- " \n",
- " return predictions\n",
- " \n",
- " def compute_manhattan_distances(self, test_img):\n",
- " num_test = test_img.shape[0]\n",
- " num_train = self.train_img.shape[0]\n",
- "# print(num_test, num_train)\n",
- " \n",
- " dists = np.zeros((num_test, num_train))\n",
- " \n",
- " for i in range(num_test):\n",
- " dists[i] = np.sum(abs(self.train_img - test_img[i]), axis = 1)\n",
- " \n",
- " return(dists)\n",
- " "
+ "print(\"Actual class of test image:\", test_lbl[211])\n",
+ "plt.imshow(test_img[211].reshape((28,28)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Let's print the shapes of data to make sure everything's on track."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 19,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Training images size: (60000, 784)\n",
- "Training labels size: (60000,)\n",
- "Testing images size: (10000, 784)\n",
- "Training labels size: (10000,)\n"
- ]
- }
- ],
- "source": [
- "print(\"Training images size:\", train_img.shape)\n",
- "print(\"Training labels size:\", train_lbl.shape)\n",
- "print(\"Testing images size:\", test_img.shape)\n",
- "print(\"Training labels size:\", test_lbl.shape)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 20,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "learner = kNN_learner()\n",
- "learner.train(train_img, train_lbl)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Let us predict the classes of first 100 test images."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 21,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# takes ~17 Secs. to execute this cell\n",
- "num_test = 100\n",
- "predictions = learner.predict_labels(test_img[:num_test], k=3)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Let's compare the performances of both implementations. It took 20 Secs. to predict one image using our native implementations and 17 Secs. to predict 100 images in faster implementations. That's 110 times faster.\n",
- "\n",
- "Now, test the accuracy of our predictions:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 22,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Accuracy of predictions: 98.0 %\n"
- ]
- }
- ],
- "source": [
- "# print(predictions)\n",
- "# print(test_lbl[:num_test])\n",
- "\n",
- "num_correct = np.sum([predictions == test_lbl[:num_test]])\n",
- "num_accuracy = (float(num_correct) / num_test) * 100\n",
- "print(\"Accuracy of predictions:\", num_accuracy, \"%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Introduction to Scikit-Learn\n",
- "\n",
- "In this section we will solve this MNIST problem using Scikit-Learn. Learn more about Scikit-Learn [here](http://scikit-learn.org/stable/index.html). As we are using this library, we don't need to define our own functions (kNN or Support Vector Machines aka SVMs) to classify digits.\n",
+ "Hurray! We've got it correct. Don't worry if our algorithm predicted a wrong class. With this techinique we have only ~97% accuracy on this dataset. Let's try with a different test image and hope we get it this time.\n",
"\n",
- "Let's start by importing necessary modules for kNN and SVM."
+ "You might have recognized that our algorithm took ~20 seconds to predict a single image. How would we even predict all 10,000 test images? Yeah, the implementations we have in our learning module are not optimized to run on this particular dataset, as they are written with readability in mind, instead of efficiency."
]
- },
- {
- "cell_type": "code",
- "execution_count": 41,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "from sklearn.neighbors import NearestNeighbors\n",
- "from sklearn import svm"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 42,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "data": {
- "text/plain": [
- "LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
- " intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
- " multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
- " verbose=0)"
- ]
- },
- "execution_count": 42,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "# takes ~3 mins to execute the cell\n",
- "SVMclf = svm.LinearSVC()\n",
- "SVMclf.fit(train_img, train_lbl)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 43,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "predictions = SVMclf.predict(test_img)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 44,
- "metadata": {
- "collapsed": false
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Accuracy of predictions: 88.25 %\n"
- ]
- }
- ],
- "source": [
- "num_correct = np.sum(predictions == test_lbl)\n",
- "num_accuracy = (float(num_correct)/len(test_lbl)) * 100\n",
- "print(\"Accuracy of predictions:\", num_accuracy, \"%\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "You might observe that this accuracy is far less than what we got using native kNN implementation. But we can tweak the parameters to get higher accuracy on this problem which we are going to explain in coming sections."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": []
}
],
"metadata": {
@@ -924,13 +834,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.5.1"
- },
- "widgets": {
- "state": {},
- "version": "1.1.1"
+ "version": "3.5.2"
}
},
"nbformat": 4,
- "nbformat_minor": 0
+ "nbformat_minor": 2
}
pFad - Phonifier reborn
Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.
Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.
Alternative Proxies:
Alternative Proxy
pFad Proxy
pFad v3 Proxy
pFad v4 Proxy