+{"status":"ok","feed":{"url":"https://medium.com/feed/@devil-cyber","title":"Stories by Manikant Kumar on Medium","link":"https://medium.com/@devil-cyber?source=rss-9970de7e9d9f------2","author":"","description":"Stories by Manikant Kumar on Medium","image":"https://cdn-images-1.medium.com/fit/c/150/150/0*Ha7Ur0A84nuvN0wf"},"items":[{"title":"Linear Regression From Scratch Using MXNet","pubDate":"2021-06-14 17:21:19","link":"https://devil-cyber.medium.com/linear-regression-from-scratch-using-mxnet-e39afd9ae56f?source=rss-9970de7e9d9f------2","guid":"https://medium.com/p/e39afd9ae56f","author":"Manikant Kumar","thumbnail":"https://cdn-images-1.medium.com/max/1024/0*KyZiYIxTVwf_mSiK","description":"\n<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/0*KyZiYIxTVwf_mSiK\"><figcaption>Photo by <a href=\"https://unsplash.com/@pietrozj?utm_source=medium&utm_medium=referral\">Pietro Jeng</a> on\u00a0<a href=\"https://unsplash.com/?utm_source=medium&utm_medium=referral\">Unsplash</a></figcaption></figure><p>Linear Regression is one of the most important supervised machine learning algorithm. I think this is the one of the first machine learning algorithm people learn when they start their machine learning\u00a0journey.</p>\n<p>Apache <a href=\"https://mxnet.apache.org/versions/1.8.0/\">MXNet</a> is an open-source deep learning software framework, used to train, and deploy deep neural networks. It is scalable, allowing for fast model training, and supports a flexible programming model and multiple programming languages (including C++, Python, Java, Julia, Matlab, JavaScript, Go, R, Scala,\u00a0Perl,</p>\n<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/220/1*dnFljsilN6-wpo0orHg3Bw.png\"><figcaption>In linear regression, the observations (<strong>red</strong>) are assumed to be the result of random deviations (<strong>green</strong>) from an underlying relationship (<strong>blue</strong>) between a dependent variable (<em>y</em>) and an independent variable\u00a0(<em>x</em>).</figcaption></figure><p>We will construct an artificial datasets according to a linear model with additive noise. We will keep the data low-dimensional so we can visualize it easily. In the following code snippet, we generate a datasets containing 1000 examples, each consisting of 2 features sampled from a standard normal distribution. Thus our synthetic datasets will be a matrix of (1000 x\u00a02)</p>\n<p>At first install all the required\u00a0library:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Now import all the required\u00a0library:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Now create the datasets:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Now we can see the linear relation between the label and the 2nd column of the datasets:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/492/1*5bpMb8lJgwtSoSn8wwn87g.png\"></figure><p>Training models consists of making multiple passes over the datasets, grabbing one mini batch of examples at a time, and using them to update our model. Since this process is so fundamental to training machine learning algorithms, it is worth defining a utility function to shuffle the datasets and access it in mini\u00a0batches.</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Initializing Model Parameters & Defining the\u00a0Model:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Initializing Model Parameters & Defining the\u00a0Model:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Defining the Loss Function:Since updating our model requires taking the gradient of our loss function, we ought to define the loss function first. Here we will use the squared loss\u00a0function</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Defining the Optimization Algorithm:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Now that we have all of the parts in place, we are ready to implement the main training loop.In each epoch, we will iterate through the entire datasets (using the data_iter function) once passing through every example in the training datasets (assuming that the number of examples is divisible by the batch size). The number of epochs num_epochs and the learning rate lr are both hyper parameters, which we set here to 3 and 0.03, respectively. Unfortunately, setting hyper parameters is tricky and requires some adjustment by trial and\u00a0error.</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>You can also get the whole code at <a href=\"https://colab.research.google.com/drive/1E_KJKNaRPk02j4aG9pgoOwEwhNW-Q76c?usp=sharing\">google\u00a0colab</a>:</p>\n<a href=\"https://medium.com/media/74324b8c0509ddb5824b841e7267d4e1/href\">https://medium.com/media/74324b8c0509ddb5824b841e7267d4e1/href</a><p>Connect with me via <a href=\"https://github.com/devil-cyber\">github</a> or\u00a0<a href=\"http://manikant.codes/\">blogpage</a></p>\n<p>Thanks for reading & suggestion is welcome\u00a0!!!</p>\n<img src=\"https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e39afd9ae56f\" width=\"1\" height=\"1\" alt=\"\">\n","content":"\n<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/0*KyZiYIxTVwf_mSiK\"><figcaption>Photo by <a href=\"https://unsplash.com/@pietrozj?utm_source=medium&utm_medium=referral\">Pietro Jeng</a> on\u00a0<a href=\"https://unsplash.com/?utm_source=medium&utm_medium=referral\">Unsplash</a></figcaption></figure><p>Linear Regression is one of the most important supervised machine learning algorithm. I think this is the one of the first machine learning algorithm people learn when they start their machine learning\u00a0journey.</p>\n<p>Apache <a href=\"https://mxnet.apache.org/versions/1.8.0/\">MXNet</a> is an open-source deep learning software framework, used to train, and deploy deep neural networks. It is scalable, allowing for fast model training, and supports a flexible programming model and multiple programming languages (including C++, Python, Java, Julia, Matlab, JavaScript, Go, R, Scala,\u00a0Perl,</p>\n<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/220/1*dnFljsilN6-wpo0orHg3Bw.png\"><figcaption>In linear regression, the observations (<strong>red</strong>) are assumed to be the result of random deviations (<strong>green</strong>) from an underlying relationship (<strong>blue</strong>) between a dependent variable (<em>y</em>) and an independent variable\u00a0(<em>x</em>).</figcaption></figure><p>We will construct an artificial datasets according to a linear model with additive noise. We will keep the data low-dimensional so we can visualize it easily. In the following code snippet, we generate a datasets containing 1000 examples, each consisting of 2 features sampled from a standard normal distribution. Thus our synthetic datasets will be a matrix of (1000 x\u00a02)</p>\n<p>At first install all the required\u00a0library:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Now import all the required\u00a0library:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Now create the datasets:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Now we can see the linear relation between the label and the 2nd column of the datasets:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/492/1*5bpMb8lJgwtSoSn8wwn87g.png\"></figure><p>Training models consists of making multiple passes over the datasets, grabbing one mini batch of examples at a time, and using them to update our model. Since this process is so fundamental to training machine learning algorithms, it is worth defining a utility function to shuffle the datasets and access it in mini\u00a0batches.</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Initializing Model Parameters & Defining the\u00a0Model:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Initializing Model Parameters & Defining the\u00a0Model:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Defining the Loss Function:Since updating our model requires taking the gradient of our loss function, we ought to define the loss function first. Here we will use the squared loss\u00a0function</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Defining the Optimization Algorithm:</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>Now that we have all of the parts in place, we are ready to implement the main training loop.In each epoch, we will iterate through the entire datasets (using the data_iter function) once passing through every example in the training datasets (assuming that the number of examples is divisible by the batch size). The number of epochs num_epochs and the learning rate lr are both hyper parameters, which we set here to 3 and 0.03, respectively. Unfortunately, setting hyper parameters is tricky and requires some adjustment by trial and\u00a0error.</p>\n<a href=\"https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href\">https://medium.com/media/47ccc52d715dd0638efb8cb0a9ffa220/href</a><p>You can also get the whole code at <a href=\"https://colab.research.google.com/drive/1E_KJKNaRPk02j4aG9pgoOwEwhNW-Q76c?usp=sharing\">google\u00a0colab</a>:</p>\n<a href=\"https://medium.com/media/74324b8c0509ddb5824b841e7267d4e1/href\">https://medium.com/media/74324b8c0509ddb5824b841e7267d4e1/href</a><p>Connect with me via <a href=\"https://github.com/devil-cyber\">github</a> or\u00a0<a href=\"http://manikant.codes/\">blogpage</a></p>\n<p>Thanks for reading & suggestion is welcome\u00a0!!!</p>\n<img src=\"https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e39afd9ae56f\" width=\"1\" height=\"1\" alt=\"\">\n","enclosure":{},"categories":["data-science","deep-learning","mxnet","machine-learning","linear-regression"]}]}
0 commit comments