311 lines
8.8 KiB
Plaintext
311 lines
8.8 KiB
Plaintext
|
{
|
||
|
|
"cells": [
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"# Ungraded Lab: Cost Function \n",
|
||
|
|
"\n",
|
||
|
|
"In this ungraded lab, you will implement the `cost` function for linear regression with one variable. The term 'cost' in this assignment might be a little confusing since the data is housing cost. Here, cost is a measure how well our model is predicting the actual value of the house. We will use the term 'price' for the data.\n",
|
||
|
|
"\n",
|
||
|
|
"First, let's run the cell below to import [matplotlib](http://matplotlib.org), which is a famous library to plot graphs in Python. "
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"import matplotlib.pyplot as plt"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"## Problem Statement\n",
|
||
|
|
"\n",
|
||
|
|
"Let's use the same two data points as before - a house with 1000 square feet sold for \\\\$200,000 and a house with 2000 square feet sold for \\\\$400,000.\n",
|
||
|
|
"\n",
|
||
|
|
"That is our dataset contains has the following two points - \n",
|
||
|
|
"\n",
|
||
|
|
"| Size (feet$^2$) | Price (1000s of dollars) |\n",
|
||
|
|
"| -------------------| ------------------------ |\n",
|
||
|
|
"| 1000 | 200 |\n",
|
||
|
|
"| 2000 | 400 |\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# X_train is the input features, in this case (size in square feet)\n",
|
||
|
|
"# y_train is the actual value (price in 1000s of dollars)\n",
|
||
|
|
"X_train = [1000, 2000] \n",
|
||
|
|
"y_train = [200, 400]"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# routine to plot the data points\n",
|
||
|
|
"def plt_house(X, y,f_w=None):\n",
|
||
|
|
" plt.scatter(X, y, marker='x', c='r', label=\"Actual Value\")\n",
|
||
|
|
"\n",
|
||
|
|
" # Set the title\n",
|
||
|
|
" plt.title(\"Housing Prices\")\n",
|
||
|
|
" # Set the y-axis label\n",
|
||
|
|
" plt.ylabel('Price (in 1000s of dollars)')\n",
|
||
|
|
" # Set the x-axis label\n",
|
||
|
|
" plt.xlabel('Size (feet^2)')\n",
|
||
|
|
" # print predictions\n",
|
||
|
|
" if f_w != None:\n",
|
||
|
|
" plt.plot(X, f_w, c='b', label=\"Our Prediction\")\n",
|
||
|
|
" plt.legend()\n",
|
||
|
|
" plt.show()\n",
|
||
|
|
" \n",
|
||
|
|
"plt_house(X_train,y_train)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Computing Cost\n",
|
||
|
|
"\n",
|
||
|
|
"The cost is:\n",
|
||
|
|
" $$J(\\mathbf{w}) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w}}(\\mathbf{x}^{(i)}) - y^{(i)})^2$$ \n",
|
||
|
|
" \n",
|
||
|
|
"where \n",
|
||
|
|
" $$f_{\\mathbf{w}}(\\mathbf{x}^{(i)}) = w_0x_0^{(i)} + w_1x_1^{(i)} \\tag{1}$$\n",
|
||
|
|
" \n",
|
||
|
|
"- $f_{\\mathbf{w}}(\\mathbf{x}^{(i)})$ is our prediction for example $i$ using our parameters $\\mathbf{w}$. \n",
|
||
|
|
"- $(f_{\\mathbf{w}}(\\mathbf{x}^{(i)}) -y^{(i)})^2$ is the squared difference between the actual value and our prediction. \n",
|
||
|
|
"- These differences are summed over all the $m$ examples and averaged to produce the cost, $J(\\mathbf{w})$. \n",
|
||
|
|
"Note, in lecture summation ranges are typically from 1 to m while in code, we will run 0 to m-1."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"<details>\n",
|
||
|
|
"<summary>\n",
|
||
|
|
" <font size='3', color='darkgreen'><b>Hints</b></font>\n",
|
||
|
|
"</summary>\n",
|
||
|
|
"\n",
|
||
|
|
"```\n",
|
||
|
|
"#Function to calculate the cost\n",
|
||
|
|
"def compute_cost(X, y, w):\n",
|
||
|
|
" \n",
|
||
|
|
" m = len(X)\n",
|
||
|
|
" cost = 0\n",
|
||
|
|
" \n",
|
||
|
|
" for i in range(m):\n",
|
||
|
|
" \n",
|
||
|
|
" # Calculate the model prediction\n",
|
||
|
|
" f_w = w[0] + w[1]*X[i]\n",
|
||
|
|
" \n",
|
||
|
|
" # Calculate the cost\n",
|
||
|
|
" cost = cost + (f_w - y[i])**2\n",
|
||
|
|
" \n",
|
||
|
|
" total_cost = 1/(2*m) * cost\n",
|
||
|
|
"\n",
|
||
|
|
" return total_cost\n",
|
||
|
|
"```"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"#Function to calculate the cost\n",
|
||
|
|
"def compute_cost(X, y, w):\n",
|
||
|
|
" \n",
|
||
|
|
" m = len(X)\n",
|
||
|
|
" cost = 0\n",
|
||
|
|
" \n",
|
||
|
|
" for i in range(m):\n",
|
||
|
|
" ### START CODE HERE ### \n",
|
||
|
|
"\n",
|
||
|
|
" ### END CODE HERE ### \n",
|
||
|
|
" total_cost = 1/(2*m) * cost\n",
|
||
|
|
"\n",
|
||
|
|
" return total_cost"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"w_p = [1, 2] # w0 = w[0], w1 = w[1] \n",
|
||
|
|
"\n",
|
||
|
|
"total_cost = compute_cost(X_train, y_train, w_p)\n",
|
||
|
|
"print(\"Total cost :\", total_cost)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"**Expected Output**:\n",
|
||
|
|
"```Total cost : 4052700.5```"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"In the next lab, we will minimise the cost by optimizing our parameters $\\mathbf{w}$ using gradient descent. For now, we can try various values manually. To to keep it simple, we know from the previous lab that $w_0 = 0$ produces a minimum. So, we'll set $w_0$ to zero and vary $w_1$."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# Print w1 vs cost to see minimum\n",
|
||
|
|
"\n",
|
||
|
|
"w1_list = [-0.6, -0.4, -0.2, 0, 0.2, 0.4, 0.6]\n",
|
||
|
|
"cost_list = []\n",
|
||
|
|
"\n",
|
||
|
|
"for w1 in w1_list:\n",
|
||
|
|
" w_p = [0, w1]\n",
|
||
|
|
" total_cost = compute_cost(X_train, y_train, w_p)\n",
|
||
|
|
" cost_list.append(total_cost)\n",
|
||
|
|
" \n",
|
||
|
|
"plt.plot(w1_list, cost_list)\n",
|
||
|
|
"plt.title(\"Cost vs w1\")\n",
|
||
|
|
"plt.ylabel('Cost')\n",
|
||
|
|
"plt.xlabel('w1')\n",
|
||
|
|
"plt.show()"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"# We can see a global minimum at w1 = 0.2 Therefore, let's try w = [0,0.2] \n",
|
||
|
|
"# to see if that fits the data\n",
|
||
|
|
"w_p = [0, 0.2] # w0 = 0, w1 = 0.2\n",
|
||
|
|
"\n",
|
||
|
|
"total_cost = compute_cost(X_train, y_train,w_p)\n",
|
||
|
|
"print(\"Total cost :\", total_cost)\n",
|
||
|
|
"f_w = [w_p[0] + w_p[1]*X_train[0], w_p[0] + w_p[1]*X_train[1]]\n",
|
||
|
|
"plt_house(X_train, y_train, f_w)\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"\n",
|
||
|
|
"We can see how our cost varies as we modify both $w_0$ and $w_1$ by plotting in 3D or in contour plots."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"from mpl_toolkits.mplot3d import axes3d\n",
|
||
|
|
"import matplotlib.pyplot as plt\n",
|
||
|
|
"import numpy as np\n",
|
||
|
|
"\n",
|
||
|
|
"w0 = np.arange(-500, 500, 5)\n",
|
||
|
|
"w1 = np.arange(-0.2, 0.8, 0.005)\n",
|
||
|
|
"w0,w1 = np.meshgrid(w0,w1)\n",
|
||
|
|
"z=np.zeros_like(w0)\n",
|
||
|
|
"n,_ = w0.shape\n",
|
||
|
|
"for i in range(n):\n",
|
||
|
|
" for j in range(n):\n",
|
||
|
|
" z[i][j] = compute_cost(X_train, y_train, [w0[i][j],w1[i][j]] )\n",
|
||
|
|
"\n",
|
||
|
|
"fig = plt.figure(figsize=(12,6))\n",
|
||
|
|
"plt.subplots_adjust( wspace=0.5 )\n",
|
||
|
|
"#===============\n",
|
||
|
|
"# First subplot\n",
|
||
|
|
"#===============\n",
|
||
|
|
"# set up the axes for the first plot\n",
|
||
|
|
"ax = fig.add_subplot(1, 2, 1, projection='3d')\n",
|
||
|
|
"ax.plot_surface(w1, w0, z, rstride=8, cstride=8, alpha=0.3)\n",
|
||
|
|
"\n",
|
||
|
|
"ax.set_xlabel('w_1')\n",
|
||
|
|
"ax.set_ylabel('w_0')\n",
|
||
|
|
"ax.set_zlabel('cost')\n",
|
||
|
|
"plt.title('3D plot of cost vs w0, w1')\n",
|
||
|
|
"# Customize the view angle \n",
|
||
|
|
"ax.view_init(elev=20., azim=-65)\n",
|
||
|
|
"\n",
|
||
|
|
"#===============\n",
|
||
|
|
"# Second subplot\n",
|
||
|
|
"#===============\n",
|
||
|
|
"# set up the axes for the second plot\n",
|
||
|
|
"ax = fig.add_subplot(1, 2, 2)\n",
|
||
|
|
"CS = ax.contour(w1, w0, z,[0,50,1000,5000,10000,25000,50000])\n",
|
||
|
|
"plt.clabel(CS, inline=1, fmt='%1.0f', fontsize=10)\n",
|
||
|
|
"plt.title('Contour plot of cost vs (w0,w1)')\n",
|
||
|
|
"\n",
|
||
|
|
"ax.set_xlabel('w_1')\n",
|
||
|
|
"ax.set_ylabel('w_0')\n",
|
||
|
|
"\n",
|
||
|
|
"plt.show()"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"<details>\n",
|
||
|
|
"<summary>\n",
|
||
|
|
" <font size='3'><b>Expected graph</b></font>\n",
|
||
|
|
"</summary>\n",
|
||
|
|
" <img src=\"./figures/ThreeD_And_ContourLab3.PNG\" alt=\"Contour Plot\">\n",
|
||
|
|
"<\\details>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": null,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": []
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"metadata": {
|
||
|
|
"kernelspec": {
|
||
|
|
"display_name": "Python 3",
|
||
|
|
"language": "python",
|
||
|
|
"name": "python3"
|
||
|
|
},
|
||
|
|
"language_info": {
|
||
|
|
"codemirror_mode": {
|
||
|
|
"name": "ipython",
|
||
|
|
"version": 3
|
||
|
|
},
|
||
|
|
"file_extension": ".py",
|
||
|
|
"mimetype": "text/x-python",
|
||
|
|
"name": "python",
|
||
|
|
"nbconvert_exporter": "python",
|
||
|
|
"pygments_lexer": "ipython3",
|
||
|
|
"version": "3.8.6"
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"nbformat": 4,
|
||
|
|
"nbformat_minor": 5
|
||
|
|
}
|