From ebbfa7372e587d6bc7ef895c067705f0c909b06e Mon Sep 17 00:00:00 2001
From: yul \n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
Notation | Description
| Python (if applicable) |\n",
+ "|: ------------|: ------------------------------------------------------------||\n",
+ "| $a$ | scalar, non bold ||\n",
+ "| $\\mathbf{a}$ | vector, bold ||\n",
+ "| **Regression** | | | |\n",
+ "| $\\mathbf{x}$ | Training Example feature values (in this lab - Size (1000 sqft)) | `x_train` | \n",
+ "| $\\mathbf{y}$ | Training Example targets (in this lab Price (1000s of dollars)) | `y_train` \n",
+ "| $x^{(i)}$, $y^{(i)}$ | $i_{th}$Training Example | `x_i`, `y_i`|\n",
+ "| m | Number of training examples | `m`|\n",
+ "| $w$ | parameter: weight | `w` |\n",
+ "| $b$ | parameter: bias | `b` | \n",
+ "| $f_{w,b}(x^{(i)})$ | The result of the model evaluation at $x^{(i)}$ parameterized by $w,b$: $f_{w,b}(x^{(i)}) = wx^{(i)}+b$ | `f_wb` | \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Tools\n",
+ "In this lab you will make use of: \n",
+ "- NumPy, a popular library for scientific computing\n",
+ "- Matplotlib, a popular library for plotting data"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import numpy as np\n",
+ "import matplotlib.pyplot as plt\n",
+ "plt.style.use('./deeplearning.mplstyle')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Problem Statement\n",
+ "
\n",
+ "\n",
+ "As in the lecture, you will use the motivating example of housing price prediction. \n",
+ "This lab will use a simple data set with only two data points - a house with 1000 square feet(sqft) sold for \\\\$300,000 and a house with 2000 square feet sold for \\\\$500,000. These two points will constitute our *data or training set*. In this lab, the units of size are 1000 sqft and the units of price are 1000s of dollars.\n",
+ "\n",
+ "| Size (1000 sqft) | Price (1000s of dollars) |\n",
+ "| -------------------| ------------------------ |\n",
+ "| 1.0 | 300 |\n",
+ "| 2.0 | 500 |\n",
+ "\n",
+ "You would like to fit a linear regression model (shown above as the blue straight line) through these two points, so you can then predict price for other houses - say, a house with 1200 sqft.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Please run the following code cell to create your `x_train` and `y_train` variables. The data is stored in one-dimensional NumPy arrays."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "x_train = [1. 2.]\n",
+ "y_train = [300. 500.]\n"
+ ]
+ }
+ ],
+ "source": [
+ "# x_train is the input variable (size in 1000 square feet)\n",
+ "# y_train is the target (price in 1000s of dollars)\n",
+ "x_train = np.array([1.0, 2.0])\n",
+ "y_train = np.array([300.0, 500.0])\n",
+ "print(f\"x_train = {x_train}\")\n",
+ "print(f\"y_train = {y_train}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ">**Note**: The course will frequently utilize the python 'f-string' output formatting described [here](https://docs.python.org/3/tutorial/inputoutput.html) when printing. The content between the curly braces is evaluated when producing the output."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Number of training examples `m`\n",
+ "You will use `m` to denote the number of training examples. Numpy arrays have a `.shape` parameter. `x_train.shape` returns a python tuple with an entry for each dimension. `x_train.shape[0]` is the length of the array and number of examples as shown below."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "x_train.shape: (2,)\n",
+ "Number of training examples is: 2\n"
+ ]
+ }
+ ],
+ "source": [
+ "# m is the number of training examples\n",
+ "print(f\"x_train.shape: {x_train.shape}\")\n",
+ "m = x_train.shape[0]\n",
+ "print(f\"Number of training examples is: {m}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "One can also use the Python `len()` function as shown below."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Number of training examples is: 2\n"
+ ]
+ }
+ ],
+ "source": [
+ "# m is the number of training examples\n",
+ "m = len(x_train)\n",
+ "print(f\"Number of training examples is: {m}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Training example `x_i, y_i`\n",
+ "\n",
+ "You will use (x$^{(i)}$, y$^{(i)}$) to denote the $i^{th}$ training example. Since Python is zero indexed, (x$^{(0)}$, y$^{(0)}$) is (1.0, 300.0) and (x$^{(1)}$, y$^{(1)}$) is (2.0, 500.0). \n",
+ "\n",
+ "To access a value in a Numpy array, one indexes the array with the desired offset. For example the syntax to access location zero of `x_train` is `x_train[0]`.\n",
+ "Run the next code block below to get the $i^{th}$ training example."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(x^(0), y^(0)) = (1.0, 300.0)\n"
+ ]
+ }
+ ],
+ "source": [
+ "i = 0 # Change this to 1 to see (x^1, y^1)\n",
+ "\n",
+ "x_i = x_train[i]\n",
+ "y_i = y_train[i]\n",
+ "print(f\"(x^({i}), y^({i})) = ({x_i}, {y_i})\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Plotting the data"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can plot these two points using the `scatter()` function in the `matplotlib` library, as shown in the cell below. \n",
+ "- The function arguments `marker` and `c` show the points as red crosses (the default is blue dots).\n",
+ "\n",
+ "You can use other functions in the `matplotlib` library to set the title and labels to display"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAERCAYAAAB8eMxzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de1QV9d7H8Tc30VAuKqWgmYmgIoiaiQSIiihFXlrVScmIzjE6abfT0+2sk+XxeSofO5w0KzN71LyXXRRLLUpEMI9ZXtpmeE1NJBVU0HS7gXn+oHYSChthIN2f11os9v7Nnpnvz7X87OE3M79xMQzDQEREnIZrYxcgIiINS8EvIuJkFPwiIk5GwS8i4mQU/CIiTkbBLyLiZBT88od03XXXsWHDhkptDzzwAM8//7xp+wwNDeWrr76q123OmTMHd3d3mjdvjo+PD5GRkXz55ZcNWoPI7yn4RX6xfft2+vTpU+/bjY+P59SpUxw7dozY2Fhuu+02fn/7jM1mM7UGkfMp+OWyNXXqVDp27MjVV19NWloaZ86cASqOsocOHWr/3A8//EDTpk0BKC8vZ9y4cbRu3dp+BP5r6J7/V0ZcXBz//Oc/ueGGG/Dx8WHUqFGcO3fOvs3nn38ef39/goODefnll7nuuutqrNfDw4OUlBQKCgooLCwkLi6OCRMm0KtXL3x9favUcPr0aR588EECAwPx8/NjzJgx9m0tXbqU0NBQWrZsybBhwzhy5AgAP/30EwkJCfj4+NCqVSvGjx9/qf+8cgVT8MtlafXq1bz88st8+umn7Nq1iz179jBp0qQa1/v000/ZuHEj+/bto6ioiFdeeQVX1wv/N3jvvff44IMP2L9/P9u2bWPJkiUAZGRkMHfuXL766ivWr1/P0qVLHar53LlzzJkzh8DAQFq3bg3AokWLWLp0KceOHavy+UcffZQDBw6wdetWjhw5QlpaGgBfffUVf/vb31i8eDE//fQTXbp04a9//SsA//73v+ncuTOFhYX8+OOPlb4sRH6l4Jc/rMGDB+Pr62v/mT17tn3ZkiVLSEtLo3Pnzvj4+DBhwgQWL15c4zY9PDwoKSlh586duLq6EhkZiZub2wU/O3bsWK699lp8fX255ZZb2Lp1KwDvv/8+Y8eO5brrrqN169Y8/PDD1e7z888/x9fXl8DAQL766is+/PDDSvu4/vrradasWaV1ysvLmTdvHtOmTaN169Z4eHgQHR0NwNtvv8348eMJCwvDw8ODCRMmsHz5ckpLS/Hw8ODw4cMcOnSIZs2a0bdv3xr/TcT5KPjlD+uzzz7jxIkT9p/U1FT7svz8fNq3b29/36FDBw4fPlzjNgcNGsTYsWNJSUkhMDCw2pPFV199tf31VVddxalTpwAoKCigXbt29mWBgYE17vPEiRMcPXqUrKysSmP45/fhfEePHuXcuXMXHEI6cOAAEydOtH8htmvXDnd3dwoKCnjiiScIDAwkKiqK7t278/7771dbmzgnBb9clgICAjh48KD9/YEDB2jbti0AXl5e/Pzzz/ZlP/30U6V1H3/8cSwWC9nZ2bz99tt89tlntdp3mzZtOHTokP39+a9ry8XF5YLt/v7+NGnShP3791dZFhgYyAsvvFDpS/HMmTO0a9cOb29vXn31VQ4dOsS//vUvRo8ebf/CEvmVgl8uS3fccQczZ85k9+7dFBcXM2nSJO68804AwsPD+frrr8nLy6OkpISXXnrJvt6mTZv4+uuvKSsrw9vbG3d394sO9VzMbbfdxltvvcX+/fspLCxk+vTp9do3AFdXV+655x4eeeQRCgsLsdls5ObmApCamsq0adPsQ09FRUUsW7YMgE8++YQffvgBAD8/P1xcXGrdP7nyKfjlspSYmMhjjz3GoEGDCAoKokOHDkyYMAGAkJAQnnzySfr160ePHj0YMmSIfb2TJ0+SkpKCj48P4eHhjBo1ioEDB9Zq38OGDePuu++md+/e9O3bl1tuuQVPT8967R9Aeno6AQEBhIaGcs011zBz5kwAoqKimDJlCvfccw/e3t706tXL/qXw/fffExsbS/PmzUlOTmbu3LlVzh+IuGg+fpG6eeedd5gzZw5ffPFFY5ci4hAd8YtcgmXLllFaWsqBAwd4+eWXGT58eGOXJOIwHfGLXIL+/fvzzTff4OXlxZ133snLL79MkyZNGrssEYco+EVEnIyGekREnIx7YxdQnZMnTzZ2CSIilzUfH58qbTriFxFxMgp+EREn84ce6jnfhf5cERGRqmoaJtcRv4iIk1Hwi4g4GQW/iIiTUfCLiPzRTJwIvz5YaPHiivf1yLTg/+GHH7jmmmuIi4sjISEBgClTphAdHU1ycrL9OacLFiwgKiqKpKQkiouLzSpHROTy8PzzFT/JyTBiRMXvX9vqialH/IMHDyYrK4tPP/2Uo0ePsmbNGnJycggPD+ejjz7CZrMxY8YMsrOzGTNmDG+++aaZ5YiI/LFNnPjb0X15OSxbVvH798vqyNTgX7NmDTExMfz73/9m48aNxMXFARAfH8+GDRvYuXMnYWFhuLu729tERJxWSAi4XiSWXV0rltcD067jb9u2LTt37sTT05Phw4dTXFzMNddcA1Rck3/8+HFOnDiBt7d3pTYREad1110VY/q/PFGtkltvrVheD0w74vf09MTLywt3d3eSkpIICgqyj+EXFxfbHxT9+zYREae1eDFkZFx4WUbGbyd868i04C8pKbG/zs3NJSgoiLVr1wKQmZlJZGQkwcHBWCwWysrK7G0iIk4rL++3Mf3fKy+vWF4PTAv+devW0bt3b6KioggICKBv377ExsYSHR3Nli1bGDFiBB4eHowdO5aYmBjmzp1LWlqaWeWIiPzxPfdcxQ9UjOkPH/7bmP/5y+roD/0glvPnm9BcPSLiNCZOrDiR++uYf15erUK/puxU8IuIXGFqyk7duSsi4mQU/CIiTkbBLyLiZBT8IiJORsEvIuJkFPwiIk5GwS8i4mQU/CIiTkbBLyLiZBT8IiJORsEvIuJkFPwiIk5GwS8i4mQU/CIiTkbBLyLiZEwP/vT0dPtTt+Li4oiLi6Njx4688sorAISEhNjbv/vuO7PLERFxeu5mbtxqtbJ161YAIiIiyMrKAmD48OEkJSUB4O/vb28XERHzmXrEP2vWLFJSUiq1nT59moKCAoKCggAoKioiNjaWtLQ0zp49a2Y5IiKCicFvs9lYu3YtAwcOrNS+cuVKhg4dan+fk5NDdnY2HTp0YObMmWaVIyIivzAt+OfNm8fo0aOrtH/44Yfcdttt9vctW7YEYOTIkVgsFrPKERGRX5gW/Hl5ebzxxhsMHTqU7du38+qrr2Kz2dixYwc9evQA4Ny5c1itVgByc3Pp1KmTWeWIiMgvTDu5O3nyZPvr6OhoHnroIVavXl1p6Of48eMkJibSvHlz/Pz8mD9/vlnliIjIL1wMwzAau4iLOXnypP21j49PI1YiInL5qCk7dQOXiIiTUfCLiDgZBb+IiJNR8IuIOBkFv4iIk1Hwi4g4GQW/iIiTUfCLiDgZBb+IiJNR8IuIOJka5+qxWq2sXLmS//znPxQUFNC0aVO6dOlCQkICXbt2bYgaRUSkHlU7V8+ECRP47LPPiImJoWfPnvj7+2O1Wtm9ezfr1q3j1KlTvPTSS0RERJhSnObqERGpvZqys9rgX7NmDQMGDLjoxo8fP87BgwcJDw+vY5kXpuAXEam9OgX/hZgd9udT8IuI1F69zM4ZExNDSUkJR48epWfPnjz88MOMGzeu/qoUEZEG41Dwnz59mhYtWrB06VLGjh1LVlYWubm5ZtcmIiImcCj4S0tLWbt2LXPmzOHWW2+1t4mIyOXHoeB/9dVX+de//sXIkSMJDw9n37591Z70PV96ejrR0dFAxVhTXFwccXFxFBUVAbBgwQKioqJISkqiuLj4ErshIiKOqvHkbllZGX/5y1+YPXt2rTdutVq5//772bNnDzk5OURHR5OTk2NfbrPZGDhwIGvWrOH999/nwIEDPPHEE/blOrkrIlJ7dT656+bmRkFBAWfOnKn1zmfNmkVKSor9/Y4dO4iJieHpp5/GMAx27txJWFgY7u7uxMfHs2HDhlrvQ0REaqfGO3cBWrVqRc+ePRkyZAheXl729hdeeOGi69hsNtauXcu4ceOYMGECALt27cLPz48HHniAjIwMWrVqhbe3N1DxrXT8+PG69EVERBzgUPAPHjyYwYMH12rD8+bNY/To0ZXaWrZsCcCIESPYvHkzw4cPt4/rFxcX4+vrW6t9iIhI7TkU/OcP1zgqLy+PLVu2MGPGDLZv387UqVMZP348bm5u5ObmEhYWRnBwMBaLhbKyMjIzM4mMjKz1fkREpHYcCn6LxcKTTz7Jjh07sFqt9vb8/PyLrjN58mT76+joaPr370+fPn3w8vLi+uuvZ+LEibi5uTF27FhiYmLw8/Nj4cKFdeiKiIg4wqEpGyIjI3njjTe47777yMnJYfbs2Zw4cYJ//OMfphanq3pERGqvXqZsKC0tpWfPnpSVleHl5cX48eNZuXJl/VUpIiINxqGhnquuugqbzUZ4eDgTJ04kICCAU6dOmV2biIiYwKEj/jlz5lBaWsprr72GYRjs2LGD999/3+zaRETEBLWelrkhaYxfRKT2asrOaod62rZti4uLC4ZhXPB3dVf1iIjIH1O1wX/48OGGqkNERBpItcH/6aefVrtyQkJCvRYjIiLmqzb4Fy1adNFlLi4uCn4RkcuQTu6KiFxh6uUGrt27d5OQkED79u1p3749Q4YMYc+ePfVXpYiINBiHgj81NZXHHnuMgwcPcvDgQR577DHuvfdek0sTEREzOBT8xcXFJCYm2t8PHTpUj0kUEblMOTRlQ2hoKI8//jijRo3CxcWFxYsXExoaanZtIiJiAodO7p45c4bp06ezfv16DMPgpptuYvz48TRr1szU4nRyV0Sk9mrKTl3VIyJyhanTlA0dO3bExcXlosv37t1bh9JERKQxVBv833//PYZh8D//8z+0adOGP/3pTxiGwZIlS3RyV0TkMuXQUE+PHj3YunVrpbYbb7yRjRs31riD9PR0PvjgA+bNm8c999yDi4sL7dq1Y968ebi5uRESEkLbtm0BeP311+nWrZt9XQ31iIjUXr3cwNWqVStef/11CgsLKSws5LXXXsPLy6vG9axWq/0Lw9fXl4yMDLKzs+nYsSOffPIJAP7+/mRlZZGVlVUp9EVExBwOBf/ixYvZvn078fHxxMfHs2PHDpYsWVLjerNmzSIlJQUAPz8/fH19AXB3d8fNzQ2AoqIiYmNjSUtL4+zZs5faDxERcZBpV/XYbDaSk5N59913iY6OJicnB4D8/HzuuOMO1q5di7u7O0VFRbRs2ZIXXniB5s2b8/DDD9u3oaEeEZHaq5ehnksxb948Ro8eXanNarWSkpLCW2+9hbt7xXnlli1bAjBy5EgsFotZ5YiIyC8cunP3UuTl5bFlyxZmzJjB9u3befXVV9m0aRMPPvigfSz/3LlzGIaBp6cnubm5dOrUyaxyRETkF9UO9YwaNYpFixbxv//7vzz55JOXvJPo6GimTJlCQkICvXv3BuCRRx4hKiqKxMREmjdvjp+fH/Pnz6dFixb29TTUIyJSe3W6c7dbt24sWbKEu+66i48++ojffzQ4OLgeS61KwS8iUnt1Cv7Fixczd+5ccnJyuOGGGyqv6OLCF198UY+lVqXgFxGpvXqZq2fy5Mk89dRT9VuZAxT8IiK1Vy/B//PPPzN16lT7JZnR0dE8+uijmp1TROQPqF4u57zvvvs4ceIEL730Ei+++CLFxcWkpqbWX5UiItJgHLqc8/vvv2fx4sX29+Hh4URERJhWlIiImMehI35vb29WrFhhf//xxx/j7e1tWlEiImIeh8b49+zZw8MPP8yWLVtwdXWlR48eTJ061fQbrjTGLyJSe3oCl4iIk2m0uXpEROSPScEvIuJkFPwiIk7GoeD/29/+RklJCVarlQEDBtC2bVtmz55tdm0iImICh4J/zZo1tGjRgmXLltG1a1f27NnDK6+8YnZtIiJiAoeC/+zZs1itVpYuXcrtt9/OVVddZXZdIiJiEoeCPy0tjfbt21NSUsKAAQM4cOBApXnzRUTk8nHJ1/GXlpbaH59oFl3HLyJSe3W6jt8wDN577z3uvfdeEhISSEhIICUlhXfffRdXV8cuCEpPTyc6OhqAKVOmEB0dTXJyMjabDYAFCxYQFRVFUlISxcXFDndMREQuTbXpfe+997Jq1SrGjBnDtGnTmDZtGvfccw+rV68mJSWlxo1brVa2bt0KwNGjR1mzZg05OTmEh4fz0UcfYbPZmDFjBtnZ2YwZM4Y333yzfnolIiIXVe1YzZdffsnOnTsrtXXp0oVBgwY59NjFWbNmkZKSwoQJE9i4cSNxcXEAxMfHs3DhQrp160ZYWBju7u7Ex8dz//33X3pPRETEIdUe8bdt25ZZs2Zx+vRpe9vp06eZOXMmbdq0qXbDNpuNtWvXMnDgQABOnDhhn9HTx8eH48ePX7BNRETMVW3wL126lO+++45evXrRrl072rVrR+/evdmxYwdLly6tdsPz5s1j9OjR9ve+vr72Mfzi4mJ8fX0v2CYiIuaqdqjH39+f9PR00tPTa73hvLw8tmzZwowZM9i+fTubNm1i48aNPPnkk2RmZhIZGUlwcDAWi4WysjJ7m4iImKvGyzm/+uorPv74Y/Lz84GK4Z9bbrmFG2+80eGdREdHk5OTw+TJk8nIyODaa69lzpw5NGnShHnz5vHGG2/g5+fHwoULK116pMs5RURqr07z8T/99NNs2rSJUaNGERgYCMChQ4dYtGgRvXv3ZvLkySaU/BsFv4hI7dUp+IODg6tc1QMV1/cHBweza9eueirzwhT8IiK1V6cbuFq0aEFmZmaV9szMTE3ZICJymar25O7ixYt56qmnSE1Nxd/fH8MwOHbsGDfccAOLFi1qqBpFRKQeOTRXj2EYFBYWAtCqVStcXFxMLww01CMicilqys4aZ1krKChg1apVla7qGTp0KG3btq3HMkVEpKFUO8Y/bdo0Bg8ezJ49e2jZsiUtW7Zk7969JCQkMG3atIaqUURE6lGNV/Vs374dDw+PSu1Wq5WwsLALXvFTnzTUIyJSe3W6qsfDw4O9e/dWad+3b5/pc/GLiIg5qk3vWbNmkZycjJubm/0Grh9//JHy8nJmzZrVIAWKiEj9cuiqnvz8fPvJ3YCAAAICAkwvDDTUIyJyKep8VQ9cOOwtFgvdu3evY3kiItLQHHt+4gXcfPPN9VmHiIg0kGqP+M+fT/98hmFQVFRkSkEiImKuaoN/1apVzJs3j+bNm1dqNwyDL774wtTCRETEHNUG/6BBg2jRogWxsbFVlvXp08e0okRExDwOXdXTWHRVj4hI7dXpBi4REbnymBb8FouFqKgoYmJiSE1NZfPmzcTFxREXF0fHjh155ZVXAAgJCbG3f/fdd2aVIyIivzBt3oWQkBDWr18PQGpqKqWlpWRlZQEwfPhwkpKSgIoHuv/aLiIi5nM4+M+cOcPhw4cpLS21twUHB1/08+dP7Obp6Un79u0BOH36NAUFBQQFBQFQVFREbGwsXbt2ZerUqTRt2rTWnRAREcc5NNTzwgsvEBISwn333UdaWhppaWk88MADNa63fPlyunfvzpEjR2jVqhUAK1euZOjQofbP5OTkkJ2dTYcOHZg5c+YldkNERBzlUPDPnTuXvLw8srKyWLNmDWvWrHHoOv5hw4ZhsVgIDAxkxYoVAHz44Yfcdttt9s+0bNkSgJEjR2KxWC6lDyIiUgsOBX9ISAhWq7VWGz7/897e3jRr1gybzcaOHTvo0aMHAOfOnbN/Ljc3l06dOtVqHyIiUnsOjfGfO3eOrl27EhUVhaenp7194cKFF11n1apVpKenA9C5c2cSEhL47LPPGDhwoP0zx48fJzExkebNm+Pn58f8+fMvtR8iIuIgh27gWrt27QXb+/fvX+8FnU83cImI1F6dpmUuLS3F3d2dfv361X9lIiLSKKoN/hEjRrBixQpCQkJwcXGxtxuGgYuLywUfyygiIn9smqtHROQKU6e5ej799NNqN37s2DE2b958iaWJiEhjqHaoZ+PGjTz33HP069ePiIgIWrdujdVqZe/evWRnZ2MYBi+88EJD1SoiIvWgxqEem81GZmYmGzZsoKCggKZNm9KlSxeGDBnC9ddfb2pxGuoREam9mrJTY/wiIlcYzccvIiKVKPhFRJyMgl9ExMk4FPybN28mMjLSPonat99+yzPPPGNqYSIiYg6Hgv/BBx9k0aJFeHt7AxAWFkZGRoaphYmIiDkcCv7y8nI6duxYqc3Nzc2UgkRExFwOTcvcuXNn+4NUCgoKmD59Or169TK1MBERMYdDR/wzZsxg3bp1uLm5kZSURGlpKdOnTze7NhERMYFDN3CVlZVVGdq5UFt90w1cIiK1Vy83cMXExFBcXFxpo7GxsfVQnoiINDSHgv/nn3+2X9EDFd8gp0+frnYdi8VCVFQUMTExpKamYhgGPj4+xMXFERcXR1FREQALFiwgKiqKpKSkSl8uIiJiDoeC39fXl+zsbPv77OzsSl8EFxISEsL69etZt24dAJs2bSIsLIysrCyysrJo2bIlNpuNGTNmkJ2dzZgxY3jzzTfr0BUREXGEQ1f1vPHGG6SkpHDq1CkAWrRowdy5c6tdx8PDw/7a09OT9u3bs2PHDmJiYrjpppt48cUX2blzJ2FhYbi7uxMfH8/9999fh66IiIgjHAr+rl27snHjRkpKSoCK4HfE8uXL+fvf/05wcDCtWrVi165d+Pn58cADD5CRkUGrVq3sfzn4+Phw/PjxS+yGiIg4qtrgnz59OuPHj+eZZ56p9MzdX9X0EJZhw4YxbNgwHnroIVasWMHIkSOBimf5bt68meHDh9vH9YuLi/H19b3UfoiIiIOqDf4OHToA0KVLl1pv2Gq14unpCYC3tzdNmjSxXwKam5tLWFgYwcHBWCwWysrKyMzMJDIy8hK6ICIitVFt8N96662UlZWRlZXF7Nmza7XhVatWkZ6eDlTc+du2bVv69OmDl5cX119/PRMnTsTNzY2xY8cSExODn58fCxcuvPSeiIiIQxy6gSsxMZEPPviAZs2aNURNdrqBS0Sk9mrKTodO7rZq1YqePXsyZMgQvLy87O160LqIyOXHoeAfPHgwgwcPNrsWERFpADUG/4cffsjRo0fp1q0bN998c0PUJCIiJqr2zt3777+fadOmUVhYyD//+U+effbZhqpLRERMUu3J3e7du7Nt2zZcXV05c+YMN910E998802DFaeTuyIitVen2TmbNGmCq2vFRxr6ih4RETFHtWP83377LQEBAQAYhkFhYSEBAQEYhoGLiwv5+fkNUqSIiNSfaoPfZrM1VB0iItJAHJqWWURErhwKfhERJ6PgFxFxMgp+EREno+AXEXEyCn4RESej4BcRcTIKfhERJ6PgFxFxMqYFv8ViISoqipiYGFJTU9m3bx8xMTHExsYyevRoysrKAAgJCSEuLo64uDi+++47s8oREZFfmBb8ISEhrF+/nnXr1gFw7NgxMjIyyM7OpmPHjnzyyScA+Pv7k5WVRVZWFt26dTOrHBER+YVpwe/h4WF/7enpSfv27fH19QXA3d0dNzc3AIqKioiNjSUtLY2zZ8+aVY6IiPzC1DH+5cuX0717d44cOUKrVq0AyM/PJzMzk4SEBABycnLIzs6mQ4cOzJw508xyREQEk4N/2LBhWCwWAgMDWbFiBVarlZSUFN566y3c3SsmBm3ZsiUAI0eOxGKxmFmOiIhgYvBbrVb7a29vb5o1a8b999/Pgw8+aB/LP3funP1zubm5dOrUyaxyRETkFzU+bP1SrVq1ivT0dAA6d+5MixYt+OCDD9i/fz9Tp07lkUceISoqisTERJo3b46fnx/z5883qxwREflFtc/cbWx65q6ISO3V6Zm7IiJy5VHwi4g4GQW/iIiTUfCLiDgZBb+IiJNR8IuIOBkFv4iIk1Hwi4g4GQW/iIiTUfCLiDgZBb+IiJNR8IuIOBkFv4iIk1Hwi4g4GQW/iIiTuTKDf+JEWLy44vXixRXvRUQEMDH4LRYLUVFRxMTEkJqaimEYTJkyhejoaJKTk7HZbAAsWLCAqKgokpKSKC4urvuOn3++4ic5GUaMqPj9a5uIiJgX/CEhIaxfv55169YBsGnTJtasWUNOTg7h4eF89NFH2Gw2ZsyYQXZ2NmPGjOHNN9+s204nTvzt6L68HJYtq/j9+2UiIk7MtOD38PCwv/b09GTnzp3ExcUBEB8fz4YNG9i5cydhYWG4u7vb2+okJARcL9IlV9eK5SIiTs7UMf7ly5fTvXt3jhw5QmlpKd7e3kDFMyCPHz/OiRMnqrTVyV13wa23XnjZrbdWLBcRcXKmBv+wYcOwWCwEBgbi7u5uH8MvLi7G19cXX1/fKm11sngxZGRceFlGxm8nfEVEnJhpwW+1Wu2vvb29KSsrY+3atQBkZmYSGRlJcHAwFouFsrIye1ud5OX9Nqb/e+XlFctFRJycu1kbXrVqFenp6QB07tyZSZMmcfjwYaKjo7n22mt59NFH8fDwYOzYscTExODn58fChQvrttPnngPDqDiJ6+paMbyTkVER+s89V/EjIuLkXAzDMBq7iIs5efKk/bWPj4/jK06cWHEi9667KoZ38vIU+iLiNGrKzisz+EVEnFhN2Xll3rkrIiIXpeAXEXEyCn4RESdj2lU99e38MSsREbl0OuIXEXEyCn4RESfzh76cU0RE6p+O+EVEnMwVE/z5+fn06tWLpk2bUlpaWmXZwIEDiYqKIjMzs5EqrH/V9XnixIn069ePfv368fnnnzdShfWvuj4DGIZBjx49mDVrViNUV/+q6+/Zs2f585//zMCBA3nooYcaqcL6V12f165dS9++fYmMjGTGjBmNVGH9+89//mN/cNVjjz1WaZkp+WVcIc6cOWMUFRUZ/fv3N2w2W6VlDz30kJGbm2uUlJQY/fv3b5wCTVBdn/fu3WsYhmEcP37ciImJaYzyTEKGLIUAAAg4SURBVFFdnw3DMD766CMjPj7eeOuttxqhuvpXXX8nT55sZGZmNlJl5qmuz7feequxf/9+o6yszLjxxhsbqcL6d/jwYePMmTOGYRjG6NGjjW3bttmXmZFfV8wRf9OmTfHz87vgsm3bttGvXz+aN29OixYtKCkpaeDqzFFdnzt27AhUPATHxcWlIcsyVXV9Bli0aBF/+tOfGrAic1XX36ysLJYvX05cXBzLly9v4MrMU12fQ0NDOXnyJFarFS8vrwauzDxt2rShadOmALi7u+Pm5mZfZkZ+XTHBX52ysjJ7+NXLA18uI88//zxpaWmNXUaDWL16Nf3798fd/bK5PaVO9uzZwy233MLHH3/MpEmTLjj0daUZMWIESUlJdOnSheTk5MYup95t27aNY8eO0a1bN3ubGfnlFMF//rdnvTzw5TLx4YcfUlhYyOjRoxu7lAYxa9YsUlNTG7uMBuPj40P//v3x8vIiKCiIn376qbFLMt0TTzxBTk4Ou3bt4p133uHnn39u7JLqTVFREePHj+ftt9+u1G5GfjnFoVF4eDhffvkl4eHhFBcX2x/3eCXbtm0br732Gh9//HFjl9Jgdu3axYgRIzh06BCGYRAdHU2XLl0auyzTREVFsW3bNnr16sUPP/yAv79/Y5dkOjc3N3x9fWnSpAmurq7YbLbGLqlelJaWcvfddzNlyhTatGlTaZkp+VUvZwr+AM6dO2cMGjTI8PX1NQYOHGhs2LDBGD9+vGEYhnHw4EFjwIABRmRkpLF69epGrrT+VNfnhIQEo3v37kb//v2NYcOGNXKl9ae6Pv9q9uzZV8zJ3er6m5+fbwwePNjo27evMWvWrEautP5U1+eVK1caN954oxEZGWlMnDixkSutPwsXLjRat25t9O/f3+jfv7+xfv16U/NLN3CJiDgZpxjjFxGR3yj4RUScjIJfRMTJKPhFRJyMgl9ExMko+OWyMWHCBEJDQwkLCyMqKopTp06Rn5/PPffcU+dtT5o0iVWrVgHw4osvcu2111a5nvrIkSPExsYSFBTEmDFjKCsrA+DMmTMMGzaMzp07k5iYyKlTpwAoLy/nL3/5C0FBQURFRZGfn1/nOm+//XZ69OjBggULePnll+3tu3fvJiUlpc7bF+eg4JfLwvr161m/fj1bt27l22+/5Z133sHDw4OAgADeeeedOm373LlzrFixgqFDhwIwePBgNmzYUOVzL774IsnJyezevRsPDw+WLl0KwMyZMwkNDWXXrl3069eP1157DYCMjAxOnz7N7t27eeCBB5g0aVKd6iwoKGDPnj1s3bqV5OTkSsEfFBTEsWPHOHjwYJ32Ic5BwS+XhcOHD+Pv72+fhycoKAhPT09++OEHIiMjAbj55puJiIggIiKCpk2bsnXrVkpKSkhOTqZPnz707duXr7/+usq2MzMz6devn/39DTfcQEBAQJXPffzxx/bpL0aPHm2/K3rFihXcfffd1bbfeeedrF69uso2s7KyCAsLIyIigptuugmA06dPM3LkSEJDQxk3bpz9L4+kpCR27txJREQEd955J4WFhURERNin8U1MTOS9996r7T+tOKN6uQ1MxGQnT540unbtaoSHhxv/9V//ZVgsFsMwDGPfvn1G3759K312+fLlxqBBg4yysjLjiSeeMJYtW2YYhmHs2bPnglP5Pvvss8bs2bOrtF9zzTWV3gcEBNhff/fdd8agQYMMwzCMbt26GcePHzcMwzB+/vlno1OnToZhGMbNN99sbN682b5OYGCgUV5eXmmbSUlJxhdffGEYhmGcOHHCMIyK6ZYff/xxwzAMIyMjw/j1v+nv+/r7+nJzc40777yzSj9Efk9H/HJZ8Pb2ZsuWLbz00ktYrVb69evHt99+W+Vz+/fv5+mnn2bBggW4urry+eef8+yzzxIREcFtt912wYnMCgoKaN26db3UWdspsKOionjqqad4/fXXsVqtQMWw1q9TSyclJXHVVVc5tC1/f38KCgpqV7A4JaeYpE2uDE2aNCExMZHExETKy8tZvXo1t99+u325zWZj1KhRTJ8+nWuuuQaoeCLXypUrLzh086umTZvaQ7c6Xl5elJSU0KJFCw4dOkTbtm0BCAgI4NChQ/j6+vLjjz9WaY+IiODs2bM0adKkyhfDM888Q2JiIitWrKBPnz588803GJc4i4rVarXP6S5SHR3xy2UhLy+Pffv2ARUzGebl5dG+fftKn3niiSe4+eabGTBggL0tPj7efrIVKmYt/b0uXbqwd+/eGmtITExk4cKFACxcuJBbbrkFqDi3MH/+/Grb3333XRISEqpsc+/evURERPCPf/yDDh06cPDgQW666SaWLFkCVJwnuNjUwy4uLpSXl9vf7969+4qejVTqj4JfLgunTp1i1KhRhIaGEh4eTkhICHfccUelz0ydOpV3333XfoI3Ly+PCRMmcOjQIcLDw+nWrRsLFiyosu0hQ4awbt06+/v//u//pl27dhw9epR27drx5ptvAvD3v/+d+fPnExQUhNVqtf+1kZaWhsViISgoiPXr1zNu3DgAhg8fTrNmzejUqRNvvPEGzz77bJV9p6en2/sUGhpKjx49GDduHLt376Znz56sXbvW/tfL740ePZru3bvbT+5mZ2czZMiQS/jXFWej2TlFqBhL/7//+z+uvvrqxi6lijZt2tQ4dl9aWsqAAQNYs2aN0zyBTC6dgl+EiiEgm81G7969G7uUKhwJ/v3797Nnzx4GDhzYQFXJ5UzBLyLiZDTGLyLiZBT8IiJORsEvIuJkFPwiIk5GwS8i4mQU/CIiTub/AdtOrR2H7MLzAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "
As described in lecture, the model function for linear regression (which is a function that maps from `x` to `y`) is represented as \n",
+ "\n",
+ "$$ f_{w,b}(x^{(i)}) = wx^{(i)} + b \\tag{1}$$\n",
+ "\n",
+ "The formula above is how you can represent straight lines - different values of $w$ and $b$ give you different straight lines on the plot.
\n",
+ "\n",
+ "Let's try to get a better intuition for this through the code blocks below. Let's start with $w = 100$ and $b = 100$. \n",
+ "\n",
+ "**Note: You can come back to this cell to adjust the model's w and b parameters**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "w: 100\n",
+ "b: 100\n"
+ ]
+ }
+ ],
+ "source": [
+ "w = 100\n",
+ "b = 100\n",
+ "print(f\"w: {w}\")\n",
+ "print(f\"b: {b}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now, let's compute the value of $f_{w,b}(x^{(i)})$ for your two data points. You can explicitly write this out for each data point as - \n",
+ "\n",
+ "for $x^{(0)}$, `f_wb = w * x[0] + b`\n",
+ "\n",
+ "for $x^{(1)}$, `f_wb = w * x[1] + b`\n",
+ "\n",
+ "For a large number of data points, this can get unwieldy and repetitive. So instead, you can calculate the function output in a `for` loop as shown in the `compute_model_output` function below.\n",
+ "> **Note**: The argument description `(ndarray (m,))` describes a Numpy n-dimensional array of shape (m,). `(scalar)` describes an argument without dimensions, just a magnitude. \n",
+ "> **Note**: `np.zero(n)` will return a one-dimensional numpy array with $n$ entries \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def compute_model_output(x, w, b):\n",
+ " \"\"\"\n",
+ " Computes the prediction of a linear model\n",
+ " Args:\n",
+ " x (ndarray (m,)): Data, m examples \n",
+ " w,b (scalar) : model parameters \n",
+ " Returns\n",
+ " y (ndarray (m,)): target values\n",
+ " \"\"\"\n",
+ " m = x.shape[0]\n",
+ " f_wb = np.zeros(m)\n",
+ " for i in range(m):\n",
+ " f_wb[i] = w * x[i] + b\n",
+ " \n",
+ " return f_wb"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now let's call the `compute_model_output` function and plot the output.."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAERCAYAAAB8eMxzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deVhU9f4H8DeboiQMCMqi4oK4IIgLOqDI6oIr+XT1KpnZTazEflnXrFuZZEVeTbMylyhRg7Q0F7TUSBDBcA/Ea4IoLiCKCIKKCMP5/THOOCM4DDCHbd6v5+HROTNz5nMo3xy+53u+HwNBEAQQEZHeMGzsAoiIqGEx+ImI9AyDn4hIzzD4iYj0DIOfiEjPMPiJiPQMg5+apK5duyIlJUVt2yuvvILFixeL9pkuLi44fvy4TvcZFRUFY2NjPPPMM7CwsIBUKsWff/7ZoDUQPYnBT/TI2bNn4eHhofP9BgYG4u7du7h16xZGjBiByZMn48nbZ8rLy0WtgUgVg5+arVWrVqFbt27o0KED5syZg9LSUgDys+wxY8YoX5ednQ1TU1MAQGVlJebOnQtra2vlGbgidFV/y/D19cVHH32EwYMHw8LCAtOmTcPDhw+V+1y8eDFsbGzg7OyM5cuXo2vXrjXWa2JigpkzZyIvLw8FBQXw9fXFokWLMHDgQEgkkio13Lt3D6+99hocHBxgaWmJGTNmKPe1bds2uLi4wMrKChMnTsTNmzcBADdu3MCoUaNgYWGB9u3bIywsrK7fXmrBGPzULO3fvx/Lly/HgQMHkJmZiaysLCxZsqTG9x04cADHjh3DpUuXcPv2bXzxxRcwNKz+n8HPP/+MX375BZcvX0ZaWhq2bt0KAIiNjcXGjRtx/PhxHDlyBNu2bdOq5ocPHyIqKgoODg6wtrYGAPz444/Ytm0bbt26VeX1b7zxBq5cuYLU1FTcvHkTc+bMAQAcP34cb775JrZs2YIbN26gd+/eePXVVwEAK1euRM+ePVFQUIBr166p/bAgUmDwU5M1cuRISCQS5deGDRuUz23duhVz5sxBz549YWFhgUWLFmHLli017tPExAQlJSXIyMiAoaEhpFIpjIyMqn3t7Nmz0aVLF0gkEowbNw6pqakAgO3bt2P27Nno2rUrrK2t8frrr2v8zD/++AMSiQQODg44fvw4duzYofYZ3bt3R5s2bdTeU1lZic2bN+PLL7+EtbU1TExMMHz4cADAd999h7CwMLi6usLExASLFi3C7t27UVFRARMTE1y/fh05OTlo06YNhg4dWuP3hPQPg5+arN9//x1FRUXKr1mzZimfy83NRefOnZWPHR0dcf369Rr3GRAQgNmzZ2PmzJlwcHDQeLG4Q4cOyr+3bdsWd+/eBQDk5eWhU6dOyuccHBxq/MyioiLk5+cjISFBbQxf9RhU5efn4+HDh9UOIV25cgXh4eHKH4idOnWCsbEx8vLysGDBAjg4OMDLywv9+vXD9u3bNdZG+onBT82Svb09rl69qnx85coV2NnZAQDMzMxw//595XM3btxQe+9bb72F9PR0JCYm4rvvvsPvv/9eq8+2tbVFTk6O8rHq32vLwMCg2u02NjZo1aoVLl++XOU5BwcHfPrpp2o/FEtLS9GpUyeYm5vjq6++Qk5ODj7//HNMnz5d+QOLSIHBT83SP/7xD6xfvx4XLlxAcXExlixZgilTpgAA3NzccPLkSZw/fx4lJSX47LPPlO87ceIETp48CZlMBnNzcxgbGz91qOdpJk+ejG+//RaXL19GQUEBvv76a50eGwAYGhrihRdewP/93/+hoKAA5eXlSE5OBgDMmjULX375pXLo6fbt29i1axcA4Ndff0V2djYAwNLSEgYGBrU+Pmr5GPzULAUFBWH+/PkICAiAk5MTHB0dsWjRIgBAr1698Pbbb8PT0xP9+/fH6NGjle+7c+cOZs6cCQsLC7i5uWHatGnw9/ev1WdPnDgRzz//PAYNGoShQ4di3LhxaN26tU6PDwBWrFgBe3t7uLi4oGPHjli/fj0AwMvLC8uWLcMLL7wAc3NzDBw4UPlD4e+//8aIESPwzDPPICQkBBs3bqxy/YDIgOvxE9XPpk2bEBUVhYMHDzZ2KURa4Rk/UR3s2rULFRUVuHLlCpYvX45JkyY1dklEWuMZP1Ed+Pj44NSpUzAzM8OUKVOwfPlytGrVqrHLItIKg5+ISM9wqIeISM8YN3YBmty5c6exSyAiatYsLCyqbOMZPxGRnmHwExHpmSY91KOqul9XiIioqpqGyXnGT0SkZ5rNGb9CWVkZ7ty589TFraj2BEGAmZkZzMzMGrsUImoAzS7479y5A2tr66c2z6C6uXnzJoOfSE80u/Q0MDBg6IuAv0ERNSHh4YCisdCWLfLHOiRagmZnZ6Njx47w9fXFqFGjAADLli3D8OHDERISouxzGh0dDS8vL4wfPx7FxcVilVMn+/fvh7e3N3x9ffHmm29CJpPVeh8WFhbw8/PD8OHDkZGRUav3JiQkKBuFzJs3r9rXZGdnKxcHy8vLwyeffFLrGomoCVm8WP4VEgIEB8v/VGzTEVFPnUeOHImEhAQcOHAA+fn5iI+PR1JSEtzc3LBz506Ul5dj7dq1SExMxIwZM7Bu3Toxy6mVW7du4ZNPPsG+ffuQkJAAGxsbfPvttzW+r7KyUu2xq6sr4uPj8fnnn+O///2vxtdq8tVXX1W7XTX4bW1t8d5772m9TyJqYsLDH5/dV1YCu3bJ/3zyuXoSNfjj4+Ph7e2NlStX4tixY/D19QUABAYGIiUlBRkZGXB1dYWxsbFyW20YGNTvS5O9e/dixowZynHv+fPnK3ulKnqfZmdn48UXXwQASKVSvPrqq/j3v/9d7f7c3d1x7do1REVFYerUqRg3bhzS0tLw0UcfwdfXF/7+/soGGi+99BICAwOxefNm5fsVn5mcnIxhw4bBz88PW7duxfr167F582YEBAQgOzsbzz//PADghx9+gFQqxbBhw5QNOzw9PREWFgZ3d3fs27evVt9rImoAvXoBTxvKNjSUP68Dol3ctbOzQ0ZGBlq3bo1JkyahuLgYHTt2BCAf/igsLERRURHMzc3VtjUV169fh6urq/KxqakpHj58+NTX37p1C++9955aL1ZViYmJ6PXoP5pEIsHWrVtx5swZ5OTkICEhAefOnUNERAT+9a9/wcjICHFxcfj000+rfOY777yDXbt2wdraGpWVlejYsSO6d++Ojz/+WPmDQyaT4csvv0RycjJycnIQFhaGPXv2oKCgAIsWLUJ5eTnCwsIwZsyYen6XiEin/vlP+Zj+o45qaiZMkD+vA6IFf+vWrZVdicaPHw9zc3Nlb9Li4mJlo2jFuL5iW1NhZ2eH3Nxc5eMHDx7AxMRE7TWqC5t26NCh2tA/c+YM/Pz8IJFI8M0332D//v0YNGgQAODcuXNISEhQ/iZkZ2eHixcvYsCAAQCAQYMG4c8//6yyT2trawB46kXu/Px8ODo6wsTEBF27dlXezGFjY6NsIF5UVKTV94GIGtCWLUBsbPXPxcbKn9dB+Is21FNSUqL8e3JyMpycnHDo0CEAQFxcHKRSKZydnZGeng6ZTKbc1lQEBQVh06ZNuHfvHgBg5cqVCA4OBiD/IQDIQ13haSGsGOPfsWOHshm44rW9evXCqFGjkJCQgISEBGzatAndunVTDs2cPn26yv4MDAxQUFAAQH6NwMTEpMpFZxsbG2RnZ6O8vBzZ2dnKu55VZ+5wNW6iJuj8+cdj+k+qrJQ/rwOiBf/hw4cxaNAgeHl5wd7eHkOHDsWIESMwfPhw/PXXXwgODoaJiQlmz54Nb29vbNy4EXPmzKnVZwhC/b406dChA959912MGTMGPj4+uHHjBkJDQwEA48aNw/Dhw5U/yOqqf//+sLW1ha+vL/z8/LBhwwYMHToUZWVlCAgIqHYWUEREBCZMmAA/Pz/8/PPP6NevH5KTkzF16lTla4yMjBAWFgZvb29Mnz4dS5YsqVedRNRAPvxQ/gXIx/QnTXo85q/6XD016UYsqutNKM5a8/PzYWNj01gltVj8vhI1IeHh8gu5ijH/8+drFfrVZaeqZnfnLhFRi6ca8jq6oKuKt8ASEekZBj8RkZ5h8BMR6RkGPxGRnmHw18DJyQlbFKvkVeOvv/7CqVOnarXPhIQEvP/++8rHxcXFyiUZFLy8vFBaWlrlvS+++CIuXLhQq88jIlLVMoNfR0uapqamwtvbG7FPu5MOdQv+J5mbm8PCwgLXrl0DAFy6dAl2dnZo06ZNvfZLRFSdlhf8OlzS9JdffsFrr72G+/fvo6ysDACwc+dOSKVS+Pn54dChQ1i/fj2WLVuGkJAQREVFITIy8lEZi5GQkIDc3FzlssyvvfbaUz/r2Wefxc6dOwEAO3bswLPPPosNGzbA19cXgwcPxoEDB544zMWIi4sDIP8tIDs7G6WlpZg2bRr8/f0xdepUlJeXY8eOHRgyZAj8/f3x66+/1vp7QEQtT8sKfh0vaXrq1Cl4eHhgzJgxiIuLQ2VlJT755BPEx8crVx4NDQ3FggULEB0dXe0+rK2t8fvvvyMpKQnFxcXIzMys9nWTJk3Cnj17AAB79uzB+PHjMXXqVCQkJOCPP/7A8uXLa6w3MjISEydOxMGDB+Hr64tt27Zh+/bt+Omnn3Dw4EEEBQXV6viJqGVqWTdwKZY0rW6ti1ouaZqVlYX09HSMGTMGZWVlcHZ2xuDBg+Ho6KgcgnlyfZ7q1sIpKCjAq6++iqKiImRnZ6st/KbKxsYGFRUVOH/+PFq3bg2JRIIdO3Zg1apVEAQBN2/erPGzzp07h5MnT2LdunV48OABpk2bhvfffx8ff/wxKioq8N5776Fnz55afw+IqGVqWWf8//ynfOnS6tRySdPt27cjMjIS+/btQ3x8PK5fvw5ra2tcuXJFuUjbk4ukWVhY4Pr16wAeL+AWExOD4OBgJCQkYNiwYRoXR5swYQLmzJmDiRMnApCvy/Pbb79h165dVX7IKD5LEAScPXsWgHzRt7fffhsJCQlISUnBa6+9BkdHR0RGRiI0NBQrVqzQ+viJqOVqWcGvzZKmWtq7dy+8vLyUj/v27YukpCS8++678PHxgb+/Pw4fPgypVIoffvgB8+bNQ0BAAPbt26cMbgDw9/fH559/juDgYOVKn08zefJkHD58GJMmTQIgX856xIgReO+996osWT158mR88cUXmDJlCiwtLQEAoaGh2LFjBwICAuDv749Tp05h8eLF8PHxwbx589QWciMi/dWyFmkLD9d8EXfxYp2tbtfScJE2opajpkXaWtYZfwMtaUpE1Jy1rIu7gPys3sCgXkuaEhG1ZC0v+AHRlzQlImrOmt1QjyAIqHxaazKqsyZ8qYeIdKzZnfFLJBLcunVLbR471Y8gCDAzM2vsMoiogTS74G/VqhU6dOjQ2GUQETVbzW6oh4iI6ofBT0SkZxj8RER6hsFPRKRnGPxERHqGwU9EpGdED/4VK1Yo+8laWFjA19cXvr6+uH37NgAgOjoaXl5eGD9+PIqLi8Uuh4hI74ka/GVlZUhNTVU+dnV1RUJCAhISEmBlZYXy8nKsXbsWiYmJmDFjBtatWydmOUREBJGDPzIyEjNnzlQ+PnfuHLy9vfHOO+9AEARkZGTA1dUVxsbGCAwMREpKipjlEBERRAz+8vJyHDp0CP7+/sptmZmZSExMRGFhIWJjY1FUVARzc3MA8mGgwsJCscohIqJHRAv+zZs3Y/r06WrbrKysYGBggODgYKSnp0MikSjH9YuLi6t0mSIiIt0TLfjPnz+PNWvWYMyYMTh79ixWrVql7E2bnJyMHj16wNnZGenp6ZDJZIiLi4NUKhWrHCIiekS0RdqWLl2q/Pvw4cPh4+MDDw8PmJmZoXv37ggPD4eRkRFmz54Nb29vWFpaIiYmRqxyiIjokWbXc5eIiDTTr567RERUoxqHesrKyvDbb7/h6NGjyMvLg6mpKXr37o1Ro0ahT58+DVEjERHpkMahnkWLFuH333+Ht7c3BgwYABsbG5SVleHChQs4fPgw7t69i88++wzu7u6iFMehHiKi2qspOzUGf3x8PPz8/J6688LCQly9ehVubm71LLN6DH4iotqrV/BXR+ywV8XgJyKqPZ1c3PX29kZJSQny8/MxYMAAvP7665g7d67uqiQiogajVfDfu3cP7dq1w7Zt2zB79mwkJCQgOTlZ7NqIiEgEWgV/RUUFDh06hKioKEyYMEG5jYiImh+tgv+rr77C559/jmeffRZubm64dOmSxou+RETUdNV4cVcmk+Hll1/Ghg0bGqomJV7cJSKqvXpf3DUyMkJeXh5KS0t1WxkRETUKrRZpa9++PQYMGIDRo0fDzMxMuf3TTz8VrTAiIhKHVsE/cuRIjBw5UuxaiIioAXB1TiKiFkYnN3Clp6dj7Nix6NatG+zt7ZVfRETU/Gg11PPyyy9jzZo1eOmll5CUlIQNGzagqKhI7NqIiEgEWt/ANWDAAMhkMpiZmSEsLAy//fab2LUREZEItDrjb9u2LcrLy+Hm5obw8HDY29vj7t27YtdGREQi0OqMPyoqChUVFVi9ejUEQcC5c+ewfft2sWsjIiIRcFYPEVELU1N2ahzqsbOzg4GBAQRBqPbP3Nxc3VdMRESi0hj8169fb6g6iIiogWgM/gMHDmh886hRo3RaDBERiU9j8P/4449Pfc7AwIDBT0TUDIl+cXfFihX45ZdfkJSUhGXLlmHXrl1wdHREVFQUTExMEB0djdWrV8PKygoxMTEwNzdXvpcXd4mIak8nSzZcuHABo0aNQufOndG5c2eMHj0aWVlZNb6vrKwMqampAID8/HzEx8cjKSkJbm5u2LlzJ8rLy7F27VokJiZixowZWLdunbbHRUREdaRV8M+aNQvz58/H1atXcfXqVcyfPx8vvvhije+LjIzEzJkzAQDHjh2Dr68vACAwMBApKSnIyMiAq6srjI2NlduIiEhcWgV/cXExgoKClI/HjBmD4uJije8pLy/HoUOH4O/vDwAoKipSDuNYWFigsLCw2m1ERCQurZZscHFxwVtvvYVp06bBwMAAW7ZsgYuLi8b3bN68GdOnT1c+lkgkyMnJASD/QSKRSCCRSJQ/QBTbiIhIXFqd8X/33XewtbXFJ598giVLlqBDhw747rvvNL7n/PnzWLNmDcaMGYOzZ8/ixIkTOHToEAAgLi4OUqkUzs7OSE9Ph0wmU24jIiJxNciSDcOHD0dSUhKWLl2K2NhYdOnSBVFRUWjVqhU2b96MNWvWwNLSEjExMWpXoDmrh4io9mrKTo3B361bNxgYGDx15xcvXqxneZox+ImIaq9ea/X8/fffEAQBn3zyCWxtbTF16lQIgoCtW7fWeHGXiIiaJq2Gevr376+cj68wZMgQHDt2TLTCAJ7xExHVhU5u4Grfvj2++eYbFBQUoKCgAKtXr4aZmZnuqiQiogajVfBv2bIFZ8+eRWBgIAIDA3Hu3Dls3bpV7NqIiEgEbMRCRNTC6GSoh4iIWg4GPxGRntEY/NOmTQMA/Pe//22QYoiISHwa5/GnpqbizJkz2LhxI5599lk8eTnA2dlZ1OKIiEj3NF7c3bJlCzZu3IikpCQMHjxY/Y0GBjh48KCoxfHiLhFR7dVryQaFpUuXYuHChbqtTAsMfiKi2tNJ8N+/fx+rVq1CUlISAPmia2+88QbatGmjw1KrYvATEdWeTqZzvvTSSygqKsJnn32GiIgIFBcXY9asWbqrkoiIGoxWjVj+/vtvbNmyRfnYzc0N7u7uohVFRETi0eqM39zcHHv27FE+3rt3r7JlIhERNS9ajfFnZWXh9ddfx19//QVDQ0P0798fq1atQo8ePUQtjmP8RES1p5OLu42FwU9EVHtcq4eIiNQw+ImI9AyDn4hIz2gV/G+++SZKSkpQVlYGPz8/2NnZYcOGDWLXRkREItAq+OPj49GuXTvs2rULffr0QVZWFr744guxayMiIhFoFfwPHjxAWVkZtm3bhueeew5t27YVuy4iIhKJVsE/Z84cdO7cGSUlJfDz88OVK1fQrl07sWsjIiIR1Hkef0VFBYyNtVrxoc44j5+IqPZqyk6NyS0IArZt24a9e/ciNzcXAGBnZ4dx48bhueee0/jB6enpCA0NhZGREZycnPD9999DIpFgwIABAIBffvkFVlZWiI6OxurVq2FlZYWYmBguBUFEJDKNZ/wzZ86EsbExpk+fDgcHBwBATk4OYmJi8PDhQ2zevPmpOy4vL4eJiQkAYNasWXjttdcwf/585dLOitf4+/sjPj4e27dvx5UrV7BgwQLl8zzjJyKqvXqd8f/555/IyMhQ29a7d28EBATU2HZREfoA0Lp1a3Tu3Bnnzp2Dt7c3hg0bhoiICGRkZMDV1RXGxsYIDAxEaGioVgdFRER1p/Hirp2dHSIjI3Hv3j3ltnv37mH9+vWwtbWtcee7d+9Gv379cPPmTbRv3x6ZmZlITExEYWEhYmNjUVRUpBzasbCwQGFhYT0Ph4iIaqIx+Ldt24b//e9/GDhwIDp16oROnTph0KBBOHfuHLZt21bjzidOnIj09HQ4ODhgz549sLKygoGBAYKDg5Geng6JRILi4mIAQHFxMSQSiW6OioiInkrjUI+NjQ1WrFiBFStW1HrHZWVlaN26NQD5ev6tWrWCTCaDkZERkpOT4erqCmdnZ6Snp0MmkyEuLg5SqbRuR0FERFqrcT7m8ePHq53VM2TIEI3v27dvn/IHRs+ePWFnZwcPDw+YmZmhe/fuCA8Ph5GREWbPng1vb29YWloiJiZGB4dERESaaJzV88477+DEiROYNm2a2qyeH3/8EYMGDcLSpUtFLY6zeoiIaq9ejVicnZ2rzOoB5PP7nZ2dkZmZqaMyq8fgJyKqvXo1YmnXrh3i4uKqbI+Li+OSDUREzZTGMf4tW7Zg4cKFmDVrFmxsbCAIAm7duoXBgwfjxx9/bKgaiYhIh7Raq0cQBBQUFAAA2rdvDwMDA9ELAzjUQ0RUF/W6cxcA8vLysG/fPrVZPWPGjIGdnZ0OyyQiooaicYz/yy+/xMiRI5GVlQUrKytYWVnh4sWLGDVqFL788suGqpGIiHSoxlk9Z8+eVVt3B5DfnOXq6lrtjB9d4lAPEVHt1WtWj4mJCS5evFhl+6VLl0Rfi5+IiMShMb0jIyMREhICIyMj5Q1c165dQ2VlJSIjIxukQCIi0i2tZvXk5uYqL+7a29vD3t5e9MIADvUQEdVFvWf1ANWHfXp6Ovr161fP8oiIqKFp1Wy9OmPHjtVlHURE1EA0nvFPnz692u2CIOD27duiFEREROLSGPz79u3D5s2b8cwzz6htFwQBBw8eFLUwIiISh8bgDwgIQLt27TBixIgqz3l4eIhWFBERiUerWT2NhbN6iIhqr143cBERUcvD4Cci0jMMfiIiPaP1gjulpaW4fv06KioqlNucnZ1FKYqIiMSjVfB/+umnWLt2Lbp3765swmJgYMApnUREzZBWwb9x40acP38ebdq0EbseIiISmVZj/L169UJZWZnYtRARUQPQ6oz/4cOH6NOnD7y8vNC6dWvl9piYGNEKIyIicWgV/O+++67YdRARUQPRGPwVFRUwNjaGp6dnrXecnp6O0NBQGBkZwcnJCd9//z2WL1+OXbt2wdHREVFRUTAxMUF0dDRWr14NKysrxMTEwNzcvM4HoxQeDvTqBfzzn8CWLcD588CHH9Z/v0RELYDGMf7g4GAA8jH+3r17K78UjzXp1asXjhw5gsOHDwMATpw4gfj4eCQlJcHNzQ07d+5EeXk51q5di8TERMyYMQPr1q2r/xEtXiz/CgkBgoPlfyq2ERE1cUVFwIEDwJUr4n2GxjP+PXv2AJD32K0t1QbtrVu3RkZGBnx9fQEAgYGBiImJQd++feHq6gpjY2MEBgYiNDS01p+jJjxc/gUAlZXArl3qzxkY8MyfiJqMigrgzBkgJQU4elT+9fff8ue++AL4v/8T53M1Bv+BAwcwatSopz5/69YtXL16FQMGDKj2+d27d+M///kPnJ2dUVFRoRzGsbCwQGFhIYqKiqpsq5devQBDQ3noP8nQUP48EVEjuXZNHu6KoD9xAigtrf61R4+KV4fG4D927Bg+/PBDeHp6wt3dHdbW1igrK8PFixeRmJgIQRDw6aefPvX9EydOxMSJEzFv3jwYGxujuLgYAFBcXAyJRAKJRFJlW70oxvRVz/QVJkyQP09E1ADu3QNOnnwc8ikpwKPW5VpJSRGvNo3B//7772PhwoWIi4tDSkoKkpOTYWpqit69e2PlypXo3r37U99bVlamnPppbm4OmUyGQ4cO4e2330ZcXBykUimcnZ2Rnp4OmUym3FYvW7YAsbHVPxcbK3+e4U9EOlZZKZ9Dojpkc+YMIJPVfl8GBkC/fsDQofKhIGOtF9bRXo27NDExQVBQEIKCgmq143379mHFihUAgJ49e2LJkiW4fv06hg8fji5duuCNN96AiYkJZs+eDW9vb1haWtb/voDz56sf5gEe/5chIqqnW7fUh2yOHQNUlsCvlY4dAalU/jV0KDB4MNCunW7rfVLLa8SyeLH8Qq6hoXx4JzZWHvoffsiZPURUa2VlQGqq+pDNxYt125epKTBwoDzgFUHfpYv8LF+XaspOEX6JaGSLF8u/i5zHT0S1JAhAdrZ6yJ8+DTx8WLf99eypHvJubkCrVjotuU5a3hk/EZGW7twBjh9/PC6fkgLk59dtX5aWwJAhj0N+yBCgfXvd1qstnZzxnz59Gq+++iry8/ORlZWFM2fOICYmBhEREbqrlIhIRDIZcPas+tn8uXPys/zaMjaWn70rQn7oUPnZvWEzaW2l1Rm/p6cnYmJiMHnyZJw+fRoA0K9fP6Snp4taHM/4iaiucnPVz+RPnJBPsayLzp3Vh2wGDgTattVtvbqkkzP+yspKdOvWTW2bkZFRPUsjItKN0lL5nHnVmTZXr9ZtX2Zm8pk1qmfz9va6rbexaRX8PXv2VC7fkJeXh6+//hoDBw4UtTAioupUVgKZmepn82lp8jnvtWVgAPTpox7yLi7izJ1vSpnTC4kAABKQSURBVLQa6rl79y6WLFmCP/74A4B8rZ0PPvgAZmZmohbHoR4iKiiQz5NXhPyxY0BdV3exsVEPeQ8PoCVGS03ZqVXwy2SyKkM71W3TNQY/kX55+FB+9q46ZJOZWbd9tWr1eM68Yny+a1fdz5lvinQyxu/t7Y19+/YpF1S7c+cOxo4di+TkZB2VSUT6RhDkSw+rhvypU8CDB3XbX/fuj8/mpVKgf39ApWEgqdAq+O/fv6/WIMXCwgL36np5nIj0UkmJfGaNatDn5dVtX+bm6mfyQ4bIh3FIO1oFv0QiQWJiIkaMGAEASExM1E2nLCJqkWQy+Rx51ZA/e/bpS2lpYmgonzOvGvSKFdipbrQK/jVr1mDmzJm4e/cuAKBdu3bYuHGjqIURUfNx44Z6yB8/Lj/Drwt7e/Uhm0GD5FMsSXdqtWRDyaP/ku3EXjruEV7cJWp6HjyQr1+jugRxdnbd9tWmjXzOvOrNUZ066bRcvVSvi7tff/01wsLC8O6778KgmkvhmpqwEFHzJwhAVpb62fxffwHl5XXbX+/e6iHfrx+g0qWVGojG4Hd0dASAGhurE1HLUFQknyevejZfUFC3fbVvrz4u7+EhX8iMGp/G4J8wYQJkMhkSEhKwYcOGhqqJiBrAk42+U1Lq3qvIxARwd1c/m+/RQz/mzDdHNV7cNTIyQl5eHkpLS9GmTZuGqImIRHDtmvqZvKZG3zXp2lU95AcMkDcZoeZBq1k97du3x4ABAzB69Gi1ZRo4xk/UNN2793jOfF0afatq104+TKO61EHHjrqtlxqWVsE/cuRIjBw5UuxaiKgOnmz0nZICpKfXrdG3oaF8kTLVs/k+fQAuxtuy1Bj8O3bsQH5+Pvr27YuxY8c2RE1EpEF+vvqZ/PHjdW/0bWurHvIN0eibGp/G4A8NDUVmZiakUik++ugj/Pnnn1iyZElD1Uak98rK5NMnVYO+vo2+VYdsxGj0TU2fxhu4+vXrh7S0NBgaGqK0tBTDhg3DqVOnGqw43sBF+kSMRt+qId9UGn2T+Op1A1erVq1g+GhBDM7oIdIt1UbfirBvCY2+qenTGPxnzpyB/aOeY4IgoKCgAPb29hAEAQYGBsit6zQBIj1TUSFfpEw15HXV6FsqlZ/dc8iGtKUx+Mvrel82kZ7TdaNv1SGbpt7om5o+0TpLHj16FPPnz4eRkREGDx6MlStXwsLCAgMGDAAA/PLLL7CyskJ0dDRWr14NKysrxMTEcLlnanbu35c3ENFVo28Pj8ch3xIbfVPjEy34HR0dcfDgQZiamiIkJARnzpyBq6srEhISlK8pLy/H2rVrkZiYiO3bt2PdunVYsGCBWCUR1Ztqo29FyOuq0bdUCvTt2/IbfVPjE+1/MVtb28cfYmwMIyMjnDt3Dt7e3hg2bBgiIiKQkZEBV1dXGBsbIzAwEKGhoWKVQ1QnikbfipDXVaNvqVQ+Z56T1agxiH5ukZaWhlu3bqFv377IzMyEpaUlXnnlFcTGxqJ9+/bKoR0LCwsU1vVfFJEOsNE36QtRg//27dsICwvDTz/9BACwsrICAAQHB+P06dOYNGkSiouLAQDFxcWQSCRilkOkVF2j75Mn5TdM1UWPHuohz0bf1JSJFvwVFRV4/vnnsWzZMtja2uLevXswNTWFkZERkpOT4erqCmdnZ6Snp0MmkyEuLg5SqVSsckjPKRp9q65OWddG3xYW8nnybPRNzZVowf/zzz/j+PHjWLhwIQAgIiICc+fOhZmZGbp3747w8HAYGRlh9uzZ8Pb2hqWlJWJiYsQqh/SIotG3asjrotG3Ynyejb6puatVz92GxiUbSBtiNPpWhDwbfVNzVK8lG4iamicbfaekAJcv121fbPRN+orBT02WotG36pANG30T1R+Dn5qMwkL5PHlFyOui0bci5Nnom+gxBj81CjEafauuZ8NG30RPx+CnBqHa6DslRT5nno2+iRoHg590TrXRtyLs2eibqOlg8FO9VFYCf/+tvgRxfRt9q4Y8G30T6R6Dn2rlyUbfx44Bj1bdqDVbW/WQZ6NvoobB4KenUm30rRiy0VWjb6lU3mCEF2CJGh6DnwDI58xfuqQe8rpq9C2Vypc94Jx5oqaBwa+ndN3oW7VjFBt9EzVtDH49oOtG3/37qy9BzEbfRM0Lg78FUjT6VoS8rhp9S6Xycfo2bXRbLxE1LAZ/M6do9K26no0uGn0rwt7OTrf1ElHjY/A3I4pG36ohn5patznzBgbyxt6qQzZs9E2kH/jPvAl7stH30aNAUVHd9tWhg3rIe3gAj9odE5GeYfA3EYpG36ohr4tG34ohGzb6JiIFBn8jUDT6Vg15XTT6VoQ8G30TkSYM/gZQUvJ4zrxits2NG3XbFxt9E1F9Mfh17MlG3ykp8jn0dZkzb2QEuLqy0TcR6RaDv57y8tTP5I8fB+7erdu+HBzUQ56NvolIDAz+WnjwQD5nXjXo69voW3V1Sjb6JqKGwOB/CkEALlxQD/nU1Po1+lYNeVdXzpknosbB6HlEtdG3Yp35+jT6Vg35IUMAiUS39RIR1ZVeBn95ubzRt+p6Nrpq9C2VAt27c848ETVdogX/0aNHMX/+fBgZGWHw4MFYuXIlli1bhl27dsHR0RFRUVEwMTFBdHQ0Vq9eDSsrK8TExMBchNtJdd3oWzXk3d3Z6JuImhcDQajLRMOa5eXlQSKRwNTUFCEhIXjllVcQERGBX3/9FUuXLkX37t0RHBwMf39/xMfHY/v27bhy5QoWLFig3MedO3eUf7ewsKjV59+7B8yYUf9G34o582z0TUTNRU3ZKdoZv62t7eMPMTZGWloafH19AQCBgYGIiYlB37594erqCmNjYwQGBiI0NFRnn9+2LXDkiPY3ShkaAv36qa9n07s3G30TUcsj+hh/Wloabt26BYlEAqNHKWphYYHCwkIUFRUph3YU23TFwEAe4Lt3V/+8aqNvqVQ+tfKZZ3T28URETZaowX/79m2EhYXhp59+wsmTJ5GTkwMAKC4uhkQigUQiQXFxsdo2XVIEv6mp/GYo1Zuj2OibiPSVaDf/V1RU4Pnnn8eyZctga2sLDw8PHDp0CAAQFxcHqVQKZ2dnpKenQyaTKbfp0owZ8u5TxcVAUhLw+efAP/4BdOnC0Cci/SXaGf/PP/+M48ePY+HChQCAiIgIjBgxAsOHD0eXLl3wxhtvwMTEBLNnz4a3tzcsLS0RExOj0xo6d5Z/ERHRY6LN6tGF+szqISLSVzVlJ9d5JCLSMwx+IiI9w+AnItIzDH4iIj3TbBZpU71YQUREdcczfiIiPcPgJyLSM016Hj8REekez/iJiPRMiwn+3NxcDBw4EKampqioqKjynL+/P7y8vBAXF9dIFeqepmMODw+Hp6cnPD098ccffzRShbqn6ZgBQBAE9O/fH5GRkY1Qne5pOt4HDx7gX//6F/z9/TFv3rxGqlD3NB3zoUOHMHToUEilUqxdu7aRKtS9o0ePwsvLC97e3pg/f77ac6Lkl9BClJaWCrdv3xZ8fHyE8vJytefmzZsnJCcnCyUlJYKPj0/jFCgCTcd88eJFQRAEobCwUPD29m6M8kSh6ZgFQRB27twpBAYGCt9++20jVKd7mo536dKlQlxcXCNVJh5NxzxhwgTh8uXLgkwmE4YMGdJIFere9evXhdLSUkEQBGH69OlCWlqa8jkx8qvFnPGbmprC0tKy2ufS0tLg6emJZ555Bu3atUNJSUkDVycOTcfcrVs3AEDr1q1h0IKWItV0zADw448/YurUqQ1Ykbg0HW9CQgJ2794NX19f7H5a44lmSNMxu7i44M6dOygrK4OZmVkDVyYeW1tbmD7q4WpsbKzsXQKIk18tJvg1kclkyvDTdcOXpm7x4sWYM2dOY5fRIPbv3w8fHx8YGzeb21PqJSsrC+PGjcPevXuxZMmSaoe+Wprg4GCMHz8evXv3RkhISGOXo3OKxlV9+/ZVbhMjv/Qi+FV/eorR8KWp2rFjBwoKCjB9+vTGLqVBREZGYtasWY1dRoOxsLCAj48PzMzM4OTkhBva9hltxhYsWICkpCRkZmZi06ZNuH//fmOXpDOKxlXfffed2nYx8ksvTo3c3Nzw559/ws3NDcXFxcp2jy1ZWloaVq9ejb179zZ2KQ0mMzMTwcHByMnJgSAIGD58OHr37t3YZYnGy8sLaWlpGDhwILKzs2FjY9PYJYnOyMgIEokErVq1gqGhIcrLyxu7JJ14snGVKlHySydXCpqAhw8fCgEBAYJEIhH8/f2FlJQUISwsTBAEQbh69arg5+cnSKVSYf/+/Y1cqe5oOuZRo0YJ/fr1E3x8fISJEyc2cqW6o+mYFTZs2NBiLu5qOt7c3Fxh5MiRwtChQ4XIyMhGrlR3NB3zb7/9JgwZMkSQSqVCeHh4I1eqOzExMYK1tbXg4+Mj+Pj4CEeOHBE1v3gDFxGRntGLMX4iInqMwU9EpGcY/EREeobBT0SkZxj8RER6hsFPzcaiRYvg4uICV1dXeHl54e7du8jNzcULL7xQ730vWbIE+/btAwBERESgS5cuVeZT37x5EyNGjICTkxNmzJgBmUwGACgtLcXEiRPRs2dPBAUF4e7duwCAyspKvPzyy3BycoKXlxdyc3PrXedzzz2H/v37Izo6GsuXL1duv3DhAmbOnFnv/ZN+YPBTs3DkyBEcOXIEqampOHPmDDZt2gQTExPY29tj06ZN9dr3w4cPsWfPHowZMwYAMHLkSKSkpFR5XUREBEJCQnDhwgWYmJhg27ZtAID169fDxcUFmZmZ8PT0xOrVqwEAsbGxuHfvHi5cuIBXXnkFS5YsqVedeXl5yMrKQmpqKkJCQtSC38nJCbdu3cLVq1fr9RmkHxj81Cxcv34dNjY2ynV4nJyc0Lp1a2RnZ0MqlQIAxo4dC3d3d7i7u8PU1BSpqakoKSlBSEgIPDw8MHToUJw8ebLKvuPi4uDp6al8PHjwYNjb21d53d69e5XLX0yfPl15V/SePXvw/PPPa9w+ZcoU7N+/v8o+ExIS4OrqCnd3dwwbNgwAcO/ePTz77LNwcXHB3Llzlb95jB8/HhkZGXB3d8eUKVNQUFAAd3d35TK+QUFB+Pnnn2v7rSV9pJPbwIhEdufOHaFPnz6Cm5ub8O9//1tIT08XBEEQLl26JAwdOlTttbt37xYCAgIEmUwmLFiwQNi1a5cgCIKQlZVV7VK+H3zwgbBhw4Yq2zt27Kj22N7eXvn3//3vf0JAQIAgCILQt29fobCwUBAEQbh//77Qo0cPQRAEYezYscLp06eV73FwcBAqKyvV9jl+/Hjh4MGDgiAIQlFRkSAI8uWW33rrLUEQBCE2NlZQ/DN98lifrC85OVmYMmVKleMgehLP+KlZMDc3x19//YXPPvsMZWVl8PT0xJkzZ6q87vLly3jnnXcQHR0NQ0ND/PHHH/jggw/g7u6OyZMnV7uQWV5eHqytrXVSZ22XwPby8sLChQvxzTffoKysDIB8WEuxtPT48ePRtm1brfZlY2ODvLy82hVMekkvFmmjlqFVq1YICgpCUFAQKisrsX//fjz33HPK58vLyzFt2jR8/fXX6NixIwB5R67ffvut2qEbBVNTU2XoamJmZoaSkhK0a9cOOTk5sLOzAwDY29sjJycHEokE165dq7Ld3d0dDx48QKtWrar8YHj33XcRFBSEPXv2wMPDA6dOnYJQx1VUysrKlGu6E2nCM35qFs6fP49Lly4BkK9keP78eXTu3FntNQsWLMDYsWPh5+en3BYYGKi82ArIVy19Uu/evXHx4sUaawgKCkJMTAwAICYmBuPGjQMgv7bwww8/aNz+008/YdSoUVX2efHiRbi7u+P999+Ho6Mjrl69imHDhmHr1q0A5NcJnrb0sIGBASorK5WPL1y40KJXIyXdYfBTs3D37l1MmzYNLi4ucHNzQ69evfCPf/xD7TWrVq3CTz/9pLzAe/78eSxatAg5OTlwc3ND3759ER0dXWXfo0ePxuHDh5WPP/74Y3Tq1An5+fno1KkT1q1bBwD4z3/+gx9++AFOTk4oKytT/rYxZ84cpKenw8nJCUeOHMHcuXMBAJMmTUKbNm3Qo0cPrFmzBh988EGVz16xYoXymFxcXNC/f3/MnTsXFy5cwIABA3Do0CHlby9Pmj59Ovr166e8uJuYmIjRo0fX4btL+oarcxJBPpb+/fffo0OHDo1dShW2trY1jt1XVFTAz88P8fHxetOBjOqOwU8E+RBQeXk5Bg0a1NilVKFN8F++fBlZWVnw9/dvoKqoOWPwExHpGY7xExHpGQY/EZGeYfATEekZBj8RkZ5h8BMR6RkGPxGRnvl/YAYw2WFJiVEAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "\n",
+ " Hints\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n", + "
\n", + "
\n", + "
^j}
zhZ&5Bkxy7bTBK1LJWY&Ui-Ry?DqW2KqVb5J9Ao(yj!w;8j_2dHX_Ht^<~_;BYN_RPJ}r5-Hng0{
zXCxo54KHW&S>zM>5tYo3Dlb2#Qu%R}&QFv_%A@76@_2co63Vectors, as you will use them in this course, are ordered arrays of numbers. In notation, vectors are denoted with lower case bold letters such as $\\mathbf{x}$. The elements of a vector are all the same type. A vector does not, for example, contain both characters and numbers. The number of elements in the array is often referred to as the *dimension* though mathematicians may prefer *rank*. The vector shown has a dimension of $n$. The elements of a vector can be referenced with an index. In math settings, indexes typically run from 1 to n. In computer science and these labs, indexing will typically run from 0 to n-1. In notation, elements of a vector, when referenced individually will indicate the index in a subscript, for example, the $0^{th}$ element, of the vector $\\mathbf{x}$ is $x_0$. Note, the x is not bold in this case. \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 3.2 NumPy Arrays\n",
+ "\n",
+ "NumPy's basic data structure is an indexable, n-dimensional *array* containing elements of the same type (`dtype`). Right away, you may notice we have overloaded the term 'dimension'. Above, it was the number of elements in the vector, here, dimension refers to the number of indexes of an array. A one-dimensional or 1-D array has one index. In Course 1, we will represent vectors as NumPy 1-D arrays. \n",
+ "\n",
+ " - 1-D array, shape (n,): n elements indexed [0] through [n-1]\n",
+ " "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 3.3 Vector Creation\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Data creation routines in NumPy will generally have a first parameter which is the shape of the object. This can either be a single value for a 1-D result or a tuple (n,m,...) specifying the shape of the result. Below are examples of creating vectors using these routines."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# NumPy routines which allocate memory and fill arrays with value\n",
+ "a = np.zeros(4); print(f\"np.zeros(4) : a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
+ "a = np.zeros((4,)); print(f\"np.zeros(4,) : a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
+ "a = np.random.random_sample(4); print(f\"np.random.random_sample(4): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Some data creation routines do not take a shape tuple:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# NumPy routines which allocate memory and fill arrays with value but do not accept shape as input argument\n",
+ "a = np.arange(4.); print(f\"np.arange(4.): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
+ "a = np.random.rand(4); print(f\"np.random.rand(4): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "values can be specified manually as well. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# NumPy routines which allocate memory and fill with user specified values\n",
+ "a = np.array([5,4,3,2]); print(f\"np.array([5,4,3,2]): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
+ "a = np.array([5.,4,3,2]); print(f\"np.array([5.,4,3,2]): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "These have all created a one-dimensional vector `a` with four elements. `a.shape` returns the dimensions. Here we see a.shape = `(4,)` indicating a 1-d array with 4 elements. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 3.4 Operations on Vectors\n",
+ "Let's explore some operations using vectors.\n",
+ "\n",
+ "### 3.4.1 Indexing\n",
+ "Elements of vectors can be accessed via indexing and slicing. NumPy provides a very complete set of indexing and slicing capabilities. We will explore only the basics needed for the course here. Reference [Slicing and Indexing](https://NumPy.org/doc/stable/reference/arrays.indexing.html) for more details. \n",
+ "**Indexing** means referring to *an element* of an array by its position within the array. \n",
+ "**Slicing** means getting a *subset* of elements from an array based on their indices. \n",
+ "NumPy starts indexing at zero so the 3rd element of an vector $\\mathbf{a}$ is `a[2]`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#vector indexing operations on 1-D vectors\n",
+ "a = np.arange(10)\n",
+ "print(a)\n",
+ "\n",
+ "#access an element\n",
+ "print(f\"a[2].shape: {a[2].shape} a[2] = {a[2]}, Accessing an element returns a scalar\")\n",
+ "\n",
+ "# access the last element, negative indexes count from the end\n",
+ "print(f\"a[-1] = {a[-1]}\")\n",
+ "\n",
+ "#indexs must be within the range of the vector or they will produce and error\n",
+ "try:\n",
+ " c = a[10]\n",
+ "except Exception as e:\n",
+ " print(\"The error message you'll see is:\")\n",
+ " print(e)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.2 Slicing\n",
+ "Slicing creates an array of indices using a set of three values (`start:stop:step`). A subset of values is also valid. Its use is best explained by example:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#vector slicing operations\n",
+ "a = np.arange(10)\n",
+ "print(f\"a = {a}\")\n",
+ "\n",
+ "#access 5 consecutive elements (start:stop:step)\n",
+ "c = a[2:7:1]; print(\"a[2:7:1] = \", c)\n",
+ "\n",
+ "# access 3 elements separated by two \n",
+ "c = a[2:7:2]; print(\"a[2:7:2] = \", c)\n",
+ "\n",
+ "# access all elements index 3 and above\n",
+ "c = a[3:]; print(\"a[3:] = \", c)\n",
+ "\n",
+ "# access all elements below index 3\n",
+ "c = a[:3]; print(\"a[:3] = \", c)\n",
+ "\n",
+ "# access all elements\n",
+ "c = a[:]; print(\"a[:] = \", c)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.3 Single vector operations\n",
+ "There are a number of useful operations that involve operations on a single vector."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "a = np.array([1,2,3,4])\n",
+ "print(f\"a : {a}\")\n",
+ "# negate elements of a\n",
+ "b = -a \n",
+ "print(f\"b = -a : {b}\")\n",
+ "\n",
+ "# sum all elements of a, returns a scalar\n",
+ "b = np.sum(a) \n",
+ "print(f\"b = np.sum(a) : {b}\")\n",
+ "\n",
+ "b = np.mean(a)\n",
+ "print(f\"b = np.mean(a): {b}\")\n",
+ "\n",
+ "b = a**2\n",
+ "print(f\"b = a**2 : {b}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.4 Vector Vector element-wise operations\n",
+ "Most of the NumPy arithmetic, logical and comparison operations apply to vectors as well. These operators work on an element-by-element basis. For example \n",
+ "$$ \\mathbf{a} + \\mathbf{b} = \\sum_{i=0}^{n-1} a_i + b_i $$"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "a = np.array([ 1, 2, 3, 4])\n",
+ "b = np.array([-1,-2, 3, 4])\n",
+ "print(f\"Binary operators work element wise: {a + b}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Of course, for this to work correctly, the vectors must be of the same size:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#try a mismatched vector operation\n",
+ "c = np.array([1, 2])\n",
+ "try:\n",
+ " d = a + c\n",
+ "except Exception as e:\n",
+ " print(\"The error message you'll see is:\")\n",
+ " print(e)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.5 Scalar Vector operations\n",
+ "Vectors can be 'scaled' by scalar values. A scalar value is just a number. The scalar multiplies all the elements of the vector."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "a = np.array([1, 2, 3, 4])\n",
+ "\n",
+ "# multiply a by a scalar\n",
+ "b = 5 * a \n",
+ "print(f\"b = 5 * a : {b}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.6 Vector Vector dot product\n",
+ "The dot product is a mainstay of Linear Algebra and NumPy. This is an operation used extensively in this course and should be well understood. The dot product is shown below."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The dot product multiplies the values in two vectors element-wise and then sums the result.\n",
+ "Vector dot product requires the dimensions of the two vectors to be the same. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's implement our own version of the dot product below:\n",
+ "\n",
+ "**Using a for loop**, implement a function which returns the dot product of two vectors. The function to return given inputs $a$ and $b$:\n",
+ "$$ x = \\sum_{i=0}^{n-1} a_i b_i $$\n",
+ "Assume both `a` and `b` are the same shape."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def my_dot(a, b): \n",
+ " \"\"\"\n",
+ " Compute the dot product of two vectors\n",
+ " \n",
+ " Args:\n",
+ " a (ndarray (n,)): input vector \n",
+ " b (ndarray (n,)): input vector with same dimension as a\n",
+ " \n",
+ " Returns:\n",
+ " x (scalar): \n",
+ " \"\"\"\n",
+ " x=0\n",
+ " for i in range(a.shape[0]):\n",
+ " x = x + a[i] * b[i]\n",
+ " return x"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# test 1-D\n",
+ "a = np.array([1, 2, 3, 4])\n",
+ "b = np.array([-1, 4, 3, 2])\n",
+ "print(f\"my_dot(a, b) = {my_dot(a, b)}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Note, the dot product is expected to return a scalar value. \n",
+ "\n",
+ "Let's try the same operations using `np.dot`. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# test 1-D\n",
+ "a = np.array([1, 2, 3, 4])\n",
+ "b = np.array([-1, 4, 3, 2])\n",
+ "c = np.dot(a, b)\n",
+ "print(f\"NumPy 1-D np.dot(a, b) = {c}, np.dot(a, b).shape = {c.shape} \") \n",
+ "c = np.dot(b, a)\n",
+ "print(f\"NumPy 1-D np.dot(b, a) = {c}, np.dot(a, b).shape = {c.shape} \")\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Above, you will note that the results for 1-D matched our implementation."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.7 The Need for Speed: vector vs for loop\n",
+ "We utilized the NumPy library because it improves speed memory efficiency. Let's demonstrate:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "np.random.seed(1)\n",
+ "a = np.random.rand(10000000) # very large arrays\n",
+ "b = np.random.rand(10000000)\n",
+ "\n",
+ "tic = time.time() # capture start time\n",
+ "c = np.dot(a, b)\n",
+ "toc = time.time() # capture end time\n",
+ "\n",
+ "print(f\"np.dot(a, b) = {c:.4f}\")\n",
+ "print(f\"Vectorized version duration: {1000*(toc-tic):.4f} ms \")\n",
+ "\n",
+ "tic = time.time() # capture start time\n",
+ "c = my_dot(a,b)\n",
+ "toc = time.time() # capture end time\n",
+ "\n",
+ "print(f\"my_dot(a, b) = {c:.4f}\")\n",
+ "print(f\"loop version duration: {1000*(toc-tic):.4f} ms \")\n",
+ "\n",
+ "del(a);del(b) #remove these big arrays from memory"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "So, vectorization provides a large speed up in this example. This is because NumPy makes better use of available data parallelism in the underlying hardware. GPU's and modern CPU's implement Single Instruction, Multiple Data (SIMD) pipelines allowing multiple operations to be issued in parallel. This is critical in Machine Learning where the data sets are often very large."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.8 Vector Vector operations in Course 1\n",
+ "Vector Vector operations will appear frequently in course 1. Here is why:\n",
+ "- Going forward, our examples will be stored in an array, `X_train` of dimension (m,n). This will be explained more in context, but here it is important to note it is a 2 Dimensional array or matrix (see next section on matrices).\n",
+ "- `w` will be a 1-dimensional vector of shape (n,).\n",
+ "- we will perform operations by looping through the examples, extracting each example to work on individually by indexing X. For example:`X[i]`\n",
+ "- `X[i]` returns a value of shape (n,), a 1-dimensional vector. Consequently, operations involving `X[i]` are often vector-vector. \n",
+ "\n",
+ "That is a somewhat lengthy explanation, but aligning and understanding the shapes of your operands is important when performing vector operations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# show common Course 1 example\n",
+ "X = np.array([[1],[2],[3],[4]])\n",
+ "w = np.array([2])\n",
+ "c = np.dot(X[1], w)\n",
+ "\n",
+ "print(f\"X[1] has shape {X[1].shape}\")\n",
+ "print(f\"w has shape {w.shape}\")\n",
+ "print(f\"c has shape {c.shape}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "# 4 Matrices\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 4.1 Abstract\n",
+ "Matrices, are two dimensional arrays. The elements of a matrix are all of the same type. In notation, matrices are denoted with capitol, bold letter such as $\\mathbf{X}$. In this and other labs, `m` is often the number of rows and `n` the number of columns. The elements of a matrix can be referenced with a two dimensional index. In math settings, numbers in the index typically run from 1 to n. In computer science and these labs, indexing will run from 0 to n-1. \n",
+ "
Vectors, as you will use them in this course, are ordered arrays of numbers. In notation, vectors are denoted with lower case bold letters such as $\\mathbf{x}$. The elements of a vector are all the same type. A vector does not, for example, contain both characters and numbers. The number of elements in the array is often referred to as the *dimension* though mathematicians may prefer *rank*. The vector shown has a dimension of $n$. The elements of a vector can be referenced with an index. In math settings, indexes typically run from 1 to n. In computer science and these labs, indexing will typically run from 0 to n-1. In notation, elements of a vector, when referenced individually will indicate the index in a subscript, for example, the $0^{th}$ element, of the vector $\\mathbf{x}$ is $x_0$. Note, the x is not bold in this case. \n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 3.2 NumPy Arrays\n",
+ "\n",
+ "NumPy's basic data structure is an indexable, n-dimensional *array* containing elements of the same type (`dtype`). Right away, you may notice we have overloaded the term 'dimension'. Above, it was the number of elements in the vector, here, dimension refers to the number of indexes of an array. A one-dimensional or 1-D array has one index. In Course 1, we will represent vectors as NumPy 1-D arrays. \n",
+ "\n",
+ " - 1-D array, shape (n,): n elements indexed [0] through [n-1]\n",
+ " "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 3.3 Vector Creation\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Data creation routines in NumPy will generally have a first parameter which is the shape of the object. This can either be a single value for a 1-D result or a tuple (n,m,...) specifying the shape of the result. Below are examples of creating vectors using these routines."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# NumPy routines which allocate memory and fill arrays with value\n",
+ "a = np.zeros(4); print(f\"np.zeros(4) : a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
+ "a = np.zeros((4,)); print(f\"np.zeros(4,) : a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
+ "a = np.random.random_sample(4); print(f\"np.random.random_sample(4): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Some data creation routines do not take a shape tuple:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# NumPy routines which allocate memory and fill arrays with value but do not accept shape as input argument\n",
+ "a = np.arange(4.); print(f\"np.arange(4.): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
+ "a = np.random.rand(4); print(f\"np.random.rand(4): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "values can be specified manually as well. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# NumPy routines which allocate memory and fill with user specified values\n",
+ "a = np.array([5,4,3,2]); print(f\"np.array([5,4,3,2]): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
+ "a = np.array([5.,4,3,2]); print(f\"np.array([5.,4,3,2]): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "These have all created a one-dimensional vector `a` with four elements. `a.shape` returns the dimensions. Here we see a.shape = `(4,)` indicating a 1-d array with 4 elements. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 3.4 Operations on Vectors\n",
+ "Let's explore some operations using vectors.\n",
+ "\n",
+ "### 3.4.1 Indexing\n",
+ "Elements of vectors can be accessed via indexing and slicing. NumPy provides a very complete set of indexing and slicing capabilities. We will explore only the basics needed for the course here. Reference [Slicing and Indexing](https://NumPy.org/doc/stable/reference/arrays.indexing.html) for more details. \n",
+ "**Indexing** means referring to *an element* of an array by its position within the array. \n",
+ "**Slicing** means getting a *subset* of elements from an array based on their indices. \n",
+ "NumPy starts indexing at zero so the 3rd element of an vector $\\mathbf{a}$ is `a[2]`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#vector indexing operations on 1-D vectors\n",
+ "a = np.arange(10)\n",
+ "print(a)\n",
+ "\n",
+ "#access an element\n",
+ "print(f\"a[2].shape: {a[2].shape} a[2] = {a[2]}, Accessing an element returns a scalar\")\n",
+ "\n",
+ "# access the last element, negative indexes count from the end\n",
+ "print(f\"a[-1] = {a[-1]}\")\n",
+ "\n",
+ "#indexs must be within the range of the vector or they will produce and error\n",
+ "try:\n",
+ " c = a[10]\n",
+ "except Exception as e:\n",
+ " print(\"The error message you'll see is:\")\n",
+ " print(e)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.2 Slicing\n",
+ "Slicing creates an array of indices using a set of three values (`start:stop:step`). A subset of values is also valid. Its use is best explained by example:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#vector slicing operations\n",
+ "a = np.arange(10)\n",
+ "print(f\"a = {a}\")\n",
+ "\n",
+ "#access 5 consecutive elements (start:stop:step)\n",
+ "c = a[2:7:1]; print(\"a[2:7:1] = \", c)\n",
+ "\n",
+ "# access 3 elements separated by two \n",
+ "c = a[2:7:2]; print(\"a[2:7:2] = \", c)\n",
+ "\n",
+ "# access all elements index 3 and above\n",
+ "c = a[3:]; print(\"a[3:] = \", c)\n",
+ "\n",
+ "# access all elements below index 3\n",
+ "c = a[:3]; print(\"a[:3] = \", c)\n",
+ "\n",
+ "# access all elements\n",
+ "c = a[:]; print(\"a[:] = \", c)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.3 Single vector operations\n",
+ "There are a number of useful operations that involve operations on a single vector."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "a = np.array([1,2,3,4])\n",
+ "print(f\"a : {a}\")\n",
+ "# negate elements of a\n",
+ "b = -a \n",
+ "print(f\"b = -a : {b}\")\n",
+ "\n",
+ "# sum all elements of a, returns a scalar\n",
+ "b = np.sum(a) \n",
+ "print(f\"b = np.sum(a) : {b}\")\n",
+ "\n",
+ "b = np.mean(a)\n",
+ "print(f\"b = np.mean(a): {b}\")\n",
+ "\n",
+ "b = a**2\n",
+ "print(f\"b = a**2 : {b}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.4 Vector Vector element-wise operations\n",
+ "Most of the NumPy arithmetic, logical and comparison operations apply to vectors as well. These operators work on an element-by-element basis. For example \n",
+ "$$ \\mathbf{a} + \\mathbf{b} = \\sum_{i=0}^{n-1} a_i + b_i $$"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "a = np.array([ 1, 2, 3, 4])\n",
+ "b = np.array([-1,-2, 3, 4])\n",
+ "print(f\"Binary operators work element wise: {a + b}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Of course, for this to work correctly, the vectors must be of the same size:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "#try a mismatched vector operation\n",
+ "c = np.array([1, 2])\n",
+ "try:\n",
+ " d = a + c\n",
+ "except Exception as e:\n",
+ " print(\"The error message you'll see is:\")\n",
+ " print(e)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.5 Scalar Vector operations\n",
+ "Vectors can be 'scaled' by scalar values. A scalar value is just a number. The scalar multiplies all the elements of the vector."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "a = np.array([1, 2, 3, 4])\n",
+ "\n",
+ "# multiply a by a scalar\n",
+ "b = 5 * a \n",
+ "print(f\"b = 5 * a : {b}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.6 Vector Vector dot product\n",
+ "The dot product is a mainstay of Linear Algebra and NumPy. This is an operation used extensively in this course and should be well understood. The dot product is shown below."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The dot product multiplies the values in two vectors element-wise and then sums the result.\n",
+ "Vector dot product requires the dimensions of the two vectors to be the same. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's implement our own version of the dot product below:\n",
+ "\n",
+ "**Using a for loop**, implement a function which returns the dot product of two vectors. The function to return given inputs $a$ and $b$:\n",
+ "$$ x = \\sum_{i=0}^{n-1} a_i b_i $$\n",
+ "Assume both `a` and `b` are the same shape."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def my_dot(a, b): \n",
+ " \"\"\"\n",
+ " Compute the dot product of two vectors\n",
+ " \n",
+ " Args:\n",
+ " a (ndarray (n,)): input vector \n",
+ " b (ndarray (n,)): input vector with same dimension as a\n",
+ " \n",
+ " Returns:\n",
+ " x (scalar): \n",
+ " \"\"\"\n",
+ " x=0\n",
+ " for i in range(a.shape[0]):\n",
+ " x = x + a[i] * b[i]\n",
+ " return x"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# test 1-D\n",
+ "a = np.array([1, 2, 3, 4])\n",
+ "b = np.array([-1, 4, 3, 2])\n",
+ "print(f\"my_dot(a, b) = {my_dot(a, b)}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Note, the dot product is expected to return a scalar value. \n",
+ "\n",
+ "Let's try the same operations using `np.dot`. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# test 1-D\n",
+ "a = np.array([1, 2, 3, 4])\n",
+ "b = np.array([-1, 4, 3, 2])\n",
+ "c = np.dot(a, b)\n",
+ "print(f\"NumPy 1-D np.dot(a, b) = {c}, np.dot(a, b).shape = {c.shape} \") \n",
+ "c = np.dot(b, a)\n",
+ "print(f\"NumPy 1-D np.dot(b, a) = {c}, np.dot(a, b).shape = {c.shape} \")\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Above, you will note that the results for 1-D matched our implementation."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.7 The Need for Speed: vector vs for loop\n",
+ "We utilized the NumPy library because it improves speed memory efficiency. Let's demonstrate:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "np.random.seed(1)\n",
+ "a = np.random.rand(10000000) # very large arrays\n",
+ "b = np.random.rand(10000000)\n",
+ "\n",
+ "tic = time.time() # capture start time\n",
+ "c = np.dot(a, b)\n",
+ "toc = time.time() # capture end time\n",
+ "\n",
+ "print(f\"np.dot(a, b) = {c:.4f}\")\n",
+ "print(f\"Vectorized version duration: {1000*(toc-tic):.4f} ms \")\n",
+ "\n",
+ "tic = time.time() # capture start time\n",
+ "c = my_dot(a,b)\n",
+ "toc = time.time() # capture end time\n",
+ "\n",
+ "print(f\"my_dot(a, b) = {c:.4f}\")\n",
+ "print(f\"loop version duration: {1000*(toc-tic):.4f} ms \")\n",
+ "\n",
+ "del(a);del(b) #remove these big arrays from memory"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "So, vectorization provides a large speed up in this example. This is because NumPy makes better use of available data parallelism in the underlying hardware. GPU's and modern CPU's implement Single Instruction, Multiple Data (SIMD) pipelines allowing multiple operations to be issued in parallel. This is critical in Machine Learning where the data sets are often very large."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "### 3.4.8 Vector Vector operations in Course 1\n",
+ "Vector Vector operations will appear frequently in course 1. Here is why:\n",
+ "- Going forward, our examples will be stored in an array, `X_train` of dimension (m,n). This will be explained more in context, but here it is important to note it is a 2 Dimensional array or matrix (see next section on matrices).\n",
+ "- `w` will be a 1-dimensional vector of shape (n,).\n",
+ "- we will perform operations by looping through the examples, extracting each example to work on individually by indexing X. For example:`X[i]`\n",
+ "- `X[i]` returns a value of shape (n,), a 1-dimensional vector. Consequently, operations involving `X[i]` are often vector-vector. \n",
+ "\n",
+ "That is a somewhat lengthy explanation, but aligning and understanding the shapes of your operands is important when performing vector operations."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# show common Course 1 example\n",
+ "X = np.array([[1],[2],[3],[4]])\n",
+ "w = np.array([2])\n",
+ "c = np.dot(X[1], w)\n",
+ "\n",
+ "print(f\"X[1] has shape {X[1].shape}\")\n",
+ "print(f\"w has shape {w.shape}\")\n",
+ "print(f\"c has shape {c.shape}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "# 4 Matrices\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "## 4.1 Abstract\n",
+ "Matrices, are two dimensional arrays. The elements of a matrix are all of the same type. In notation, matrices are denoted with capitol, bold letter such as $\\mathbf{X}$. In this and other labs, `m` is often the number of rows and `n` the number of columns. The elements of a matrix can be referenced with a two dimensional index. In math settings, numbers in the index typically run from 1 to n. In computer science and these labs, indexing will run from 0 to n-1. \n",
+ "
zfOf`GhR&ag>(Om}$&iLNfJ6=9k%xO%>YQ>VC9PC?BF8rC7=x%v=%cVbfe_P?#;23Y
z>^kkklNT#L!B@kLcIOE{2iXYLr6x!0N6CUMqMoUXl(ZT(JX?Nlj^zu1@lQ%TWM>A=
zGTKQAw}$>j(>luG6?J;-<6eRTvX~%s?6>+BhnMtFJvXAgThZ4VsqFWFpw$M&&p(Q955aO0`0ULjW