{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3892507e",
   "metadata": {},
   "source": [
    "<div style=\"\n",
    "    border: 2px solid #4CAF50; \n",
    "    padding: 15px; \n",
    "    background-color: #f4f4f4; \n",
    "    border-radius: 10px; \n",
    "    align-items: center;\">\n",
    "\n",
    "<h1 style=\"margin: 0; color: #4CAF50;\">Neural Networks: Numpy, PyTorch, Google Colab</h1>\n",
    "<h2 style=\"margin: 5px 0; color: #555;\">DSAI</h2>\n",
    "<h3 style=\"margin: 5px 0; color: #555;\">Jakob Eggl</h3>\n",
    "\n",
    "<div style=\"flex-shrink: 0;\">\n",
    "    <img src=\"https://www.htl-grieskirchen.at/wp/wp-content/uploads/2022/11/logo_bildschirm-1024x503.png\" alt=\"Logo\" style=\"width: 250px; height: auto;\"/>\n",
    "</div>\n",
    "<p1> © 2025/26 Jakob Eggl. Nutzung oder Verbreitung nur mit ausdrücklicher Genehmigung des Autors.</p1>\n",
    "</div>\n",
    "<div style=\"flex: 1;\">\n",
    "</div>   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "138e9e63",
   "metadata": {},
   "source": [
    "# Numpy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e58901d",
   "metadata": {},
   "source": [
    "Wir wollen uns nun ``numpy`` widmen. Es ist eine grundlegende - wenn nicht sogar *DIE* - Bibliothek, auf der die meisten weiteren (und von uns auch bisher schon verwendeten) Bibliotheken aufbauen.\n",
    "\n",
    "Für numpy gibt es viele Tutorials online, unter anderem wurden [Tutorial1](https://numpy.org/doc/stable/user/quickstart.html) und [Tutorial2](https://cs231n.github.io/python-numpy-tutorial/) für diesen Abschnitt des Notebooks verwendet."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbc12329",
   "metadata": {},
   "source": [
    "## Was ist Numpy?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85b0ddf5",
   "metadata": {},
   "source": [
    "* **Num**erical **Py**thon\n",
    "* Open Source\n",
    "* ermöglicht numerisches Rechnen mit Python\n",
    "* Hauptobjekte in Numpy: (mehrdimensionale) *Arrays*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db4f4f5e",
   "metadata": {},
   "source": [
    "Bisher haben wir uns manchmal schon implizit mit ``numpy`` Objekten beschäftigt:\n",
    "* Bei scikit-learn (``sklearn``) werden haben wir oft mit numpy Objekten gearbeitet (zBsp.: in vielen Fällen waren *X* und *y* numpy arrays).\n",
    "* Pandas Dataframes können auch einfach in numpy arrays konvertiert werden.\n",
    "\n",
    "Bisher war das aber bei uns immer ein eher unwichtiges Detail. Wir betrachten nun weitere Gründe, warum wir uns gut in numpy auskennen sollen."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88dc22e9",
   "metadata": {},
   "source": [
    "Wir importieren ``numpy`` ganz einfach mit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "717661df",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d40fe311",
   "metadata": {},
   "source": [
    "## Warum Numpy?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34300d89",
   "metadata": {},
   "source": [
    "#### Vorteile:\n",
    "* **sehr schnell** und **effizient** (ist in *C* geschrieben)\n",
    "* Viele vektorisierte (vectorized) Operationen\n",
    "* Große Anzahl an mathematischen Funktionen und Operationen implementiert\n",
    "* Super integriert in alle gängigen Bibliotheken\n",
    "* Effizienter Speicherverbrauch (überall gleicher Datentyp in einem array)\n",
    "\n",
    "#### Nachteile:\n",
    "* (Fast) nur numerische Datentypen (Integer, Float, Double, etc.) möglich\n",
    "* Pro array nur ein Datentyp erlaubt (viele Datasets bei uns haben bisher oft auch andere Features gehabt (Name, Adresse, etc.))\n",
    "* Namen der Features und Label(s) gehen verloren."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ba1dd8",
   "metadata": {},
   "source": [
    "Betrachten wir den Geschwindigkeitsunterschied anhand von einem kurzem Beispiel:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e9e6441e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Listenzeit: 0.11680150032043457\n",
      "NumPy-Zeit: 0.005227804183959961\n",
      "Numpy ist in diesem Fall 22.342363296392577 mal schneller\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "n_elements = 3000000\n",
    "\n",
    "a = list(range(n_elements))\n",
    "b = list(range(n_elements))\n",
    "\n",
    "start = time.time()\n",
    "z = zip(a, b)\n",
    "c = [x + y for x, y in z]\n",
    "time_1 = time.time() - start\n",
    "print(f\"Listenzeit: {time_1}\")\n",
    "\n",
    "a_np = np.array(a)\n",
    "b_np = np.array(b)\n",
    "\n",
    "start = time.time()\n",
    "c_np = a_np + b_np\n",
    "time_2 = time.time() - start\n",
    "print(f\"NumPy-Zeit: {time_2}\")\n",
    "\n",
    "print(f\"Numpy ist in diesem Fall {time_1/time_2} mal schneller\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4f48f44",
   "metadata": {},
   "source": [
    "![Vectored_Meme](../resources/Vectored.png)\n",
    "\n",
    "(von https://imageresizer.com/pt/gerador-de-memes/editar/you-just-got-vectored)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0750b461",
   "metadata": {},
   "source": [
    "## Numpy Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e07283cf",
   "metadata": {},
   "source": [
    "Nun beschäftigen wir uns mit dem zentralen Element in numpy: Das **numpy array**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "78b046b0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n"
     ]
    }
   ],
   "source": [
    "# Short Recap on Python Lists\n",
    "\n",
    "a_list = list(range(10))\n",
    "\n",
    "print(a_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8d30d466",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 2, 2, 3, 4, 5, 6, 7, 8, 9]\n"
     ]
    }
   ],
   "source": [
    "a_list[1]=2\n",
    "print(a_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fb7fd093",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now to numpy arrays. We now convert our list to a numpy array.\n",
    "\n",
    "a_array = np.array(a_list) # we only need to pass a list to the np.array function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "db374151",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 2 2 3 4 5 6 7 8 9]\n",
      "<class 'numpy.ndarray'>\n",
      "int64\n"
     ]
    }
   ],
   "source": [
    "print(a_array)\n",
    "print(type(a_array))\n",
    "print(a_array.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f1637084",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now what about this list?\n",
    "a_second_list = ['hello', 'world!']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b7fb78f5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['hello', 'world!']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a_second_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5728c71b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Converting to numpy?\n",
    "a_second_array = np.array(a_second_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "07bc23c8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['hello' 'world!']\n",
      "<class 'numpy.ndarray'>\n",
      "<U6\n"
     ]
    }
   ],
   "source": [
    "print(a_second_array)\n",
    "print(type(a_second_array))\n",
    "print(a_second_array.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c05f8c3",
   "metadata": {},
   "source": [
    "Es funktioniert also schon, jedoch wird nun das ganze Array als *<U6* Datentyp gespeichert. Das sind unicode strings mit maximal 6 Zeichen.\n",
    "\n",
    "Somit können wir auch die Texte \"kürzen\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e3efc821",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['hello', 'world'], dtype='<U5')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([\"hello\", \"world!\"], dtype=\"U5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8556b9fa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['he', 'wo'], dtype='<U2')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Extreme Example\n",
    "np.array([\"hello\", \"world!\"], dtype=\"U2\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c057cd8a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['hello', 'world!'], dtype='<U7')"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array([\"hello\", \"world!\"], dtype=\"U7\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55281898",
   "metadata": {},
   "source": [
    "Es gibt noch weitere Informationen, welche wir über das Array herausfinden können."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "cf075555",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5,)\n",
      "1\n",
      "5\n"
     ]
    }
   ],
   "source": [
    "# numpy arrays also have other properties that we can use to get information about the array\n",
    "a = np.array([1, 2, 3, 4, 5])\n",
    "print(a.shape)\n",
    "print(a.ndim)\n",
    "print(a.size)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efc69e8b",
   "metadata": {},
   "source": [
    "Somit ist *a* bei uns ein Vektor der Länge 5. Wir können aber auch eine Matrix erstellen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "1331ded7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# matrix in numpy\n",
    "a = np.array([[1, 2, 3], [4, 5, 6]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "84c8e033",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2 3]\n",
      " [4 5 6]]\n",
      "(2, 3)\n",
      "2\n",
      "6\n"
     ]
    }
   ],
   "source": [
    "print(a)\n",
    "print(a.shape)\n",
    "print(a.ndim)\n",
    "print(a.size)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e029c15",
   "metadata": {},
   "source": [
    "Wir haben also nun eine Matrix mit 2 Zeilen (erster Eintrag in ``.shape``) und 3 Spalten (zweiter Eintrag). Dies ist genau gleich wie in der Mathematik.\n",
    "\n",
    "**Merkregel:** Zeilen zuerst, Spalten später. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bc08897",
   "metadata": {},
   "source": [
    "### Operationen mit Numpy Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f32379c7",
   "metadata": {},
   "source": [
    "Im folgenden sehen wir einige Beispiele von Befehlen, die in numpy verwendet werden können um Arrays zu erstellen, bearbeiten und abzufragen. Diese Liste ist nur ein kleiner Teil aller Möglichkeiten, die es in ``numpy`` gibt.\n",
    "\n",
    "Mit (einem Teil von) den folgenden Befehlen werden wir im Rahmen des Bauens von neuronalen Netzwerken arbeiten, weswegen wir sie kurz besprechen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "0ed4ad6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can also change the shape of the numpy array\n",
    "reshaped_a = a.reshape(3, 2)  # we now switch the number of columns and rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "b8dd7360",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "**************************************************\n",
      "Zeros Array:\n",
      "[[0. 0. 0. 0.]\n",
      " [0. 0. 0. 0.]\n",
      " [0. 0. 0. 0.]]\n",
      "**************************************************\n",
      "Ones Array:\n",
      "[[1. 1. 1. 1. 1.]\n",
      " [1. 1. 1. 1. 1.]]\n",
      "**************************************************\n",
      "Random Array:\n",
      "[[0.29255354 0.67094282 0.38152982]\n",
      " [0.07006982 0.21032955 0.97910274]]\n",
      "**************************************************\n",
      "Random -1 to 1 Array:\n",
      "[[ 0.4538663   0.8185449   0.41463727]\n",
      " [ 0.33390644 -0.5025175  -0.55260822]]\n",
      "**************************************************\n",
      "Constant 7 array\n",
      "[[7 7 7]\n",
      " [7 7 7]]\n"
     ]
    }
   ],
   "source": [
    "# There are of course many other ways on how to create a numpy array, without manually writing down each entry\n",
    "zeros_array = np.zeros((3, 4))  \n",
    "ones_array = np.ones((2, 5))    \n",
    "random_array = np.random.rand(2, 3)  # uniform between 0 and 1\n",
    "random_neg1_pos1_array = np.random.uniform(-1, 1, (2, 3))  \n",
    "seven_array = np.full((2, 3), 7)\n",
    "print(\"*\"*50)\n",
    "print(\"Zeros Array:\")\n",
    "print(zeros_array)\n",
    "print(\"*\"*50)\n",
    "print(\"Ones Array:\")\n",
    "print(ones_array)\n",
    "print(\"*\"*50)\n",
    "print(\"Random Array:\")\n",
    "print(random_array)\n",
    "print(\"*\"*50)\n",
    "print(\"Random -1 to 1 Array:\")\n",
    "print(random_neg1_pos1_array)\n",
    "print(\"*\"*50)\n",
    "print(\"Constant 7 array\")\n",
    "print(seven_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b149b68",
   "metadata": {},
   "source": [
    "> **Übung:** Was wäre mit den bekannten Methoden also der schnellste Weg, um eine $3\\times 3$ Matrix zu erstellen, welche aufsteigend die Zahlen 1-9 beinhaltet?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a4115e3",
   "metadata": {},
   "source": [
    "## Indexing von Numpy Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e00496f0",
   "metadata": {},
   "source": [
    "Es gibt viele Möglichkeiten, wie in numpy ein Array indiziert werden kann. Im einfachsten Fall ist das gleich wie in den meisten gängigen Programmiersprachen. Es gibt aber auch Eigenheiten (zBsp slicing) für numpy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "7dac2775",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2 3]\n",
      " [6 7]]\n",
      "2\n",
      "77\n"
     ]
    }
   ],
   "source": [
    "# Create the following rank 2 array with shape (3, 4)\n",
    "# [[ 1  2  3  4]\n",
    "#  [ 5  6  7  8]\n",
    "#  [ 9 10 11 12]]\n",
    "a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]]) # this can be done more elegantly (see previous exercise)\n",
    "\n",
    "# Use slicing to pull out the subarray consisting of the first 2 rows\n",
    "# and columns 1 and 2; b is the following array of shape (2, 2):\n",
    "# [[2 3]\n",
    "#  [6 7]]\n",
    "b = a[:2, 1:3]\n",
    "print(b) # prints [[2 3], [6 7]]\n",
    "\n",
    "# A slice of an array is a view into the same data, so modifying it\n",
    "# will modify the original array.\n",
    "print(a[0, 1])   # Prints \"2\"\n",
    "b[0, 0] = 77     # b[0, 0] is the same piece of data as a[0, 1]\n",
    "print(a[0, 1])   # Prints \"77\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b4bf033",
   "metadata": {},
   "source": [
    "Wir sehen also auch, dass wir durch slicing immer noch auf das gleiche Array referenzieren! Es können auch beide Indexmethoden gemischt werden:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0ab228f2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5 6 7 8] (4,)\n",
      "[[5 6 7 8]] (1, 4)\n",
      "[ 2  6 10] (3,)\n",
      "[[ 2]\n",
      " [ 6]\n",
      " [10]] (3, 1)\n"
     ]
    }
   ],
   "source": [
    "# Create the following rank 2 array with shape (3, 4)\n",
    "# [[ 1  2  3  4]\n",
    "#  [ 5  6  7  8]\n",
    "#  [ 9 10 11 12]]\n",
    "a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])\n",
    "\n",
    "# Two ways of accessing the data in the middle row of the array.\n",
    "# Mixing integer indexing with slices yields an array of lower rank,\n",
    "# while using only slices yields an array of the same rank as the\n",
    "# original array:\n",
    "row_r1 = a[1, :]    # Rank 1 view of the second row of a\n",
    "row_r2 = a[1:2, :]  # Rank 2 view of the second row of a\n",
    "print(row_r1, row_r1.shape)  # Prints \"[5 6 7 8] (4,)\"\n",
    "print(row_r2, row_r2.shape)  # Prints \"[[5 6 7 8]] (1, 4)\"\n",
    "\n",
    "# We can make the same distinction when accessing columns of an array:\n",
    "col_r1 = a[:, 1]\n",
    "col_r2 = a[:, 1:2]\n",
    "print(col_r1, col_r1.shape)  # Prints \"[ 2  6 10] (3,)\"\n",
    "print(col_r2, col_r2.shape)  # Prints \"[[ 2]\n",
    "                             #          [ 6]\n",
    "                             #          [10]] (3, 1)\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a42724a6",
   "metadata": {},
   "source": [
    "Wir können auch numpy arrays mit anderen numpy arrays (zbsp integer-array) indizieren:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "80643fac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2]\n",
      " [3 4]\n",
      " [5 6]]\n",
      "[1 4 5]\n",
      "[1 4 5]\n",
      "[2 2]\n",
      "[2 2]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "a = np.array([[1,2], [3, 4], [5, 6]])\n",
    "print(a)\n",
    "\n",
    "# An example of integer array indexing.\n",
    "# The returned array will have shape (3,) and\n",
    "print(a[[0, 1, 2], [0, 1, 0]])  # Prints \"[1 4 5]\"\n",
    "\n",
    "# The above example of integer array indexing is equivalent to this:\n",
    "print(np.array([a[0, 0], a[1, 1], a[2, 0]]))  # Prints \"[1 4 5]\"\n",
    "\n",
    "# When using integer array indexing, you can reuse the same\n",
    "# element from the source array:\n",
    "print(a[[0, 0], [1, 1]])  # Prints \"[2 2]\"\n",
    "\n",
    "# Equivalent to the previous integer array indexing example\n",
    "print(np.array([a[0, 1], a[0, 1]]))  # Prints \"[2 2]\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ddd9330",
   "metadata": {},
   "source": [
    "Dies kann auch verwendet werden um zBsp ein Element von jeder Zeile zu erhalten oder verändern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "d549ccbd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1  2  3]\n",
      " [ 4  5  6]\n",
      " [ 7  8  9]\n",
      " [10 11 12]]\n",
      "[ 1  6  7 11]\n",
      "[[11  2  3]\n",
      " [ 4  5 16]\n",
      " [17  8  9]\n",
      " [10 21 12]]\n"
     ]
    }
   ],
   "source": [
    "# Create a new array from which we will select elements\n",
    "a = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])\n",
    "\n",
    "print(a)  # prints \"array([[ 1,  2,  3],\n",
    "          #                [ 4,  5,  6],\n",
    "          #                [ 7,  8,  9],\n",
    "          #                [10, 11, 12]])\"\n",
    "\n",
    "# Create an array of indices\n",
    "b = np.array([0, 2, 0, 1])\n",
    "\n",
    "# Select one element from each row of a using the indices in b\n",
    "print(a[np.arange(4), b])  # Prints \"[ 1  6  7 11]\"\n",
    "\n",
    "# Mutate one element from each row of a using the indices in b\n",
    "a[np.arange(4), b] += 10\n",
    "\n",
    "print(a)  # prints \"array([[11,  2,  3],\n",
    "          #                [ 4,  5, 16],\n",
    "          #                [17,  8,  9],\n",
    "          #                [10, 21, 12]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbf1586f",
   "metadata": {},
   "source": [
    "Auch mit einem Boolean-Array kann indiziert werden."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "b668187f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2]\n",
      " [3 4]\n",
      " [5 6]]\n",
      "[[False False]\n",
      " [ True  True]\n",
      " [ True  True]]\n",
      "[3 4 5 6]\n",
      "[3 4 5 6]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "a = np.array([[1,2], [3, 4], [5, 6]])\n",
    "print(a)\n",
    "\n",
    "bool_idx = (a > 2)   # Find the elements of a that are bigger than 2;\n",
    "                     # this returns a numpy array of Booleans of the same\n",
    "                     # shape as a, where each slot of bool_idx tells\n",
    "                     # whether that element of a is > 2.\n",
    "\n",
    "print(bool_idx)      # Prints \"[[False False]\n",
    "                     #          [ True  True]\n",
    "                     #          [ True  True]]\"\n",
    "\n",
    "# We use boolean array indexing to construct a rank 1 array\n",
    "# consisting of the elements of a corresponding to the True values\n",
    "# of bool_idx\n",
    "print(a[bool_idx])  # Prints \"[3 4 5 6]\"\n",
    "\n",
    "# We can do all of the above in a single concise statement:\n",
    "print(a[a > 2])     # Prints \"[3 4 5 6]\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af7674b1",
   "metadata": {},
   "source": [
    "## Mathematische Funktionen und Matrizen/Vektorrechnung (Lineare Algebra)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e65b3f7b",
   "metadata": {},
   "source": [
    "Nachdem ja numpy vorrangig für die Durchführung von numerischen Simulationen/Programmen entwickelt wurde, möchten wir uns jetzt noch spannende und vor allem für den Machine Learning Bereich wichtige Funktionen/Operationen ansehen."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "357800bf",
   "metadata": {},
   "source": [
    "#### Addition und Subtraktion von Arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "7c759fbc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5 7 9]\n",
      "[-3 -3 -3]\n",
      "[13 17 21]\n"
     ]
    }
   ],
   "source": [
    "a = np.array([1,2,3])\n",
    "b = np.array([4,5,6])\n",
    "\n",
    "print(a+b)\n",
    "print(a-b)\n",
    "print(a+3*b)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab14d4e6",
   "metadata": {},
   "source": [
    "#### Transponieren"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "504d93d5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2]\n",
      " [3 4]\n",
      " [5 6]]\n"
     ]
    }
   ],
   "source": [
    "A = np.array([[1, 2], [3, 4], [5, 6]])\n",
    "print(A)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "09bce2af",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can transpose the array using the .T attribute\n",
    "transposed_A = A.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "c6dd9b06",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 3 5]\n",
      " [2 4 6]]\n"
     ]
    }
   ],
   "source": [
    "print(transposed_A)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "360f2a4a",
   "metadata": {},
   "source": [
    "#### Matrix-Matrix und Matrix-Vektor und Skalarprodukt und Betrag vom Vektor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "295ff193",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 23  29  35]\n",
      " [ 53  67  81]\n",
      " [ 83 105 127]]\n",
      "[[ 23  29  35]\n",
      " [ 53  67  81]\n",
      " [ 83 105 127]]\n"
     ]
    }
   ],
   "source": [
    "A = np.array([[1, 2], [3, 4], [5, 6]])\n",
    "B = np.array([[7, 8], [9, 10], [11, 12]])\n",
    "\n",
    "# Matrix multiplication (dot product) using numpy\n",
    "matrix_product = np.dot(A, B.T)\n",
    "matrix_product_2 = A @ B.T\n",
    "\n",
    "print(matrix_product)\n",
    "print(matrix_product_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "9180ea27",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[32 77]\n"
     ]
    }
   ],
   "source": [
    "A = np.array([[1, 2, 3], [4, 5, 6]])\n",
    "x = np.array([4,5,6])\n",
    "\n",
    "print(A@x) # calculates A*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "10d617ff",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13\n"
     ]
    }
   ],
   "source": [
    "# Scalarproduct (Innerproduct)\n",
    "a = np.array([5,6,7])\n",
    "b = np.array([0,1,1])\n",
    "print(np.inner(a,b))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "cb35b966",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10.488088481701515\n"
     ]
    }
   ],
   "source": [
    "# Length of a vector\n",
    "a = np.array([5,6,7])\n",
    "print(np.linalg.norm(a)) # Calculates with Pythagorean theorem"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b194550a",
   "metadata": {},
   "source": [
    "#### Lösen von Gleichungssystemen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "bd28f3ae",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Solution x: [0.8 1.3]\n"
     ]
    }
   ],
   "source": [
    "# We can also solve a system of linear equations using numpy\n",
    "A = np.array([[3, 2], [1, 4]])\n",
    "b = np.array([5, 6])\n",
    "x = np.linalg.solve(A, b)  # Solves the equation Ax = b\n",
    "print(\"Solution x:\", x)  # Prints the solution to the system of equations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5aed967d",
   "metadata": {},
   "source": [
    "> **Übung:** Wie sieht das Gleichungssystem hier aus?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "712b2694",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The matrix A is singular, cannot solve the system of equations.\n"
     ]
    }
   ],
   "source": [
    "# Note that this is not possible if the determinant of A is zero, e.g.\n",
    "A = np.array([[1, 2], [2, 4]])\n",
    "b = np.array([3, 6])\n",
    "try:\n",
    "    x = np.linalg.solve(A, b)  # This will raise an error\n",
    "except:\n",
    "    print(\"The matrix A is singular, cannot solve the system of equations.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "4efaa740",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Determinant of A: 0.0\n"
     ]
    }
   ],
   "source": [
    "# We can check that directly:\n",
    "det_A = np.linalg.det(A)\n",
    "print(\"Determinant of A:\", det_A)  # Prints the determinant of A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "53ab1c64",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Inverse of A:\n",
      " [[-2.   1. ]\n",
      " [ 1.5 -0.5]]\n",
      "[[1.00000000e+00 1.11022302e-16]\n",
      " [0.00000000e+00 1.00000000e+00]]\n"
     ]
    }
   ],
   "source": [
    "# And we can also calculate the inverse of a matrix, if it is not singular\n",
    "A = np.array([[1, 2], [3, 4]])\n",
    "A_inv = np.linalg.inv(A)  # Calculates the inverse of A\n",
    "print(\"Inverse of A:\\n\", A_inv)  # Prints the inverse of A\n",
    "\n",
    "# Sanity check:\n",
    "print(A@A_inv)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "258398a5",
   "metadata": {},
   "source": [
    "#### (Arg)Min und (Arg)Max"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "aa18cb5c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 3 4 5]\n",
      "Max value: 5 at index 4\n",
      "Min value: 1 at index 0\n"
     ]
    }
   ],
   "source": [
    "# We can also look for the maximum and minimum values in an array and its indices\n",
    "a = np.array([1, 2, 3, 4, 5])\n",
    "print(a)\n",
    "max_value = np.max(a)  # Maximum value\n",
    "max_index = np.argmax(a)  # Index of maximum value\n",
    "min_value = np.min(a)  # Minimum value\n",
    "min_index = np.argmin(a)  # Index of minimum value\n",
    "print(\"Max value:\", max_value, \"at index\", max_index)\n",
    "print(\"Min value:\", min_value, \"at index\", min_index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "c8ad38f7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2 3]\n",
      " [4 5 6]\n",
      " [7 8 9]]\n",
      "Max value in matrix: 9 at index 8\n",
      "Min value in matrix: 1 at index 0\n"
     ]
    }
   ],
   "source": [
    "# This can also be done in matrices\n",
    "A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
    "print(A)\n",
    "max_value = np.max(A)  # Maximum value in the matrix\n",
    "max_index = np.argmax(A)  # Index of maximum value in the matrix\n",
    "min_value = np.min(A)  # Minimum value in the matrix\n",
    "min_index = np.argmin(A)  # Index of minimum value in the matrix\n",
    "print(\"Max value in matrix:\", max_value, \"at index\", max_index)\n",
    "print(\"Min value in matrix:\", min_value, \"at index\", min_index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "000e5eec",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2 3]\n",
      " [4 5 6]\n",
      " [7 8 9]]\n",
      "Max values in each row: [3 6 9]\n",
      "Min values in each row: [1 4 7]\n",
      "Max values in each column: [7 8 9]\n"
     ]
    }
   ],
   "source": [
    "# And we can also get the maximum and minimum values and indices in each row or column\n",
    "\n",
    "print(A)\n",
    "\n",
    "max_values_row = np.max(A, axis=1)  # Maximum values in each row\n",
    "print(\"Max values in each row:\", max_values_row)\n",
    "min_values_row = np.min(A, axis=1)  # Minimum values in each row\n",
    "print(\"Min values in each row:\", min_values_row)\n",
    "# We can also get the maximum and minimum values in each column\n",
    "max_values_col = np.max(A, axis=0)  # Maximum values in each column\n",
    "print(\"Max values in each column:\", max_values_col)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8870e317",
   "metadata": {},
   "source": [
    "#### Mathematische Funktionen und komplexe Zahlen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "741d77e9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original array: [ 1  2  3 -4 -5]\n",
      "Sine of array: [0.84147098 0.90929743 0.14112001 0.7568025  0.95892427]\n",
      "Absolute values of array: [1 2 3 4 5]\n",
      "Square root of array: [1.         1.41421356 1.73205081 2.         2.23606798]\n",
      "[[1 2 3]\n",
      " [4 5 6]\n",
      " [7 8 9]]\n",
      "Sine of matrix:\n",
      " [[ 0.84147098  0.90929743  0.14112001]\n",
      " [-0.7568025  -0.95892427 -0.2794155 ]\n",
      " [ 0.6569866   0.98935825  0.41211849]]\n",
      "Absolute values of matrix:\n",
      " [[1 2 3]\n",
      " [4 5 6]\n",
      " [7 8 9]]\n",
      "Complex array: [1.+2.j 3.+4.j 5.+6.j]\n",
      "Real part of complex array: [1. 3. 5.]\n",
      "Imaginary part of complex array: [2. 4. 6.]\n"
     ]
    }
   ],
   "source": [
    "# We can also just apply functions like sin, abs, etc. to numpy arrays\n",
    "a = np.array([1, 2, 3, -4, -5])\n",
    "print(\"Original array:\", a)\n",
    "print(\"Sine of array:\", np.sin(a))  # Applies sine function to each element\n",
    "print(\"Absolute values of array:\", np.abs(a))  # Applies absolute value function to each\n",
    "print(\"Square root of array:\", np.sqrt(np.abs(a)))  # Applies square root function to each element (after taking absolute value)\n",
    "\n",
    "# We can also apply functions to matrices\n",
    "A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
    "print(A)\n",
    "print(\"Sine of matrix:\\n\", np.sin(A))  # Applies sine function to each element\n",
    "print(\"Absolute values of matrix:\\n\", np.abs(A))  # Applies absolute value function to each element\n",
    "\n",
    "\n",
    "# And we can also work with complex numbers in numpy\n",
    "complex_array = np.array([1 + 2j, 3 + 4j, 5 + 6j])\n",
    "print(\"Complex array:\", complex_array)\n",
    "print(\"Real part of complex array:\", np.real(complex_array))  # Extracts real part\n",
    "print(\"Imaginary part of complex array:\", np.imag(complex_array))  # Extracts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ac7d19a",
   "metadata": {},
   "source": [
    "# PyTorch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3125881",
   "metadata": {},
   "source": [
    "Nun beschäftigen wir uns mit ``PyTorch``. Es baut auf ``numpy`` und ist eine sehr weit verbreitete Bibliothek, die das Entwickeln von neuronalen Netzwerken sehr einfach ermöglicht."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e866ad8",
   "metadata": {},
   "source": [
    "Das folgende basiert hauptsächlich auf einem sehr gutem Tutorium von *Philipp Lippe* ([link](https://github.com/phlippe/uvadlc_notebooks/blob/master/docs/tutorial_notebooks/tutorial2/Introduction_to_PyTorch.ipynb))."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3298ca1d",
   "metadata": {},
   "source": [
    "Wir starten mit dem Import von dem Paket, genannt ``torch``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "2c0b4dd6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using torch 2.8.0+cpu\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "print(\"Using torch\", torch.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27b54dcb",
   "metadata": {},
   "source": [
    "Sollte die obige Zelle nicht ausführbar sein, so müssen wir PyTorch zuerst installieren. Einen Command dazu kann man sich [hier](https://pytorch.org/get-started/locally/) erstellen lassen. Wichtig ist die GPU (Cuda) Funktionalität, welche man unbedingt auswählen sollte, sofern man eine (unterstützte) NVIDIA Grafikkarte besitzt."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b2a6273",
   "metadata": {},
   "source": [
    "**Hinweis:** Fast alle gängigen NVIDIA Grafikkarten werden unterstützt. Es wird auch Apple-Silicon unterstützt."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afc7f17e",
   "metadata": {},
   "source": [
    "Nun kommen wir auch schon zu den wichtigsten Elementen in PyTorch (und somit quasi auch für unsere Neuronalen Netzwerke später), den ***Tensoren***."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "098d808d",
   "metadata": {},
   "source": [
    "#### Tensoren"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b7f6432",
   "metadata": {},
   "source": [
    "* sind eine Generalisierung von Matrizen bzw. Vektoren (also mathematische Objekte)\n",
    "* prinzipiell komplizierte Objekte, jedoch für unsere Zwecke mit Kenntnis von Matrizen und Vektoren recht leicht zu bedienen.\n",
    "* namensgebend auch zum Beispiel für die Bibliothek ``Tensorflow`` (hat den gleichen Zweck wie zBsp PyTorch)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d7a78dc",
   "metadata": {},
   "source": [
    "![Tensor](../resources/Tensor_1.png)\n",
    "\n",
    "(von https://databasecamp.de/python/tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb3d930a",
   "metadata": {},
   "source": [
    "Wie könnte der ganz rechte Tensor vom obigen Bild aussehen in der Praxis?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9df1ee8",
   "metadata": {},
   "source": [
    "![Columbus_Tensored](../resources/Columbus_Tensor.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ca1adcc",
   "metadata": {},
   "source": [
    "Eine andere Ansicht zeigt das nochmal allgemeiner (und von einer anderen Sichtweise im Vergleich zu vorher)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d2518ca",
   "metadata": {},
   "source": [
    "![Tensor_2](../resources/Tensor_2.png)\n",
    "\n",
    "(von https://brainpy.readthedocs.io/en/brainpy-1.1.x/tutorial_math/tensors.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35b7d883",
   "metadata": {},
   "source": [
    "Wir siehen hier auch die verschiedenen Dimensionen, genannt ``axis``. In welcher Reihenfolge die axis numeriert werden, hängt von der *shape* ab. \n",
    "\n",
    "So ist im obigen Bild beim \"3D tensor\" *axis 0* jene axis, die zum ersten Eintrag der Shape *(4, 3, 2)* gehört.\n",
    "\n",
    "**Wichtig:** Wir können auch in NumPy mit *Tensoren* arbeiten, jedoch werden sie dort nicht Tensoren genannt, sondern nach wie vor einfach *arrays*. Sprich auch in Numpy können wir 3d arrays (können sogar $n$-dimensional sein) arbeiten!\n",
    "\n",
    "\n",
    "\n",
    "Man muss sich in beiden Fällen (NumPy und PyTorch) aber immer bewusst sein, welche Dimension zu welcher Richtung gehört. Das folgende Code-Beispiel sollte das verdeutlichen. Wir verwenden nochmal kurz NumPy am Anfang, danach sehen wir uns das ganze in PyTorch an."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c51bae7",
   "metadata": {},
   "source": [
    "> **Übung:** Finde Beispiele für 3D und 4D Tensoren im Bereich Machine Learning. **Tipp**: Welche Art von Tensor waren unsere bisherigen \"tabellarischen Daten\" immer?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "e03bb183",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.13.1\n",
      "1.6.1\n"
     ]
    }
   ],
   "source": [
    "import scipy\n",
    "import sklearn\n",
    "print(scipy.__version__)\n",
    "print(sklearn.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "c3ed7024",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.datasets as sklearn_datasets\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "ddcc5ad3",
   "metadata": {},
   "outputs": [],
   "source": [
    "iris_data = sklearn_datasets.load_iris()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "4b87663b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'data': array([[5.1, 3.5, 1.4, 0.2],\n",
       "        [4.9, 3. , 1.4, 0.2],\n",
       "        [4.7, 3.2, 1.3, 0.2],\n",
       "        [4.6, 3.1, 1.5, 0.2],\n",
       "        [5. , 3.6, 1.4, 0.2],\n",
       "        [5.4, 3.9, 1.7, 0.4],\n",
       "        [4.6, 3.4, 1.4, 0.3],\n",
       "        [5. , 3.4, 1.5, 0.2],\n",
       "        [4.4, 2.9, 1.4, 0.2],\n",
       "        [4.9, 3.1, 1.5, 0.1],\n",
       "        [5.4, 3.7, 1.5, 0.2],\n",
       "        [4.8, 3.4, 1.6, 0.2],\n",
       "        [4.8, 3. , 1.4, 0.1],\n",
       "        [4.3, 3. , 1.1, 0.1],\n",
       "        [5.8, 4. , 1.2, 0.2],\n",
       "        [5.7, 4.4, 1.5, 0.4],\n",
       "        [5.4, 3.9, 1.3, 0.4],\n",
       "        [5.1, 3.5, 1.4, 0.3],\n",
       "        [5.7, 3.8, 1.7, 0.3],\n",
       "        [5.1, 3.8, 1.5, 0.3],\n",
       "        [5.4, 3.4, 1.7, 0.2],\n",
       "        [5.1, 3.7, 1.5, 0.4],\n",
       "        [4.6, 3.6, 1. , 0.2],\n",
       "        [5.1, 3.3, 1.7, 0.5],\n",
       "        [4.8, 3.4, 1.9, 0.2],\n",
       "        [5. , 3. , 1.6, 0.2],\n",
       "        [5. , 3.4, 1.6, 0.4],\n",
       "        [5.2, 3.5, 1.5, 0.2],\n",
       "        [5.2, 3.4, 1.4, 0.2],\n",
       "        [4.7, 3.2, 1.6, 0.2],\n",
       "        [4.8, 3.1, 1.6, 0.2],\n",
       "        [5.4, 3.4, 1.5, 0.4],\n",
       "        [5.2, 4.1, 1.5, 0.1],\n",
       "        [5.5, 4.2, 1.4, 0.2],\n",
       "        [4.9, 3.1, 1.5, 0.2],\n",
       "        [5. , 3.2, 1.2, 0.2],\n",
       "        [5.5, 3.5, 1.3, 0.2],\n",
       "        [4.9, 3.6, 1.4, 0.1],\n",
       "        [4.4, 3. , 1.3, 0.2],\n",
       "        [5.1, 3.4, 1.5, 0.2],\n",
       "        [5. , 3.5, 1.3, 0.3],\n",
       "        [4.5, 2.3, 1.3, 0.3],\n",
       "        [4.4, 3.2, 1.3, 0.2],\n",
       "        [5. , 3.5, 1.6, 0.6],\n",
       "        [5.1, 3.8, 1.9, 0.4],\n",
       "        [4.8, 3. , 1.4, 0.3],\n",
       "        [5.1, 3.8, 1.6, 0.2],\n",
       "        [4.6, 3.2, 1.4, 0.2],\n",
       "        [5.3, 3.7, 1.5, 0.2],\n",
       "        [5. , 3.3, 1.4, 0.2],\n",
       "        [7. , 3.2, 4.7, 1.4],\n",
       "        [6.4, 3.2, 4.5, 1.5],\n",
       "        [6.9, 3.1, 4.9, 1.5],\n",
       "        [5.5, 2.3, 4. , 1.3],\n",
       "        [6.5, 2.8, 4.6, 1.5],\n",
       "        [5.7, 2.8, 4.5, 1.3],\n",
       "        [6.3, 3.3, 4.7, 1.6],\n",
       "        [4.9, 2.4, 3.3, 1. ],\n",
       "        [6.6, 2.9, 4.6, 1.3],\n",
       "        [5.2, 2.7, 3.9, 1.4],\n",
       "        [5. , 2. , 3.5, 1. ],\n",
       "        [5.9, 3. , 4.2, 1.5],\n",
       "        [6. , 2.2, 4. , 1. ],\n",
       "        [6.1, 2.9, 4.7, 1.4],\n",
       "        [5.6, 2.9, 3.6, 1.3],\n",
       "        [6.7, 3.1, 4.4, 1.4],\n",
       "        [5.6, 3. , 4.5, 1.5],\n",
       "        [5.8, 2.7, 4.1, 1. ],\n",
       "        [6.2, 2.2, 4.5, 1.5],\n",
       "        [5.6, 2.5, 3.9, 1.1],\n",
       "        [5.9, 3.2, 4.8, 1.8],\n",
       "        [6.1, 2.8, 4. , 1.3],\n",
       "        [6.3, 2.5, 4.9, 1.5],\n",
       "        [6.1, 2.8, 4.7, 1.2],\n",
       "        [6.4, 2.9, 4.3, 1.3],\n",
       "        [6.6, 3. , 4.4, 1.4],\n",
       "        [6.8, 2.8, 4.8, 1.4],\n",
       "        [6.7, 3. , 5. , 1.7],\n",
       "        [6. , 2.9, 4.5, 1.5],\n",
       "        [5.7, 2.6, 3.5, 1. ],\n",
       "        [5.5, 2.4, 3.8, 1.1],\n",
       "        [5.5, 2.4, 3.7, 1. ],\n",
       "        [5.8, 2.7, 3.9, 1.2],\n",
       "        [6. , 2.7, 5.1, 1.6],\n",
       "        [5.4, 3. , 4.5, 1.5],\n",
       "        [6. , 3.4, 4.5, 1.6],\n",
       "        [6.7, 3.1, 4.7, 1.5],\n",
       "        [6.3, 2.3, 4.4, 1.3],\n",
       "        [5.6, 3. , 4.1, 1.3],\n",
       "        [5.5, 2.5, 4. , 1.3],\n",
       "        [5.5, 2.6, 4.4, 1.2],\n",
       "        [6.1, 3. , 4.6, 1.4],\n",
       "        [5.8, 2.6, 4. , 1.2],\n",
       "        [5. , 2.3, 3.3, 1. ],\n",
       "        [5.6, 2.7, 4.2, 1.3],\n",
       "        [5.7, 3. , 4.2, 1.2],\n",
       "        [5.7, 2.9, 4.2, 1.3],\n",
       "        [6.2, 2.9, 4.3, 1.3],\n",
       "        [5.1, 2.5, 3. , 1.1],\n",
       "        [5.7, 2.8, 4.1, 1.3],\n",
       "        [6.3, 3.3, 6. , 2.5],\n",
       "        [5.8, 2.7, 5.1, 1.9],\n",
       "        [7.1, 3. , 5.9, 2.1],\n",
       "        [6.3, 2.9, 5.6, 1.8],\n",
       "        [6.5, 3. , 5.8, 2.2],\n",
       "        [7.6, 3. , 6.6, 2.1],\n",
       "        [4.9, 2.5, 4.5, 1.7],\n",
       "        [7.3, 2.9, 6.3, 1.8],\n",
       "        [6.7, 2.5, 5.8, 1.8],\n",
       "        [7.2, 3.6, 6.1, 2.5],\n",
       "        [6.5, 3.2, 5.1, 2. ],\n",
       "        [6.4, 2.7, 5.3, 1.9],\n",
       "        [6.8, 3. , 5.5, 2.1],\n",
       "        [5.7, 2.5, 5. , 2. ],\n",
       "        [5.8, 2.8, 5.1, 2.4],\n",
       "        [6.4, 3.2, 5.3, 2.3],\n",
       "        [6.5, 3. , 5.5, 1.8],\n",
       "        [7.7, 3.8, 6.7, 2.2],\n",
       "        [7.7, 2.6, 6.9, 2.3],\n",
       "        [6. , 2.2, 5. , 1.5],\n",
       "        [6.9, 3.2, 5.7, 2.3],\n",
       "        [5.6, 2.8, 4.9, 2. ],\n",
       "        [7.7, 2.8, 6.7, 2. ],\n",
       "        [6.3, 2.7, 4.9, 1.8],\n",
       "        [6.7, 3.3, 5.7, 2.1],\n",
       "        [7.2, 3.2, 6. , 1.8],\n",
       "        [6.2, 2.8, 4.8, 1.8],\n",
       "        [6.1, 3. , 4.9, 1.8],\n",
       "        [6.4, 2.8, 5.6, 2.1],\n",
       "        [7.2, 3. , 5.8, 1.6],\n",
       "        [7.4, 2.8, 6.1, 1.9],\n",
       "        [7.9, 3.8, 6.4, 2. ],\n",
       "        [6.4, 2.8, 5.6, 2.2],\n",
       "        [6.3, 2.8, 5.1, 1.5],\n",
       "        [6.1, 2.6, 5.6, 1.4],\n",
       "        [7.7, 3. , 6.1, 2.3],\n",
       "        [6.3, 3.4, 5.6, 2.4],\n",
       "        [6.4, 3.1, 5.5, 1.8],\n",
       "        [6. , 3. , 4.8, 1.8],\n",
       "        [6.9, 3.1, 5.4, 2.1],\n",
       "        [6.7, 3.1, 5.6, 2.4],\n",
       "        [6.9, 3.1, 5.1, 2.3],\n",
       "        [5.8, 2.7, 5.1, 1.9],\n",
       "        [6.8, 3.2, 5.9, 2.3],\n",
       "        [6.7, 3.3, 5.7, 2.5],\n",
       "        [6.7, 3. , 5.2, 2.3],\n",
       "        [6.3, 2.5, 5. , 1.9],\n",
       "        [6.5, 3. , 5.2, 2. ],\n",
       "        [6.2, 3.4, 5.4, 2.3],\n",
       "        [5.9, 3. , 5.1, 1.8]]),\n",
       " 'target': array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "        2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "        2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]),\n",
       " 'frame': None,\n",
       " 'target_names': array(['setosa', 'versicolor', 'virginica'], dtype='<U10'),\n",
       " 'DESCR': '.. _iris_dataset:\\n\\nIris plants dataset\\n--------------------\\n\\n**Data Set Characteristics:**\\n\\n:Number of Instances: 150 (50 in each of three classes)\\n:Number of Attributes: 4 numeric, predictive attributes and the class\\n:Attribute Information:\\n    - sepal length in cm\\n    - sepal width in cm\\n    - petal length in cm\\n    - petal width in cm\\n    - class:\\n            - Iris-Setosa\\n            - Iris-Versicolour\\n            - Iris-Virginica\\n\\n:Summary Statistics:\\n\\n============== ==== ==== ======= ===== ====================\\n                Min  Max   Mean    SD   Class Correlation\\n============== ==== ==== ======= ===== ====================\\nsepal length:   4.3  7.9   5.84   0.83    0.7826\\nsepal width:    2.0  4.4   3.05   0.43   -0.4194\\npetal length:   1.0  6.9   3.76   1.76    0.9490  (high!)\\npetal width:    0.1  2.5   1.20   0.76    0.9565  (high!)\\n============== ==== ==== ======= ===== ====================\\n\\n:Missing Attribute Values: None\\n:Class Distribution: 33.3% for each of 3 classes.\\n:Creator: R.A. Fisher\\n:Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\\n:Date: July, 1988\\n\\nThe famous Iris database, first used by Sir R.A. Fisher. The dataset is taken\\nfrom Fisher\\'s paper. Note that it\\'s the same as in R, but not as in the UCI\\nMachine Learning Repository, which has two wrong data points.\\n\\nThis is perhaps the best known database to be found in the\\npattern recognition literature.  Fisher\\'s paper is a classic in the field and\\nis referenced frequently to this day.  (See Duda & Hart, for example.)  The\\ndata set contains 3 classes of 50 instances each, where each class refers to a\\ntype of iris plant.  One class is linearly separable from the other 2; the\\nlatter are NOT linearly separable from each other.\\n\\n.. dropdown:: References\\n\\n  - Fisher, R.A. \"The use of multiple measurements in taxonomic problems\"\\n    Annual Eugenics, 7, Part II, 179-188 (1936); also in \"Contributions to\\n    Mathematical Statistics\" (John Wiley, NY, 1950).\\n  - Duda, R.O., & Hart, P.E. (1973) Pattern Classification and Scene Analysis.\\n    (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.\\n  - Dasarathy, B.V. (1980) \"Nosing Around the Neighborhood: A New System\\n    Structure and Classification Rule for Recognition in Partially Exposed\\n    Environments\".  IEEE Transactions on Pattern Analysis and Machine\\n    Intelligence, Vol. PAMI-2, No. 1, 67-71.\\n  - Gates, G.W. (1972) \"The Reduced Nearest Neighbor Rule\".  IEEE Transactions\\n    on Information Theory, May 1972, 431-433.\\n  - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al\"s AUTOCLASS II\\n    conceptual clustering system finds 3 classes in the data.\\n  - Many, many more ...\\n',\n",
       " 'feature_names': ['sepal length (cm)',\n",
       "  'sepal width (cm)',\n",
       "  'petal length (cm)',\n",
       "  'petal width (cm)'],\n",
       " 'filename': 'iris.csv',\n",
       " 'data_module': 'sklearn.datasets.data'}"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91c0606d",
   "metadata": {},
   "source": [
    "Wir sind interessiert am ``.data`` Attribut, da es das eigentliche Array beinhaltet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "2360dd93",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = iris_data.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "e981dd7b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5.1 3.5 1.4 0.2]\n",
      " [4.9 3.  1.4 0.2]\n",
      " [4.7 3.2 1.3 0.2]\n",
      " [4.6 3.1 1.5 0.2]\n",
      " [5.  3.6 1.4 0.2]\n",
      " [5.4 3.9 1.7 0.4]\n",
      " [4.6 3.4 1.4 0.3]\n",
      " [5.  3.4 1.5 0.2]\n",
      " [4.4 2.9 1.4 0.2]\n",
      " [4.9 3.1 1.5 0.1]\n",
      " [5.4 3.7 1.5 0.2]\n",
      " [4.8 3.4 1.6 0.2]\n",
      " [4.8 3.  1.4 0.1]\n",
      " [4.3 3.  1.1 0.1]\n",
      " [5.8 4.  1.2 0.2]\n",
      " [5.7 4.4 1.5 0.4]\n",
      " [5.4 3.9 1.3 0.4]\n",
      " [5.1 3.5 1.4 0.3]\n",
      " [5.7 3.8 1.7 0.3]\n",
      " [5.1 3.8 1.5 0.3]\n",
      " [5.4 3.4 1.7 0.2]\n",
      " [5.1 3.7 1.5 0.4]\n",
      " [4.6 3.6 1.  0.2]\n",
      " [5.1 3.3 1.7 0.5]\n",
      " [4.8 3.4 1.9 0.2]\n",
      " [5.  3.  1.6 0.2]\n",
      " [5.  3.4 1.6 0.4]\n",
      " [5.2 3.5 1.5 0.2]\n",
      " [5.2 3.4 1.4 0.2]\n",
      " [4.7 3.2 1.6 0.2]\n",
      " [4.8 3.1 1.6 0.2]\n",
      " [5.4 3.4 1.5 0.4]\n",
      " [5.2 4.1 1.5 0.1]\n",
      " [5.5 4.2 1.4 0.2]\n",
      " [4.9 3.1 1.5 0.2]\n",
      " [5.  3.2 1.2 0.2]\n",
      " [5.5 3.5 1.3 0.2]\n",
      " [4.9 3.6 1.4 0.1]\n",
      " [4.4 3.  1.3 0.2]\n",
      " [5.1 3.4 1.5 0.2]\n",
      " [5.  3.5 1.3 0.3]\n",
      " [4.5 2.3 1.3 0.3]\n",
      " [4.4 3.2 1.3 0.2]\n",
      " [5.  3.5 1.6 0.6]\n",
      " [5.1 3.8 1.9 0.4]\n",
      " [4.8 3.  1.4 0.3]\n",
      " [5.1 3.8 1.6 0.2]\n",
      " [4.6 3.2 1.4 0.2]\n",
      " [5.3 3.7 1.5 0.2]\n",
      " [5.  3.3 1.4 0.2]\n",
      " [7.  3.2 4.7 1.4]\n",
      " [6.4 3.2 4.5 1.5]\n",
      " [6.9 3.1 4.9 1.5]\n",
      " [5.5 2.3 4.  1.3]\n",
      " [6.5 2.8 4.6 1.5]\n",
      " [5.7 2.8 4.5 1.3]\n",
      " [6.3 3.3 4.7 1.6]\n",
      " [4.9 2.4 3.3 1. ]\n",
      " [6.6 2.9 4.6 1.3]\n",
      " [5.2 2.7 3.9 1.4]\n",
      " [5.  2.  3.5 1. ]\n",
      " [5.9 3.  4.2 1.5]\n",
      " [6.  2.2 4.  1. ]\n",
      " [6.1 2.9 4.7 1.4]\n",
      " [5.6 2.9 3.6 1.3]\n",
      " [6.7 3.1 4.4 1.4]\n",
      " [5.6 3.  4.5 1.5]\n",
      " [5.8 2.7 4.1 1. ]\n",
      " [6.2 2.2 4.5 1.5]\n",
      " [5.6 2.5 3.9 1.1]\n",
      " [5.9 3.2 4.8 1.8]\n",
      " [6.1 2.8 4.  1.3]\n",
      " [6.3 2.5 4.9 1.5]\n",
      " [6.1 2.8 4.7 1.2]\n",
      " [6.4 2.9 4.3 1.3]\n",
      " [6.6 3.  4.4 1.4]\n",
      " [6.8 2.8 4.8 1.4]\n",
      " [6.7 3.  5.  1.7]\n",
      " [6.  2.9 4.5 1.5]\n",
      " [5.7 2.6 3.5 1. ]\n",
      " [5.5 2.4 3.8 1.1]\n",
      " [5.5 2.4 3.7 1. ]\n",
      " [5.8 2.7 3.9 1.2]\n",
      " [6.  2.7 5.1 1.6]\n",
      " [5.4 3.  4.5 1.5]\n",
      " [6.  3.4 4.5 1.6]\n",
      " [6.7 3.1 4.7 1.5]\n",
      " [6.3 2.3 4.4 1.3]\n",
      " [5.6 3.  4.1 1.3]\n",
      " [5.5 2.5 4.  1.3]\n",
      " [5.5 2.6 4.4 1.2]\n",
      " [6.1 3.  4.6 1.4]\n",
      " [5.8 2.6 4.  1.2]\n",
      " [5.  2.3 3.3 1. ]\n",
      " [5.6 2.7 4.2 1.3]\n",
      " [5.7 3.  4.2 1.2]\n",
      " [5.7 2.9 4.2 1.3]\n",
      " [6.2 2.9 4.3 1.3]\n",
      " [5.1 2.5 3.  1.1]\n",
      " [5.7 2.8 4.1 1.3]\n",
      " [6.3 3.3 6.  2.5]\n",
      " [5.8 2.7 5.1 1.9]\n",
      " [7.1 3.  5.9 2.1]\n",
      " [6.3 2.9 5.6 1.8]\n",
      " [6.5 3.  5.8 2.2]\n",
      " [7.6 3.  6.6 2.1]\n",
      " [4.9 2.5 4.5 1.7]\n",
      " [7.3 2.9 6.3 1.8]\n",
      " [6.7 2.5 5.8 1.8]\n",
      " [7.2 3.6 6.1 2.5]\n",
      " [6.5 3.2 5.1 2. ]\n",
      " [6.4 2.7 5.3 1.9]\n",
      " [6.8 3.  5.5 2.1]\n",
      " [5.7 2.5 5.  2. ]\n",
      " [5.8 2.8 5.1 2.4]\n",
      " [6.4 3.2 5.3 2.3]\n",
      " [6.5 3.  5.5 1.8]\n",
      " [7.7 3.8 6.7 2.2]\n",
      " [7.7 2.6 6.9 2.3]\n",
      " [6.  2.2 5.  1.5]\n",
      " [6.9 3.2 5.7 2.3]\n",
      " [5.6 2.8 4.9 2. ]\n",
      " [7.7 2.8 6.7 2. ]\n",
      " [6.3 2.7 4.9 1.8]\n",
      " [6.7 3.3 5.7 2.1]\n",
      " [7.2 3.2 6.  1.8]\n",
      " [6.2 2.8 4.8 1.8]\n",
      " [6.1 3.  4.9 1.8]\n",
      " [6.4 2.8 5.6 2.1]\n",
      " [7.2 3.  5.8 1.6]\n",
      " [7.4 2.8 6.1 1.9]\n",
      " [7.9 3.8 6.4 2. ]\n",
      " [6.4 2.8 5.6 2.2]\n",
      " [6.3 2.8 5.1 1.5]\n",
      " [6.1 2.6 5.6 1.4]\n",
      " [7.7 3.  6.1 2.3]\n",
      " [6.3 3.4 5.6 2.4]\n",
      " [6.4 3.1 5.5 1.8]\n",
      " [6.  3.  4.8 1.8]\n",
      " [6.9 3.1 5.4 2.1]\n",
      " [6.7 3.1 5.6 2.4]\n",
      " [6.9 3.1 5.1 2.3]\n",
      " [5.8 2.7 5.1 1.9]\n",
      " [6.8 3.2 5.9 2.3]\n",
      " [6.7 3.3 5.7 2.5]\n",
      " [6.7 3.  5.2 2.3]\n",
      " [6.3 2.5 5.  1.9]\n",
      " [6.5 3.  5.2 2. ]\n",
      " [6.2 3.4 5.4 2.3]\n",
      " [5.9 3.  5.1 1.8]]\n"
     ]
    }
   ],
   "source": [
    "print(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "31a1ffaa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(150, 4)\n",
      "<class 'numpy.ndarray'>\n",
      "2\n"
     ]
    }
   ],
   "source": [
    "print(X.shape)\n",
    "print(type(X))\n",
    "print(X.ndim)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed4eebcc",
   "metadata": {},
   "source": [
    "Welche Ordnung hat also dieses Array (bzw. dieser Tensor)?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5329e3f",
   "metadata": {},
   "source": [
    "Nun hat die erste Axis (axis 0), also 150 Einträge und besteht also aus unseren verschiedenen Datenpunkten. Wobei jeder Datenpunkt ein Vektor von 4 Einträgen ist. Also haben wir 4 Features. Es ist also in unserem Fall die erste Axis (axis 0) die Datenpunkt-Axis und die zweite Axis (axis 1) die Feature axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "d3e67c00",
   "metadata": {},
   "outputs": [],
   "source": [
    "# What happens when we run this code?\n",
    "X_aggregated = np.mean(X, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "ada8690f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5.84333333 3.05733333 3.758      1.19933333]\n",
      "(4,)\n"
     ]
    }
   ],
   "source": [
    "# How large do you think is now the resulting array \"X_aggregated\"?\n",
    "print(X_aggregated)\n",
    "print(X_aggregated.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05a6dc60",
   "metadata": {},
   "source": [
    "Was wäre, wenn wir ``axis=1`` verwendet hätten?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "dd5e0707",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_aggregated_2 = np.mean(X, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "790181da",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2.55  2.375 2.35  2.35  2.55  2.85  2.425 2.525 2.225 2.4   2.7   2.5\n",
      " 2.325 2.125 2.8   3.    2.75  2.575 2.875 2.675 2.675 2.675 2.35  2.65\n",
      " 2.575 2.45  2.6   2.6   2.55  2.425 2.425 2.675 2.725 2.825 2.425 2.4\n",
      " 2.625 2.5   2.225 2.55  2.525 2.1   2.275 2.675 2.8   2.375 2.675 2.35\n",
      " 2.675 2.475 4.075 3.9   4.1   3.275 3.85  3.575 3.975 2.9   3.85  3.3\n",
      " 2.875 3.65  3.3   3.775 3.35  3.9   3.65  3.4   3.6   3.275 3.925 3.55\n",
      " 3.8   3.7   3.725 3.85  3.95  4.1   3.725 3.2   3.2   3.15  3.4   3.85\n",
      " 3.6   3.875 4.    3.575 3.5   3.325 3.425 3.775 3.4   2.9   3.45  3.525\n",
      " 3.525 3.675 2.925 3.475 4.525 3.875 4.525 4.15  4.375 4.825 3.4   4.575\n",
      " 4.2   4.85  4.2   4.075 4.35  3.8   4.025 4.3   4.2   5.1   4.875 3.675\n",
      " 4.525 3.825 4.8   3.925 4.45  4.55  3.9   3.95  4.225 4.4   4.55  5.025\n",
      " 4.25  3.925 3.925 4.775 4.425 4.2   3.9   4.375 4.45  4.35  3.875 4.55\n",
      " 4.55  4.3   3.925 4.175 4.325 3.95 ]\n",
      "(150,)\n"
     ]
    }
   ],
   "source": [
    "# How large do you think is now the resulting array \"X_aggregated\"?\n",
    "print(X_aggregated_2)\n",
    "print(X_aggregated_2.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c3fe9ba",
   "metadata": {},
   "source": [
    "Und natürlich können wir auch die Richtung umdrehen. Sprich in unserem Fall die Matrix $X$ transponieren.\n",
    "\n",
    "**Hinweis:** Das Transponieren von Tensoren funktioniert auch in höheren Dimensionen. Hier muss man jedoch eine Reihenfolge angeben, in der die verschiedenen *axis* getauscht werden sollen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "bb9b954a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5.1 4.9 4.7 4.6 5.  5.4 4.6 5.  4.4 4.9 5.4 4.8 4.8 4.3 5.8 5.7 5.4 5.1\n",
      "  5.7 5.1 5.4 5.1 4.6 5.1 4.8 5.  5.  5.2 5.2 4.7 4.8 5.4 5.2 5.5 4.9 5.\n",
      "  5.5 4.9 4.4 5.1 5.  4.5 4.4 5.  5.1 4.8 5.1 4.6 5.3 5.  7.  6.4 6.9 5.5\n",
      "  6.5 5.7 6.3 4.9 6.6 5.2 5.  5.9 6.  6.1 5.6 6.7 5.6 5.8 6.2 5.6 5.9 6.1\n",
      "  6.3 6.1 6.4 6.6 6.8 6.7 6.  5.7 5.5 5.5 5.8 6.  5.4 6.  6.7 6.3 5.6 5.5\n",
      "  5.5 6.1 5.8 5.  5.6 5.7 5.7 6.2 5.1 5.7 6.3 5.8 7.1 6.3 6.5 7.6 4.9 7.3\n",
      "  6.7 7.2 6.5 6.4 6.8 5.7 5.8 6.4 6.5 7.7 7.7 6.  6.9 5.6 7.7 6.3 6.7 7.2\n",
      "  6.2 6.1 6.4 7.2 7.4 7.9 6.4 6.3 6.1 7.7 6.3 6.4 6.  6.9 6.7 6.9 5.8 6.8\n",
      "  6.7 6.7 6.3 6.5 6.2 5.9]\n",
      " [3.5 3.  3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 3.7 3.4 3.  3.  4.  4.4 3.9 3.5\n",
      "  3.8 3.8 3.4 3.7 3.6 3.3 3.4 3.  3.4 3.5 3.4 3.2 3.1 3.4 4.1 4.2 3.1 3.2\n",
      "  3.5 3.6 3.  3.4 3.5 2.3 3.2 3.5 3.8 3.  3.8 3.2 3.7 3.3 3.2 3.2 3.1 2.3\n",
      "  2.8 2.8 3.3 2.4 2.9 2.7 2.  3.  2.2 2.9 2.9 3.1 3.  2.7 2.2 2.5 3.2 2.8\n",
      "  2.5 2.8 2.9 3.  2.8 3.  2.9 2.6 2.4 2.4 2.7 2.7 3.  3.4 3.1 2.3 3.  2.5\n",
      "  2.6 3.  2.6 2.3 2.7 3.  2.9 2.9 2.5 2.8 3.3 2.7 3.  2.9 3.  3.  2.5 2.9\n",
      "  2.5 3.6 3.2 2.7 3.  2.5 2.8 3.2 3.  3.8 2.6 2.2 3.2 2.8 2.8 2.7 3.3 3.2\n",
      "  2.8 3.  2.8 3.  2.8 3.8 2.8 2.8 2.6 3.  3.4 3.1 3.  3.1 3.1 3.1 2.7 3.2\n",
      "  3.3 3.  2.5 3.  3.4 3. ]\n",
      " [1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 1.5 1.6 1.4 1.1 1.2 1.5 1.3 1.4\n",
      "  1.7 1.5 1.7 1.5 1.  1.7 1.9 1.6 1.6 1.5 1.4 1.6 1.6 1.5 1.5 1.4 1.5 1.2\n",
      "  1.3 1.4 1.3 1.5 1.3 1.3 1.3 1.6 1.9 1.4 1.6 1.4 1.5 1.4 4.7 4.5 4.9 4.\n",
      "  4.6 4.5 4.7 3.3 4.6 3.9 3.5 4.2 4.  4.7 3.6 4.4 4.5 4.1 4.5 3.9 4.8 4.\n",
      "  4.9 4.7 4.3 4.4 4.8 5.  4.5 3.5 3.8 3.7 3.9 5.1 4.5 4.5 4.7 4.4 4.1 4.\n",
      "  4.4 4.6 4.  3.3 4.2 4.2 4.2 4.3 3.  4.1 6.  5.1 5.9 5.6 5.8 6.6 4.5 6.3\n",
      "  5.8 6.1 5.1 5.3 5.5 5.  5.1 5.3 5.5 6.7 6.9 5.  5.7 4.9 6.7 4.9 5.7 6.\n",
      "  4.8 4.9 5.6 5.8 6.1 6.4 5.6 5.1 5.6 6.1 5.6 5.5 4.8 5.4 5.6 5.1 5.1 5.9\n",
      "  5.7 5.2 5.  5.2 5.4 5.1]\n",
      " [0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 0.2 0.2 0.1 0.1 0.2 0.4 0.4 0.3\n",
      "  0.3 0.3 0.2 0.4 0.2 0.5 0.2 0.2 0.4 0.2 0.2 0.2 0.2 0.4 0.1 0.2 0.2 0.2\n",
      "  0.2 0.1 0.2 0.2 0.3 0.3 0.2 0.6 0.4 0.3 0.2 0.2 0.2 0.2 1.4 1.5 1.5 1.3\n",
      "  1.5 1.3 1.6 1.  1.3 1.4 1.  1.5 1.  1.4 1.3 1.4 1.5 1.  1.5 1.1 1.8 1.3\n",
      "  1.5 1.2 1.3 1.4 1.4 1.7 1.5 1.  1.1 1.  1.2 1.6 1.5 1.6 1.5 1.3 1.3 1.3\n",
      "  1.2 1.4 1.2 1.  1.3 1.2 1.3 1.3 1.1 1.3 2.5 1.9 2.1 1.8 2.2 2.1 1.7 1.8\n",
      "  1.8 2.5 2.  1.9 2.1 2.  2.4 2.3 1.8 2.2 2.3 1.5 2.3 2.  2.  1.8 2.1 1.8\n",
      "  1.8 1.8 2.1 1.6 1.9 2.  2.2 1.5 1.4 2.3 2.4 1.8 1.8 2.1 2.4 2.3 1.9 2.3\n",
      "  2.5 2.3 1.9 2.  2.3 1.8]]\n",
      "(4, 150)\n"
     ]
    }
   ],
   "source": [
    "flipped_X = X.T \n",
    "print(flipped_X)\n",
    "print(flipped_X.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59ac3246",
   "metadata": {},
   "source": [
    "Was ist nun die axis0 und was ist die axis1?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e18d373f",
   "metadata": {},
   "source": [
    "Wir können aber auch ohne angegebener Axis die Daten aggregieren. Was ist das Ergebnis von der folgenden Code-Zeile?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "372db3a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_aggregated_3 = np.mean(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "7852f8ee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.4644999999999997\n",
      "()\n"
     ]
    }
   ],
   "source": [
    "print(X_aggregated_3)\n",
    "print(X_aggregated_3.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48e09a80",
   "metadata": {},
   "source": [
    "Wie sieht das ganze in PyTorch aus?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "b47afa75",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "ad8d55f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = iris_data.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "c0dd17ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now we convert the numpy array to a PyTorch tensor\n",
    "X_tensor = torch.tensor(X, dtype=torch.float32) # We can also skip the dtype, then it is inferred from the numpy array but it is a good practice to specify the dtype\n",
    "\n",
    "# Another possibility is\n",
    "X_tensor_2 = torch.from_numpy(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "a17f4f3e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[5.1000, 3.5000, 1.4000, 0.2000],\n",
      "        [4.9000, 3.0000, 1.4000, 0.2000],\n",
      "        [4.7000, 3.2000, 1.3000, 0.2000],\n",
      "        [4.6000, 3.1000, 1.5000, 0.2000],\n",
      "        [5.0000, 3.6000, 1.4000, 0.2000],\n",
      "        [5.4000, 3.9000, 1.7000, 0.4000],\n",
      "        [4.6000, 3.4000, 1.4000, 0.3000],\n",
      "        [5.0000, 3.4000, 1.5000, 0.2000],\n",
      "        [4.4000, 2.9000, 1.4000, 0.2000],\n",
      "        [4.9000, 3.1000, 1.5000, 0.1000],\n",
      "        [5.4000, 3.7000, 1.5000, 0.2000],\n",
      "        [4.8000, 3.4000, 1.6000, 0.2000],\n",
      "        [4.8000, 3.0000, 1.4000, 0.1000],\n",
      "        [4.3000, 3.0000, 1.1000, 0.1000],\n",
      "        [5.8000, 4.0000, 1.2000, 0.2000],\n",
      "        [5.7000, 4.4000, 1.5000, 0.4000],\n",
      "        [5.4000, 3.9000, 1.3000, 0.4000],\n",
      "        [5.1000, 3.5000, 1.4000, 0.3000],\n",
      "        [5.7000, 3.8000, 1.7000, 0.3000],\n",
      "        [5.1000, 3.8000, 1.5000, 0.3000],\n",
      "        [5.4000, 3.4000, 1.7000, 0.2000],\n",
      "        [5.1000, 3.7000, 1.5000, 0.4000],\n",
      "        [4.6000, 3.6000, 1.0000, 0.2000],\n",
      "        [5.1000, 3.3000, 1.7000, 0.5000],\n",
      "        [4.8000, 3.4000, 1.9000, 0.2000],\n",
      "        [5.0000, 3.0000, 1.6000, 0.2000],\n",
      "        [5.0000, 3.4000, 1.6000, 0.4000],\n",
      "        [5.2000, 3.5000, 1.5000, 0.2000],\n",
      "        [5.2000, 3.4000, 1.4000, 0.2000],\n",
      "        [4.7000, 3.2000, 1.6000, 0.2000],\n",
      "        [4.8000, 3.1000, 1.6000, 0.2000],\n",
      "        [5.4000, 3.4000, 1.5000, 0.4000],\n",
      "        [5.2000, 4.1000, 1.5000, 0.1000],\n",
      "        [5.5000, 4.2000, 1.4000, 0.2000],\n",
      "        [4.9000, 3.1000, 1.5000, 0.2000],\n",
      "        [5.0000, 3.2000, 1.2000, 0.2000],\n",
      "        [5.5000, 3.5000, 1.3000, 0.2000],\n",
      "        [4.9000, 3.6000, 1.4000, 0.1000],\n",
      "        [4.4000, 3.0000, 1.3000, 0.2000],\n",
      "        [5.1000, 3.4000, 1.5000, 0.2000],\n",
      "        [5.0000, 3.5000, 1.3000, 0.3000],\n",
      "        [4.5000, 2.3000, 1.3000, 0.3000],\n",
      "        [4.4000, 3.2000, 1.3000, 0.2000],\n",
      "        [5.0000, 3.5000, 1.6000, 0.6000],\n",
      "        [5.1000, 3.8000, 1.9000, 0.4000],\n",
      "        [4.8000, 3.0000, 1.4000, 0.3000],\n",
      "        [5.1000, 3.8000, 1.6000, 0.2000],\n",
      "        [4.6000, 3.2000, 1.4000, 0.2000],\n",
      "        [5.3000, 3.7000, 1.5000, 0.2000],\n",
      "        [5.0000, 3.3000, 1.4000, 0.2000],\n",
      "        [7.0000, 3.2000, 4.7000, 1.4000],\n",
      "        [6.4000, 3.2000, 4.5000, 1.5000],\n",
      "        [6.9000, 3.1000, 4.9000, 1.5000],\n",
      "        [5.5000, 2.3000, 4.0000, 1.3000],\n",
      "        [6.5000, 2.8000, 4.6000, 1.5000],\n",
      "        [5.7000, 2.8000, 4.5000, 1.3000],\n",
      "        [6.3000, 3.3000, 4.7000, 1.6000],\n",
      "        [4.9000, 2.4000, 3.3000, 1.0000],\n",
      "        [6.6000, 2.9000, 4.6000, 1.3000],\n",
      "        [5.2000, 2.7000, 3.9000, 1.4000],\n",
      "        [5.0000, 2.0000, 3.5000, 1.0000],\n",
      "        [5.9000, 3.0000, 4.2000, 1.5000],\n",
      "        [6.0000, 2.2000, 4.0000, 1.0000],\n",
      "        [6.1000, 2.9000, 4.7000, 1.4000],\n",
      "        [5.6000, 2.9000, 3.6000, 1.3000],\n",
      "        [6.7000, 3.1000, 4.4000, 1.4000],\n",
      "        [5.6000, 3.0000, 4.5000, 1.5000],\n",
      "        [5.8000, 2.7000, 4.1000, 1.0000],\n",
      "        [6.2000, 2.2000, 4.5000, 1.5000],\n",
      "        [5.6000, 2.5000, 3.9000, 1.1000],\n",
      "        [5.9000, 3.2000, 4.8000, 1.8000],\n",
      "        [6.1000, 2.8000, 4.0000, 1.3000],\n",
      "        [6.3000, 2.5000, 4.9000, 1.5000],\n",
      "        [6.1000, 2.8000, 4.7000, 1.2000],\n",
      "        [6.4000, 2.9000, 4.3000, 1.3000],\n",
      "        [6.6000, 3.0000, 4.4000, 1.4000],\n",
      "        [6.8000, 2.8000, 4.8000, 1.4000],\n",
      "        [6.7000, 3.0000, 5.0000, 1.7000],\n",
      "        [6.0000, 2.9000, 4.5000, 1.5000],\n",
      "        [5.7000, 2.6000, 3.5000, 1.0000],\n",
      "        [5.5000, 2.4000, 3.8000, 1.1000],\n",
      "        [5.5000, 2.4000, 3.7000, 1.0000],\n",
      "        [5.8000, 2.7000, 3.9000, 1.2000],\n",
      "        [6.0000, 2.7000, 5.1000, 1.6000],\n",
      "        [5.4000, 3.0000, 4.5000, 1.5000],\n",
      "        [6.0000, 3.4000, 4.5000, 1.6000],\n",
      "        [6.7000, 3.1000, 4.7000, 1.5000],\n",
      "        [6.3000, 2.3000, 4.4000, 1.3000],\n",
      "        [5.6000, 3.0000, 4.1000, 1.3000],\n",
      "        [5.5000, 2.5000, 4.0000, 1.3000],\n",
      "        [5.5000, 2.6000, 4.4000, 1.2000],\n",
      "        [6.1000, 3.0000, 4.6000, 1.4000],\n",
      "        [5.8000, 2.6000, 4.0000, 1.2000],\n",
      "        [5.0000, 2.3000, 3.3000, 1.0000],\n",
      "        [5.6000, 2.7000, 4.2000, 1.3000],\n",
      "        [5.7000, 3.0000, 4.2000, 1.2000],\n",
      "        [5.7000, 2.9000, 4.2000, 1.3000],\n",
      "        [6.2000, 2.9000, 4.3000, 1.3000],\n",
      "        [5.1000, 2.5000, 3.0000, 1.1000],\n",
      "        [5.7000, 2.8000, 4.1000, 1.3000],\n",
      "        [6.3000, 3.3000, 6.0000, 2.5000],\n",
      "        [5.8000, 2.7000, 5.1000, 1.9000],\n",
      "        [7.1000, 3.0000, 5.9000, 2.1000],\n",
      "        [6.3000, 2.9000, 5.6000, 1.8000],\n",
      "        [6.5000, 3.0000, 5.8000, 2.2000],\n",
      "        [7.6000, 3.0000, 6.6000, 2.1000],\n",
      "        [4.9000, 2.5000, 4.5000, 1.7000],\n",
      "        [7.3000, 2.9000, 6.3000, 1.8000],\n",
      "        [6.7000, 2.5000, 5.8000, 1.8000],\n",
      "        [7.2000, 3.6000, 6.1000, 2.5000],\n",
      "        [6.5000, 3.2000, 5.1000, 2.0000],\n",
      "        [6.4000, 2.7000, 5.3000, 1.9000],\n",
      "        [6.8000, 3.0000, 5.5000, 2.1000],\n",
      "        [5.7000, 2.5000, 5.0000, 2.0000],\n",
      "        [5.8000, 2.8000, 5.1000, 2.4000],\n",
      "        [6.4000, 3.2000, 5.3000, 2.3000],\n",
      "        [6.5000, 3.0000, 5.5000, 1.8000],\n",
      "        [7.7000, 3.8000, 6.7000, 2.2000],\n",
      "        [7.7000, 2.6000, 6.9000, 2.3000],\n",
      "        [6.0000, 2.2000, 5.0000, 1.5000],\n",
      "        [6.9000, 3.2000, 5.7000, 2.3000],\n",
      "        [5.6000, 2.8000, 4.9000, 2.0000],\n",
      "        [7.7000, 2.8000, 6.7000, 2.0000],\n",
      "        [6.3000, 2.7000, 4.9000, 1.8000],\n",
      "        [6.7000, 3.3000, 5.7000, 2.1000],\n",
      "        [7.2000, 3.2000, 6.0000, 1.8000],\n",
      "        [6.2000, 2.8000, 4.8000, 1.8000],\n",
      "        [6.1000, 3.0000, 4.9000, 1.8000],\n",
      "        [6.4000, 2.8000, 5.6000, 2.1000],\n",
      "        [7.2000, 3.0000, 5.8000, 1.6000],\n",
      "        [7.4000, 2.8000, 6.1000, 1.9000],\n",
      "        [7.9000, 3.8000, 6.4000, 2.0000],\n",
      "        [6.4000, 2.8000, 5.6000, 2.2000],\n",
      "        [6.3000, 2.8000, 5.1000, 1.5000],\n",
      "        [6.1000, 2.6000, 5.6000, 1.4000],\n",
      "        [7.7000, 3.0000, 6.1000, 2.3000],\n",
      "        [6.3000, 3.4000, 5.6000, 2.4000],\n",
      "        [6.4000, 3.1000, 5.5000, 1.8000],\n",
      "        [6.0000, 3.0000, 4.8000, 1.8000],\n",
      "        [6.9000, 3.1000, 5.4000, 2.1000],\n",
      "        [6.7000, 3.1000, 5.6000, 2.4000],\n",
      "        [6.9000, 3.1000, 5.1000, 2.3000],\n",
      "        [5.8000, 2.7000, 5.1000, 1.9000],\n",
      "        [6.8000, 3.2000, 5.9000, 2.3000],\n",
      "        [6.7000, 3.3000, 5.7000, 2.5000],\n",
      "        [6.7000, 3.0000, 5.2000, 2.3000],\n",
      "        [6.3000, 2.5000, 5.0000, 1.9000],\n",
      "        [6.5000, 3.0000, 5.2000, 2.0000],\n",
      "        [6.2000, 3.4000, 5.4000, 2.3000],\n",
      "        [5.9000, 3.0000, 5.1000, 1.8000]])\n"
     ]
    }
   ],
   "source": [
    "print(X_tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "1aa6ff20",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[5.1000, 3.5000, 1.4000, 0.2000],\n",
      "        [4.9000, 3.0000, 1.4000, 0.2000],\n",
      "        [4.7000, 3.2000, 1.3000, 0.2000],\n",
      "        [4.6000, 3.1000, 1.5000, 0.2000],\n",
      "        [5.0000, 3.6000, 1.4000, 0.2000],\n",
      "        [5.4000, 3.9000, 1.7000, 0.4000],\n",
      "        [4.6000, 3.4000, 1.4000, 0.3000],\n",
      "        [5.0000, 3.4000, 1.5000, 0.2000],\n",
      "        [4.4000, 2.9000, 1.4000, 0.2000],\n",
      "        [4.9000, 3.1000, 1.5000, 0.1000],\n",
      "        [5.4000, 3.7000, 1.5000, 0.2000],\n",
      "        [4.8000, 3.4000, 1.6000, 0.2000],\n",
      "        [4.8000, 3.0000, 1.4000, 0.1000],\n",
      "        [4.3000, 3.0000, 1.1000, 0.1000],\n",
      "        [5.8000, 4.0000, 1.2000, 0.2000],\n",
      "        [5.7000, 4.4000, 1.5000, 0.4000],\n",
      "        [5.4000, 3.9000, 1.3000, 0.4000],\n",
      "        [5.1000, 3.5000, 1.4000, 0.3000],\n",
      "        [5.7000, 3.8000, 1.7000, 0.3000],\n",
      "        [5.1000, 3.8000, 1.5000, 0.3000],\n",
      "        [5.4000, 3.4000, 1.7000, 0.2000],\n",
      "        [5.1000, 3.7000, 1.5000, 0.4000],\n",
      "        [4.6000, 3.6000, 1.0000, 0.2000],\n",
      "        [5.1000, 3.3000, 1.7000, 0.5000],\n",
      "        [4.8000, 3.4000, 1.9000, 0.2000],\n",
      "        [5.0000, 3.0000, 1.6000, 0.2000],\n",
      "        [5.0000, 3.4000, 1.6000, 0.4000],\n",
      "        [5.2000, 3.5000, 1.5000, 0.2000],\n",
      "        [5.2000, 3.4000, 1.4000, 0.2000],\n",
      "        [4.7000, 3.2000, 1.6000, 0.2000],\n",
      "        [4.8000, 3.1000, 1.6000, 0.2000],\n",
      "        [5.4000, 3.4000, 1.5000, 0.4000],\n",
      "        [5.2000, 4.1000, 1.5000, 0.1000],\n",
      "        [5.5000, 4.2000, 1.4000, 0.2000],\n",
      "        [4.9000, 3.1000, 1.5000, 0.2000],\n",
      "        [5.0000, 3.2000, 1.2000, 0.2000],\n",
      "        [5.5000, 3.5000, 1.3000, 0.2000],\n",
      "        [4.9000, 3.6000, 1.4000, 0.1000],\n",
      "        [4.4000, 3.0000, 1.3000, 0.2000],\n",
      "        [5.1000, 3.4000, 1.5000, 0.2000],\n",
      "        [5.0000, 3.5000, 1.3000, 0.3000],\n",
      "        [4.5000, 2.3000, 1.3000, 0.3000],\n",
      "        [4.4000, 3.2000, 1.3000, 0.2000],\n",
      "        [5.0000, 3.5000, 1.6000, 0.6000],\n",
      "        [5.1000, 3.8000, 1.9000, 0.4000],\n",
      "        [4.8000, 3.0000, 1.4000, 0.3000],\n",
      "        [5.1000, 3.8000, 1.6000, 0.2000],\n",
      "        [4.6000, 3.2000, 1.4000, 0.2000],\n",
      "        [5.3000, 3.7000, 1.5000, 0.2000],\n",
      "        [5.0000, 3.3000, 1.4000, 0.2000],\n",
      "        [7.0000, 3.2000, 4.7000, 1.4000],\n",
      "        [6.4000, 3.2000, 4.5000, 1.5000],\n",
      "        [6.9000, 3.1000, 4.9000, 1.5000],\n",
      "        [5.5000, 2.3000, 4.0000, 1.3000],\n",
      "        [6.5000, 2.8000, 4.6000, 1.5000],\n",
      "        [5.7000, 2.8000, 4.5000, 1.3000],\n",
      "        [6.3000, 3.3000, 4.7000, 1.6000],\n",
      "        [4.9000, 2.4000, 3.3000, 1.0000],\n",
      "        [6.6000, 2.9000, 4.6000, 1.3000],\n",
      "        [5.2000, 2.7000, 3.9000, 1.4000],\n",
      "        [5.0000, 2.0000, 3.5000, 1.0000],\n",
      "        [5.9000, 3.0000, 4.2000, 1.5000],\n",
      "        [6.0000, 2.2000, 4.0000, 1.0000],\n",
      "        [6.1000, 2.9000, 4.7000, 1.4000],\n",
      "        [5.6000, 2.9000, 3.6000, 1.3000],\n",
      "        [6.7000, 3.1000, 4.4000, 1.4000],\n",
      "        [5.6000, 3.0000, 4.5000, 1.5000],\n",
      "        [5.8000, 2.7000, 4.1000, 1.0000],\n",
      "        [6.2000, 2.2000, 4.5000, 1.5000],\n",
      "        [5.6000, 2.5000, 3.9000, 1.1000],\n",
      "        [5.9000, 3.2000, 4.8000, 1.8000],\n",
      "        [6.1000, 2.8000, 4.0000, 1.3000],\n",
      "        [6.3000, 2.5000, 4.9000, 1.5000],\n",
      "        [6.1000, 2.8000, 4.7000, 1.2000],\n",
      "        [6.4000, 2.9000, 4.3000, 1.3000],\n",
      "        [6.6000, 3.0000, 4.4000, 1.4000],\n",
      "        [6.8000, 2.8000, 4.8000, 1.4000],\n",
      "        [6.7000, 3.0000, 5.0000, 1.7000],\n",
      "        [6.0000, 2.9000, 4.5000, 1.5000],\n",
      "        [5.7000, 2.6000, 3.5000, 1.0000],\n",
      "        [5.5000, 2.4000, 3.8000, 1.1000],\n",
      "        [5.5000, 2.4000, 3.7000, 1.0000],\n",
      "        [5.8000, 2.7000, 3.9000, 1.2000],\n",
      "        [6.0000, 2.7000, 5.1000, 1.6000],\n",
      "        [5.4000, 3.0000, 4.5000, 1.5000],\n",
      "        [6.0000, 3.4000, 4.5000, 1.6000],\n",
      "        [6.7000, 3.1000, 4.7000, 1.5000],\n",
      "        [6.3000, 2.3000, 4.4000, 1.3000],\n",
      "        [5.6000, 3.0000, 4.1000, 1.3000],\n",
      "        [5.5000, 2.5000, 4.0000, 1.3000],\n",
      "        [5.5000, 2.6000, 4.4000, 1.2000],\n",
      "        [6.1000, 3.0000, 4.6000, 1.4000],\n",
      "        [5.8000, 2.6000, 4.0000, 1.2000],\n",
      "        [5.0000, 2.3000, 3.3000, 1.0000],\n",
      "        [5.6000, 2.7000, 4.2000, 1.3000],\n",
      "        [5.7000, 3.0000, 4.2000, 1.2000],\n",
      "        [5.7000, 2.9000, 4.2000, 1.3000],\n",
      "        [6.2000, 2.9000, 4.3000, 1.3000],\n",
      "        [5.1000, 2.5000, 3.0000, 1.1000],\n",
      "        [5.7000, 2.8000, 4.1000, 1.3000],\n",
      "        [6.3000, 3.3000, 6.0000, 2.5000],\n",
      "        [5.8000, 2.7000, 5.1000, 1.9000],\n",
      "        [7.1000, 3.0000, 5.9000, 2.1000],\n",
      "        [6.3000, 2.9000, 5.6000, 1.8000],\n",
      "        [6.5000, 3.0000, 5.8000, 2.2000],\n",
      "        [7.6000, 3.0000, 6.6000, 2.1000],\n",
      "        [4.9000, 2.5000, 4.5000, 1.7000],\n",
      "        [7.3000, 2.9000, 6.3000, 1.8000],\n",
      "        [6.7000, 2.5000, 5.8000, 1.8000],\n",
      "        [7.2000, 3.6000, 6.1000, 2.5000],\n",
      "        [6.5000, 3.2000, 5.1000, 2.0000],\n",
      "        [6.4000, 2.7000, 5.3000, 1.9000],\n",
      "        [6.8000, 3.0000, 5.5000, 2.1000],\n",
      "        [5.7000, 2.5000, 5.0000, 2.0000],\n",
      "        [5.8000, 2.8000, 5.1000, 2.4000],\n",
      "        [6.4000, 3.2000, 5.3000, 2.3000],\n",
      "        [6.5000, 3.0000, 5.5000, 1.8000],\n",
      "        [7.7000, 3.8000, 6.7000, 2.2000],\n",
      "        [7.7000, 2.6000, 6.9000, 2.3000],\n",
      "        [6.0000, 2.2000, 5.0000, 1.5000],\n",
      "        [6.9000, 3.2000, 5.7000, 2.3000],\n",
      "        [5.6000, 2.8000, 4.9000, 2.0000],\n",
      "        [7.7000, 2.8000, 6.7000, 2.0000],\n",
      "        [6.3000, 2.7000, 4.9000, 1.8000],\n",
      "        [6.7000, 3.3000, 5.7000, 2.1000],\n",
      "        [7.2000, 3.2000, 6.0000, 1.8000],\n",
      "        [6.2000, 2.8000, 4.8000, 1.8000],\n",
      "        [6.1000, 3.0000, 4.9000, 1.8000],\n",
      "        [6.4000, 2.8000, 5.6000, 2.1000],\n",
      "        [7.2000, 3.0000, 5.8000, 1.6000],\n",
      "        [7.4000, 2.8000, 6.1000, 1.9000],\n",
      "        [7.9000, 3.8000, 6.4000, 2.0000],\n",
      "        [6.4000, 2.8000, 5.6000, 2.2000],\n",
      "        [6.3000, 2.8000, 5.1000, 1.5000],\n",
      "        [6.1000, 2.6000, 5.6000, 1.4000],\n",
      "        [7.7000, 3.0000, 6.1000, 2.3000],\n",
      "        [6.3000, 3.4000, 5.6000, 2.4000],\n",
      "        [6.4000, 3.1000, 5.5000, 1.8000],\n",
      "        [6.0000, 3.0000, 4.8000, 1.8000],\n",
      "        [6.9000, 3.1000, 5.4000, 2.1000],\n",
      "        [6.7000, 3.1000, 5.6000, 2.4000],\n",
      "        [6.9000, 3.1000, 5.1000, 2.3000],\n",
      "        [5.8000, 2.7000, 5.1000, 1.9000],\n",
      "        [6.8000, 3.2000, 5.9000, 2.3000],\n",
      "        [6.7000, 3.3000, 5.7000, 2.5000],\n",
      "        [6.7000, 3.0000, 5.2000, 2.3000],\n",
      "        [6.3000, 2.5000, 5.0000, 1.9000],\n",
      "        [6.5000, 3.0000, 5.2000, 2.0000],\n",
      "        [6.2000, 3.4000, 5.4000, 2.3000],\n",
      "        [5.9000, 3.0000, 5.1000, 1.8000]], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "print(X_tensor_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "5568c323",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now we can do the same things as before\n",
    "X_tensor_agg = torch.mean(X_tensor, dim=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "8f76834f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([5.8433, 3.0573, 3.7580, 1.1993])\n"
     ]
    }
   ],
   "source": [
    "print(X_tensor_agg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79e1f809",
   "metadata": {},
   "source": [
    "Ein kleiner Unterschied ist, dass nun die Dimension wirklich mit ``dim`` abgekürzt wird und nicht mit ``axis``."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df2f9432",
   "metadata": {},
   "source": [
    "Wie kann man nun einen Tensor erstellen (ohne Daten)?\n",
    "\n",
    "-> (Fast) genauso wie in numpy!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "3eea711e",
   "metadata": {},
   "outputs": [],
   "source": [
    "zeros_tensor = torch.zeros((3,5))\n",
    "ones_tensor = torch.ones((2,5))\n",
    "my_list = [1,2,3.0, 1.123345, -3]\n",
    "my_tensor = torch.tensor(my_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "4f67937b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0., 0.]])\n",
      "tensor([[1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.]])\n",
      "tensor([ 1.0000,  2.0000,  3.0000,  1.1233, -3.0000])\n"
     ]
    }
   ],
   "source": [
    "print(zeros_tensor)\n",
    "print(ones_tensor)\n",
    "print(my_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6aa6dfb3",
   "metadata": {},
   "source": [
    "Und natürlich geht das auch mit mehreren Dimensionen (nicht nur 1 oder 2)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "73817492",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[[[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]],\n",
      "\n",
      "\n",
      "\n",
      "          [[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]]],\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "         [[[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]],\n",
      "\n",
      "\n",
      "\n",
      "          [[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]]],\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "         [[[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]],\n",
      "\n",
      "\n",
      "\n",
      "          [[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]]]],\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "        [[[[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]],\n",
      "\n",
      "\n",
      "\n",
      "          [[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]]],\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "         [[[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]],\n",
      "\n",
      "\n",
      "\n",
      "          [[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]]],\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "         [[[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]],\n",
      "\n",
      "\n",
      "\n",
      "          [[[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]],\n",
      "\n",
      "\n",
      "           [[[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]],\n",
      "\n",
      "            [[0., 0.],\n",
      "             [0., 0.],\n",
      "             [0., 0.]]]]]]])\n"
     ]
    }
   ],
   "source": [
    "# Create 7d Tensor:\n",
    "seven_d_tensor = torch.zeros((2, 3, 2, 2, 2, 3, 2))  # Example of a 7-dimensional tensor\n",
    "print(seven_d_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a2f705e",
   "metadata": {},
   "source": [
    "Bei solchen Dimensionen macht es dann natürlich wenig Sinn, sich die Daten direkt anzusehen, aber man kann sich natürlich die shape ansehen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "960129ee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 3, 2, 2, 2, 3, 2])\n"
     ]
    }
   ],
   "source": [
    "# Shape of 7d tensor\n",
    "print(seven_d_tensor.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14a0250d",
   "metadata": {},
   "source": [
    "> **Übung:** Welche Shape hat ein RGB FullHD Film, welcher in 24 FPS abgespielt wird und eine Länge von 90min hat? Wie viele Einträge hat der Tensor? Welche Reihenfolge ist sinnvoll?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef7077ea",
   "metadata": {},
   "source": [
    "> **Übung:** Was wäre, wenn wir nun 12 so Videos gleichzeitig in einem Tensor speichern möchten? Können die Videos eine unterschiedliche Länge haben?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8faa5e00",
   "metadata": {},
   "source": [
    "Ok, aber warum nutzen wir jetzt genau PyTorch und nicht einfach numpy, wenn in beiden Bibliotheken Matrizen erstellt werden können?\n",
    "\n",
    "Grund dafür sind (mitunter) folgende 3 Dinge:\n",
    "* Die wichtigsten Machine Learning Funktionen sind bereits implementiert (werden wir noch lernen)\n",
    "* Automatische Differenzierung, welche das Trainieren von neuronalen Netzwerken ermöglicht\n",
    "* Die sehr einfache Möglichkeit, die Berechnungen auf einem Hardware-beschleunigten Gerät (zBsp. die GPU) auszuführen"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dynamic Computation Graph und Automatic Differentiation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c23fb95",
   "metadata": {},
   "source": [
    "Es ist also möglich in PyTorch, sich komplizierte Funktionen zu basteln (zum Beispiel ein Neuronales Netz) und PyTorch kann automatisch die Ableitungen bzgl. den verschiedenen Parametern berechnen.\n",
    "\n",
    "Warum das Berechnen der Ableitungen wichtig ist, werden wir in den nächsten Notebooks noch genauer erfahren (Gradient Descent). Kurz gesagt, gibt uns die Ableitung der Fehlerfunktion eine Richtung vor, in die wir uns bewegen müssen."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "907a55f2",
   "metadata": {},
   "source": [
    "Sehen wir uns einmal an, wie wir auf die Gradienten zugreifen können."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "5523cd69",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1., 1., 1.])\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "x = torch.ones((3))\n",
    "print(x)\n",
    "print(x.requires_grad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "6ef67d7b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "# We can also equip the tensor with the ability to calculate gradients\n",
    "x.requires_grad_(True)\n",
    "print(x.requires_grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b131de4",
   "metadata": {},
   "source": [
    "Um nun eine Ableitung einer Funktion zu berechnen, definieren wir uns einmal eine beliebige Funktion, in diesem Fall:\n",
    "$$y(x) = \\frac{1}{n}\\sum_{i=1}^{n} \\left[(x_i + 2)^2 + 3\\right],$$\n",
    "\n",
    "wobei hier $n$ die Anzahl der Elemente von $x$ darstellt."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0528fd8d",
   "metadata": {},
   "source": [
    "In unserem Fall ist nun $x$ der Parameter, den wir optimieren (also verändern) wollen, sodass $y(x)$ maximal (oder minimal) wird, sprich ein Extremum erreicht.\n",
    "\n",
    "**Wichtig:** Später wird $y(x)$ die *Loss*-Funktion sein, und diese wollen wir natürlich minimieren. Somit kann man sich auch jetzt schon vorstellen, dass $y(x)$ eine Loss-Funktion ist."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d842a4c0",
   "metadata": {},
   "source": [
    "Wir wissen, dass bei einem Extremum die Ableitung $0$ ist, somit müssen wir die Ableitung von $y(x)$ bezüglich $x$ finden, sprich $\\frac{\\mathrm d\\, y(x)}{\\mathrm d\\, x}$.\n",
    "\n",
    "Dabei kann uns jetzt PyTorch helfen."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "100b98d0",
   "metadata": {},
   "source": [
    "Die Art, wie PyTorch die Ableitung berechnet ist mit sogenannten *Computational Graphs*. Um zu verstehen, was das ist, zerlegen wir nun unsere Funktion in kleinere Bestandteile."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2e487a2",
   "metadata": {},
   "source": [
    "Als Input verwenden wir den Vektor $x=[1,2,3]$ (zum Beispiel eine Person mit $1$ Geschwister, $2$ Urlauben pro Jahr und $3$ Mahlzeiten pro Tag)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "6f8e2b65",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0., 1., 2.], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "x = torch.arange(3, dtype=torch.float32, requires_grad=True)  # Create a tensor with requires_grad set to True (we want to calculate the gradient later)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "161d1ab8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y = 12.666666984558105\n",
      "Y tensor(12.6667, grad_fn=<MeanBackward0>)\n"
     ]
    }
   ],
   "source": [
    "a = x + 2\n",
    "b = a ** 2\n",
    "c = b + 3\n",
    "y = c.mean()\n",
    "print(f'y = {y}')\n",
    "print(\"Y\", y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4351e02",
   "metadata": {},
   "source": [
    "Hier ist der *Computational Graph* nun auch grafisch dargestellt."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e6d53d9",
   "metadata": {},
   "source": [
    "![Computational_Graph](../resources/pytorch_computation_graph.svg)\n",
    "\n",
    "(von https://github.com/phlippe/uvadlc_notebooks/blob/master/docs/tutorial_notebooks/tutorial2/Introduction_to_PyTorch.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fcb5ece",
   "metadata": {},
   "source": [
    "Nun berechnen wir die Ableitung $\\frac{\\mathrm d\\, y(x)}{\\mathrm d\\, x}$. Diese setzt sich (aufgrund der Kettenregel) aus den folgenden Ableitungen zusammen:\n",
    "\n",
    "$$\\frac{\\mathrm d\\, y(x)}{\\mathrm d\\, x_i} = \\frac{\\mathrm d\\, y}{\\mathrm d\\, c_i}\\cdot\\frac{\\mathrm d\\, c_i}{\\mathrm d\\, b_i}\\cdot \\frac{\\mathrm d\\, b_i}{\\mathrm d\\, a_i}\\cdot \\frac{\\mathrm d\\, a_i}{\\mathrm d\\, x_i}.$$\n",
    "\n",
    "**Hinweis:** Nachdem hier $x$ ein Vektor ist, müssen wir die Ableitung für jede Komponente einzeln berechnen. Dementsprechend wurde oben der Index $i$ verwendet."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aa67ccc",
   "metadata": {},
   "source": [
    "Wie können wir die Ableitung händisch berechnen?\n",
    "\n",
    "$$\n",
    "\\frac{\\mathrm d\\, a_i}{\\mathrm d\\, x_i} = 1,\\hspace{1cm}\n",
    "\\frac{\\mathrm d\\, b_i}{\\mathrm d\\, a_i} = 2\\cdot a_i,\\hspace{1cm}\n",
    "\\frac{\\mathrm d\\, c_i}{\\mathrm d\\, b_i} = 1,\\hspace{1cm}\n",
    "\\frac{\\mathrm d\\, y}{\\mathrm d\\, c_i} = \\frac{1}{3}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52884584",
   "metadata": {},
   "source": [
    "Somit ergibt sich für $x=[0,1,2]$ und $a_i=x_i+2$ die Ableitung\n",
    "\n",
    "$$\\frac{\\mathrm d\\, y(x)}{\\mathrm d\\, x} =\\left[\\frac43,2,\\frac83\\right].$$\n",
    "\n",
    "Wie kann das nun PyTorch machen?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "7c98a812",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For calculating the gradient, we need to call the backward() method on the output tensor\n",
    "y.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "29a22f7d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gradient of y with respect to x: tensor([1.3333, 2.0000, 2.6667])\n"
     ]
    }
   ],
   "source": [
    "# Then we get the gradient of y with respect to x by invoking x.grad\n",
    "print(\"Gradient of y with respect to x:\", x.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc2bffca",
   "metadata": {},
   "source": [
    "**Wichtig:** Später werden wir die Ableitungen nicht bezüglich $x$ berechnen (wir können ja unsere Daten nicht ändern), sondern werden die Parameter bezüglich der Parameter von unserem Netzwerk berechnen. Diese können wir dann jeden Schritt dementsprechend anpassen."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe0653d6",
   "metadata": {},
   "source": [
    "**Hinweis:** Das heißt, bei einer Loss-Funktion sind die Daten fix und wir haben die Parameter der Funktion/des neuronalen Netzwerks als Parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a9ea7ec",
   "metadata": {},
   "source": [
    "**Wichtig:** Nachdem hier unser Input ein Vektor ist, sprechen wir eigentlich nicht von einer Ableitung, sondern von einem **Gradient**. Ein Gradient ist ein Vektor, bei dem jede Komponente, die Ableitung bzgl. der Komponente ist."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71758179",
   "metadata": {},
   "source": [
    "#### GPU Support"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60077273",
   "metadata": {},
   "source": [
    "Wie bereits oben angekündigt, wollen wir hier nun die GPU (leider nur NVIDIA (und Apple Silicon) möglich) benutzen, sofern eine zur Verfügung steht. Mit ihr können wir die Berechnungen dann mit *Cuda* durchführen. \n",
    "\n",
    "Ob eine GPU zur Verfügung steht, können wir mit folgenden Command testen.\n",
    "\n",
    "**Hinweis:** Es kann nur dann die NVIDIA GPU verwendet werden, wenn auch die entsprechende PyTorch Version installiert ist."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4fa29bd",
   "metadata": {},
   "source": [
    "**Hinweis:** Für alle ohne NVIDIA GPU gibt es auch einige Online Tools bei denen man gratis hardwarebeschleunigt Notebooks ausführen kann (siehe Ende vom Notebook)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "e774adec",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPU is available: False\n"
     ]
    }
   ],
   "source": [
    "gpu_avail = torch.cuda.is_available()\n",
    "print(f'GPU is available: {gpu_avail}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a15900a",
   "metadata": {},
   "source": [
    "Im Allgemeinen ist es gängig, folgenden command am Beginn jedes Notebooks zu verwenden."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "d7ddbb44",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cpu\n"
     ]
    }
   ],
   "source": [
    "device = torch.device('cuda') if torch.cuda.is_available() else torch.device('mps') if torch.backends.mps.is_available() else torch.device('cpu')\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b1c4349",
   "metadata": {},
   "source": [
    "Um nun auch wirklich die Berechnungen auf der GPU durchzuführen, müssen wir unsere Berechnungen auf dieses Gerät schieben. Dies geht mit dem `to(device)` command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "3b6a40ac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1.],\n",
      "        [1., 1.],\n",
      "        [1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "x = torch.ones((3,2))\n",
    "x = x.to(device)  # Move the tensor to the GPU if available\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "f86a010e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1.],\n",
      "        [1., 1.],\n",
      "        [1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "# We can also directly create a tensor on the GPU\n",
    "x_gpu = torch.ones((3, 2), device=device)  # Create a tensor directly on the GPU\n",
    "print(x_gpu)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3485380b",
   "metadata": {},
   "source": [
    "Warum sollten wir Code überhaupt auf der GPU ausführen? Weil es viel **schneller** ist. (Faktor > 100 ohne Probleme möglich)\n",
    "\n",
    "Vergleichen wir nun die Geschwindigkeit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "586f80ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "if torch.cuda.is_available():\n",
    "    size = 5000\n",
    "\n",
    "    x = torch.randn(size, size)\n",
    "\n",
    "    ## CPU version\n",
    "    start_time = time.time()\n",
    "    _ = torch.matmul(x, x)\n",
    "    end_time = time.time()\n",
    "    print(f\"CPU time: {(end_time - start_time):6.5f}s\")\n",
    "\n",
    "    ## GPU version\n",
    "    x = x.to(device)\n",
    "    _ = torch.matmul(x, x)  # First operation to 'burn in' GPU\n",
    "    # CUDA is asynchronous, so we need to use different timing functions\n",
    "    start = torch.cuda.Event(enable_timing=True)\n",
    "    end = torch.cuda.Event(enable_timing=True)\n",
    "    start.record()\n",
    "    _ = torch.matmul(x, x)\n",
    "    end.record()\n",
    "    torch.cuda.synchronize()  # Waits for everything to finish running on the GPU\n",
    "    print(f\"GPU time: {0.001 * start.elapsed_time(end):6.5f}s\")  # Milliseconds to seconds"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6e26cd5",
   "metadata": {},
   "source": [
    "Wir sehen also, mit *cuda* ist wirklich alles **viel** schneller."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c886862b",
   "metadata": {},
   "source": [
    "**Hinweis:** Mit *cuda*, also auf der GPU müssen wir nun aber auch beachten, dass nun die Daten den GPU Speicher befüllen (GPU Memory). Dieser ist meistens geringer als der RAM, somit müssen die Modelle ggf. kleiner gemacht werden, bzw. das Dataset muss aufgeteilt werden (Details dazu gibt in einem anderen Notebook (Datasets und Dataloader))."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bd0aed8",
   "metadata": {},
   "source": [
    "![Tensors_Everywhere](../resources/Tensors_Everywhere.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77e318c9",
   "metadata": {},
   "source": [
    "# Bonus: Google Colab"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "995c383d",
   "metadata": {},
   "source": [
    "Was, wenn wir keine Grafikkarte haben und trotzdem ein Model hardwarebeschleunigt trainieren wollen?\n",
    "\n",
    "Es gibt zahlreiche Online Anbieter, wo man gratis Notebooks laufen lassen kann inkl. GPU. Zum Beispiel [Google Colab](https://colab.research.google.com/).\n",
    "\n",
    "Wir laden nun unser Notebook hier hoch und testen Colab ein wenig."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f753b93",
   "metadata": {},
   "source": [
    "**Vorteile**:\n",
    "* Gratis\n",
    "* Gute GPU's\n",
    "* Ermöglicht Verwendung auch mit wenig-leistungsfähigem Laptop"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44553735",
   "metadata": {},
   "source": [
    "**Nachteile:**\n",
    "* Runtime nur für begrenzte (inaktive) Zeit\n",
    "* Hochladen von Daten nervig"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d93c9eee",
   "metadata": {},
   "source": [
    "## Aufgabe:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e72bbbc",
   "metadata": {},
   "source": [
    "Programme ein mehrdimensionales Linear Regression Modell für 4 Input Features $x_1, x_2, x_3, x_4$ und berechne den Gradient bzgl. der Parameter $w_1, w_2, w_3, w_4, x_1, x_2, x_3, x_4, b$.\n",
    "\n",
    "Bearbeite dafür die folgenden Schritte:\n",
    "* Eingabe: Vektor $x$ der mit 4 Features (Tensor mit Shape (4))\n",
    "* Gewichtsmatrix $W$ (Größe $1\\times 4$)\n",
    "* Offset $b$ (Größe $1$)\n",
    "\n",
    "Berechne:\n",
    "* $y = Wx+b = w_1\\cdot x_1 + w_2 \\cdot x_2 + w_3 \\cdot x_3 + w_4 \\cdot x_4 + b$\n",
    "* a = $\\max(y, 0)$\n",
    "* $L=f(a)$ mit $f(a) = a^2$ (*Dummy*-Loss Funktion)\n",
    "* Ableitungen bzgl. $W$, $x$ und $b$.\n",
    "\n",
    "\n",
    "Die Gewichtsvektoren und Input kann zufällig gewählt werden."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84103a14",
   "metadata": {},
   "source": [
    "**Bonus:** Verwende dazu das Attribut **device**, sodass immer die GPU verwendet wird, falls eine verfügbar ist."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d8d80f6",
   "metadata": {},
   "source": [
    "**Bonus:** Lade nochmal das Iris Dataset und berechne den Output hier? Was muss alles geändert werden?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5233833a",
   "metadata": {},
   "source": [
    "### Lösung:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b471f56d",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "055c55dc",
   "metadata": {},
   "source": [
    "Die vorige Aufgabenstellung war der Türöffner für \"echte\" neuronale Netze, mit welchen wir uns von nun an befassen wollen."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dsai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.23"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}