{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Differentiable barcode vectorization\n",
    "\n",
    "This tutorial gives you a brief insight in the functionalities offered by the `torchph.nn.SLayerExponential` \n",
    "module. It assumes familarity with standard `PyTorch` functionality. \n",
    "\n",
    "Also, `torchph.nn.SLayerExponential` is just one *structure element* and others are available as well (see documentation)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from shared_code import check_torchph_availability\n",
    "check_torchph_availability()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchph.nn import SLayerExponential\n",
    "\n",
    "# create an instance with 3 structure elements over \\R^2\n",
    "sl = SLayerExponential(3, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`nn.SLayerExponential` is a `torch.nn.Module` ... "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "isinstance(sl, torch.nn.Module)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... now we can do all the beautiful stuff which is inherited from `torch.nn.Module`, e.g.,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter containing:\n",
      "tensor([[0.6355, 0.3604],\n",
      "        [0.3162, 0.9167],\n",
      "        [0.4922, 0.9822]], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([[3., 3.],\n",
      "        [3., 3.],\n",
      "        [3., 3.]], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "for p in sl.parameters():\n",
    "    print(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The module has **two** parameters: \n",
    "1. `centers` : controls the centers of the structure elements. \n",
    "2. `sharpness`: controls how tight the used Gaussians are. The higher the value, the tighter. \n",
    "\n",
    "Both can be initialized using the `centers_init` and `sharpness_init` keyword arguments, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter containing:\n",
      "tensor([[0.0000, 0.0000],\n",
      "        [0.5000, 0.5000],\n",
      "        [1.0000, 1.0000]], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([[1., 1.],\n",
      "        [2., 2.],\n",
      "        [3., 3.]], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "# here is an initialization example\n",
    "centers_init = torch.Tensor(\n",
    "    [\n",
    "        [0,0], \n",
    "        [0.5, 0.5], \n",
    "        [1,1]\n",
    "    ]\n",
    ")\n",
    "\n",
    "sharpness_init = torch.Tensor(\n",
    "    [\n",
    "        [1,1], \n",
    "        [2,2], \n",
    "        [3,3]\n",
    "    ]\n",
    ")\n",
    "\n",
    "sl = SLayerExponential(3, 2, \n",
    "            centers_init=centers_init, \n",
    "            sharpness_init=sharpness_init)\n",
    "\n",
    "print(sl.centers)\n",
    "print(sl.sharpness)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simplest input form for `nn.SLayerExponential` is a `list` of `torch.Tensor` objects which are treated as a *batch*. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 3])\n"
     ]
    }
   ],
   "source": [
    "# As an example, we create a batch of multisets\n",
    "mset_1 = [[0, 0]]\n",
    "mset_2 = [[0, 0], [0, 0]]\n",
    "mset_3 = [[1, 1], [0, 0]]\n",
    "mset_4 = [[0, 0], [1, 1]]\n",
    "batch = [mset_1, mset_2, mset_3,  mset_4]\n",
    "batch = [torch.Tensor(x) for x in batch]\n",
    "output = sl(batch)\n",
    "print(output.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we see the output dimensionality is `(4, 3)` since\n",
    "we have a batch of size `4` and `3` structure elements. \n",
    "\n",
    "In other words, \n",
    "`output[i, j] =` \"evaluation of structure element j on mset_i\"\n",
    "\n",
    "Lets take a look ... "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1.0000e+00, 1.3534e-01, 1.5230e-08],\n",
      "        [2.0000e+00, 2.7067e-01, 3.0460e-08],\n",
      "        [1.1353e+00, 2.7067e-01, 1.0000e+00],\n",
      "        [1.1353e+00, 2.7067e-01, 1.0000e+00]], grad_fn=<SqueezeBackward0>)\n"
     ]
    }
   ],
   "source": [
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**We observe the following:**\n",
    "\n",
    "1. The j-th stucture element approximates the multiplicity function of the given input at point `sl.centers[j]`. E.g., the output of mset_1, `output[0, :]`,  is approx. `(1, 0, 0)`. \n",
    "2. `sl.sharpness[j]` controls the amount of contribution of points not exactly on `sl.centers[j]` with respect to their distance to `sl.centers[j]`. \n",
    "3. The input is interpreted as set, i.e., it is permutation invariant, as mset_3 and mset_4 do not defer as multiset and also `output[2,:] == output[3, :]`. \n",
    "\n",
    "Maybe this becomes more clear if we increase the sharpness of our structure elements a \"little\" ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 0., 0.],\n",
      "        [2., 0., 0.],\n",
      "        [1., 0., 1.],\n",
      "        [1., 0., 1.]], grad_fn=<SqueezeBackward0>)\n"
     ]
    }
   ],
   "source": [
    "sl = SLayerExponential(3, 2, \n",
    "            centers_init=centers_init, \n",
    "            sharpness_init=10*sharpness_init)\n",
    "print(sl(batch))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a small toy model to illustrate the applicatation of `SLayerExponential`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ToyModel(torch.nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__() \n",
    "        self.slayer = SLayerExponential(50, 2)\n",
    "        self.linear = torch.nn.Linear(50, 10)\n",
    "        \n",
    "    def forward(self, inp):\n",
    "        x = self.slayer(inp)\n",
    "        x = self.linear(x)\n",
    "        return x "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([3, 10])\n"
     ]
    }
   ],
   "source": [
    "model = ToyModel()\n",
    "inp = [torch.rand(10,2), torch.rand(20,2), torch.rand(30,2)]\n",
    "out = model(inp)\n",
    "print(out.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More information about alternative structure elements, i.e., \n",
    "\n",
    "- `torchph.nn.SLayerRational` \n",
    "- `torchph.nn.SLayerRationalHat`\n",
    "\n",
    "see documentation."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}