Sparse tensors for monitors explanation

Author: n-shevko

docs/source/guide/guide_part_i.rst

@@ -278,6 +278,47 @@ Similarly, one can get the contents of a network monitor by calling :code:`netwo
 function takes no arguments; it returns a dictionary mapping network components to a sub-dictionary mapping state
 variables to their tensor-valued recording.
 
+
+:py:class:`bindsnet.network.monitors.AbstractMonitor` objects can also store sparse tensor-valued variables.
+For example, spikes can be stored efficiently using a sparse monitor:
+
+.. code-block:: python
+
+    Monitor(
+        network.layers[layer], state_vars=["s"], time=int(time / dt), device=device, sparse=True
+    )
+
+Note that using sparse tensors is advantageous only when the percentage of non-zero values is less than 4% of the total values.
+The table below compares memory consumption between sparse and dense tensors:
+
+======================= ====================== ====================== ====================
+Sparse (megabytes used) Dense (megabytes used) Ratio (Sparse/Dense) % % of non zero values
+======================= ====================== ====================== ====================
+15                      119                    13                     0.5
+30                      119                    25                     1.0
+45                      119                    38                     1.5
+60                      119                    50                     2.0
+75                      119                    63                     2.5
+89                      119                    75                     3.0
+104                     119                    87                     3.5
+119                     119                    100                    4.0
+134                     119                    113                    4.5
+149                     119                    125                    5.0
+164                     119                    138                    5.5
+179                     119                    150                    6.0
+194                     119                    163                    6.5
+209                     119                    176                    7.0
+224                     119                    188                    7.5
+239                     119                    201                    8.0
+253                     119                    213                    8.5
+268                     119                    225                    9.0
+283                     119                    238                    9.5
+======================= ====================== ====================== ====================
+
+The tensor size does not affect the values in the third column.
+This table was generated by :code:`examples/benchmark/sparse_vs_dense_tensors.py`
+
+
 Running Simulations
 -------------------
 

examples/benchmark/sparse_vs_dense_tensors.py

@@ -0,0 +1,38 @@
+import torch
+
+
+assert torch.cuda.is_available(), 'Benchmark works only on cuda'
+device = torch.device("cuda")
+
+
+def create_spikes_tensor(percent_of_true_values, sparse):
+    spikes_tensor = torch.bernoulli(
+        torch.full((500, 500, 500), percent_of_true_values, device=device)
+    ).bool()
+    if sparse:
+        spikes_tensor = spikes_tensor.to_sparse()
+
+    torch.cuda.reset_peak_memory_stats(device=device)
+    return round(torch.cuda.max_memory_allocated(device=device) / (1024 ** 2))
+
+
+print('======================= ====================== ====================== ====================')
+print('Sparse (megabytes used) Dense (megabytes used) Ratio (Sparse/Dense) % % of non zero values')
+print('======================= ====================== ====================== ====================')
+percent_of_true_values = 0.005
+while percent_of_true_values < 0.1:
+    result = {}
+    for sparse in [True, False]:
+        result[sparse] = create_spikes_tensor(percent_of_true_values, sparse)
+    percent = round((result[True] / result[False]) * 100)
+
+    row = [
+        str(result[True]).ljust(23),
+        str(result[False]).ljust(22),
+        str(percent).ljust(22),
+        str(round(percent_of_true_values * 100, 1)).ljust(20),
+    ]
+    print(' '.join(row))
+    percent_of_true_values += 0.005
+
+print('======================= ====================== ====================== ====================')