File: data_retention.rst

package info (click to toggle)
sbws 2.2.0-1
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid
  • size: 16,084 kB
  • sloc: python: 10,432; sh: 146; makefile: 38
file content (88 lines) | stat: -rw-r--r-- 2,577 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
.. _data_retention:

Data retention
==============

We often need to know for how long the measurements are written or read, to
know whether some calculations are correct.
It's usually confusing cause we have at least two defaults, 5 and 28 days.

The 28 days default is used:

* to keep measurements for this data interval in the file system
* to read measurements from this data interval to generate the BandwidthFile

  This means that all average values, like ``bw_mean`` are calculated from
  measurements from the previous 28 days.

The value comes from :const:`~sbws.globals`::

    GENERATE_PERIOD = 28 * 24 * 60 * 60

Used in :term:`generator` :func:`~sbws.core.generate.main`::

    elif scaling_method == TORFLOW_SCALING:
        fresh_days = ceil(GENERATE_PERIOD / 24 / 60 / 60)  # 28

    results = load_recent_results_in_datadir(
        fresh_days,  # 28

The 5 days default is used:

* to read the measurements from this data intervel during scanning
* to keep in memory the measurements for this data interval during scanning

The value comes from ``config.default.ini``::

  data_period = 5

Used in :term:`scanner` :func:`~sbws.core.scanner.run_speedtest`::

    measurements_period = conf.getint("general", "data_period")

:class:`~sbws.lib.resultdump.ResultDump` ``.__init__``::

    self.fresh_days = conf.getint("general", "data_period")

:meth:`~sbws.lib.resultdump.ResultDump.store_result`::

    self.data = trim_results(self.fresh_days, self.data)

:meth:`~sbws.lib.resultdump.ResultDump.enter`::

    self.data = load_recent_results_in_datadir(
        self.fresh_days, self.datadir
    )

It's also :const:`~sbws.globals`::

    MEASUREMENTS_PERIOD = 5 * 24 * 60 * 60
    PERIOD_DAYS = int(MEASUREMENTS_PERIOD / (24 * 60 * 60))  # 5

Used in :class:`~sbws.lib.relaylist.RelayList` ``.__init__``::

    measurements_period=MEASUREMENTS_PERIOD
    self._measurements_period = measurements_period

:meth:`~sbws.lib.relaylist.RelayList._init_relays`::

      days = self._measurements_period

These defaults are overwritten when calling the class from ``scanner.py``::

    measurements_period = conf.getint("general", "data_period")  # 5
    rl = RelayList(args, conf, controller, measurements_period, state)


These functions are call with either 28 or 5 days:

:func:`~sbws.lib.resultdump.trim_results`::

    data_period = fresh_days * 24 * 60 * 60

:func:`~sbws.lib.resultdump.load_recent_results_in_datadir`::

    data_period = fresh_days + 2
    oldest_day = today - timedelta(days=data_period)

    results = trim_results(fresh_days, results)