1

I have standard deviation for each 15 min of that hour so can i calculate the standard deviation of that hour using this 4 standard deviation values ?

Reason

Standard deviation of each 15 min is calculated by storing the per second reading coming from sensors and after the completion of the 15 min i calculate the SD of that 15 min slot and to calculate the whole hour SD i can not hold that per second reading that's why i need a way to calculate the SD using the each 15 min SD and same to calculate the SD of Day using the each hour SD

Edit last hour each 15 min SD

  1. 0.873
  2. 0.864
  3. 0.864
  4. 0.863
Hemant Sangle
  • 11
  • 5

1 Answers1

0

This data is not enough to compute the standard deviation of the hour. Consider two examples:

First example: your sensor shows constant zero. The standard deviation of each 15 minutes is zero, and the standard deviation of the hour is also zero.

Second example:

from 00:00 to 00:15 your sensor shows 0, from 00:15 to 00:30 it shows 1, from 00:30 to 00:45 it shows 2, and from 00:45 to 01:00 it shows 3. The standard deviation of each 15 minute interval is still zero, but the standard deviation of the hour is not.

Here is how you can compute the standard deviation of a sample using only a few memory slots.

The formula for sample standard deviation s for a sample of size n is: $$ s = \sqrt{\frac{n}{n-1}Var(X)} = \sqrt{\frac{n}{n-1}(\mathbb E(X^2) - (\mathbb EX)^2)} $$

So, all you need to store to compute the standard deviation of a sample is the sum of readings and their sum of squares. (The sample size for a per second reading is known in advance).

seed
  • 101
  • 2
  • but the thing is i can not know the n value in advance thats why i am calculating running Standard deviation using this link [https://dev.to/nestedsoftware/calculating-standard-deviation-on-streaming-data-253l] – Hemant Sangle Nov 21 '18 at 09:09
  • okay i got it so you are saying that i should hold the sum of reading ,sum of reading squares and count of that hour reading and put those in this formula and it will give me the SD of that hour am i right – Hemant Sangle Nov 21 '18 at 09:17
  • If the readings happen once per second, then during an hour there will be 60 * 60 = 360 readings. Or you can count them just to be sure, yes. – seed Nov 24 '18 at 16:17
  • Thanks and yes i have already maintain the count of readings, I will accept your answer once i verify the output – Hemant Sangle Nov 27 '18 at 03:54
  • If you in addition knows the means for the same time periods you know the sd, see https://stats.stackexchange.com/questions/332951/online-algorithm-for-the-mean-square-error or https://stats.stackexchange.com/questions/340576/online-algorithm-to-compute-rolling-standard-error-for-intercept-in-multiple-reg or https://stats.stackexchange.com/questions/326464/calculating-mean-and-sd-from-a-stream-of-numbers – kjetil b halvorsen Mar 17 '20 at 18:39