Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [real-time]

55 questions
104
votes
13 answers

Simple algorithm for online outlier detection of a generic time series

I am working with a large amount of time series. These time series are basically network measurements coming every 10 minutes, and some of them are periodic (i.e. the bandwidth), while some other aren't (i.e. the amount of routing traffic). I would…
gianluca
  • 1,921
  • 4
  • 16
  • 9
64
votes
6 answers

Efficient online linear regression

I'm analysing some data where I would like to perform ordinary linear regression, however this is not possible as I am dealing with an on-line setting with a continuous stream of input data (which will quickly get too large for memory) and need to…
mikera
  • 975
  • 1
  • 8
  • 12
61
votes
7 answers

Period detection of a generic time series

This post is the continuation of another post related to a generic method for outlier detection in time series. Basically, at this point I'm interested in a robust way to discover the periodicity/seasonality of a generic time series affected by a…
gianluca
  • 1,921
  • 4
  • 16
  • 9
38
votes
4 answers

Simple way to algorithmically identify a spike in recorded errors

We need an early warning system. I am dealing with a server that is known to have performance issues under load. Errors are recorded in a database along with a timestamp. There are some manual intervention steps that can be taken to decrease the…
dbenton
  • 383
  • 1
  • 4
  • 5
13
votes
1 answer

Algorithm for real-time normalization of time-series data?

I'm working on an algorithm that takes in a vector of the most recent data point from a number of sensor streams and compares the euclidean distance to previous vectors. The problem is that the different data streams are from completely different…
seaotternerd
  • 250
  • 2
  • 9
8
votes
1 answer

Machine learning for pattern recognition in realtime sensor data

I'm working on a project where we need to detect patterns in a sensor's output to find out if a given event occurred. Given my limited experience with machine learning, I was wondering if someone could point me in the right direction. Bellow is a…
ananio
  • 85
  • 1
  • 8
7
votes
6 answers

Modeling of real-time streaming data?

I am interested in tools/techniques that can be used for analysis of streaming data in "real-time"*, where latency is an issue. The most common example of this is probably price data from a financial market, although it also occurs in other fields…
Shane
  • 11,961
  • 17
  • 71
  • 89
7
votes
1 answer

Testing for Benford Law in real time

Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution of first digits…
Nate River
  • 171
  • 4
5
votes
2 answers

How to frequently update classification model with new training data?

How do I incorporate a new stream of data into my classification model? Do I have to retrain the model from the beginning every time I want to incorporate new data, or can I update the existing model with each new data point? Imagine I have built a…
Jane Sully
  • 788
  • 1
  • 9
  • 23
4
votes
5 answers

is K-Means clustering suited to real time applications?

I want to segment a sequence of RGB images (basically it's a video) based on their colors in real time. KMeans is an easy and intuitive algorithm to use in this case, but it's execution time is very sensitive to the clusters' centers initialization…
S.E.K.
  • 139
  • 1
  • 2
  • 8
3
votes
2 answers

Graphing real-time data from a text file

I have a process which writes statistics from a server system to a file each second in this format: label1 label2 label3 344 666 787 344 849 344 939 994 344 There are a number of different values which I need graphs for, and each value is added to…
Espen Herseth Halvorsen
  • 131
  • 3
3
votes
1 answer

Algorithms for real-time classification of segments in noisy time-series data

I’m trying to detect various features of a toy train track while driving on it: The primary input is data from an optical sensor. The following image shows the recorded signal when driving over the railway switch (at the bottom of the pictured…
rluba
  • 131
  • 3
3
votes
2 answers

Realtime data change detection

I have been asked to do some real time data analysis. The data values represent parameters of phone calls for a telco (like number of calls, call length, etc.) If the numbers suddenly drop or spike, I'm supposed to raise an alarm. Normal noise is…
Ron Romero
  • 133
  • 5
3
votes
1 answer

Real time updating of training data and classification model

Setup: I have a couple of binary classification models based on Logistic Regression and Gradient Boosted Trees. Currently I train the model offline and use it to predict the class of incoming data. Problem: One of my features is order_count i.e.…
Mihir Kale
  • 83
  • 1
  • 5
2
votes
1 answer

How to quickly collect data (survey) from students in class?

What would be an easy and simple way to get online responses from students in class for use in class demonstrations. E.g. Height and Weight. A quick survey method in a computer class could make lab work more interesting. Any ideas? Thank you for…
Adam SA
  • 591
  • 2
  • 7
  • 16
1
2 3 4