1

To ask my question, consider the following example. I have data on the price of 100 stocks over time. I want to have good indicators on the price of my stocks without having to look into the data of each stock every day.

The first thing I can do is average all the prices on the stocks at each time, but with this data set the average of all the prices is not well correlated with any of the stocks.

The next thing I could do is group stocks. Lets say I have no apriori knowledge about a natural grouping for the stocks. My thoughts would be to build the correlation matrix. I would then look at stock A and select all the stocks which have a >0.90 correlation with stock A. I would then check that they all mutually have >0.85 correlation and eliminate those that don't. I would then average the prices of those stocks for each time, and the resulting series would be reasonably correlated with the group of stocks.

This would involve a lot of trial and error and manual work though. Is there a more procedural way to go about it?

Very broadly speaking, I seek a set of variables that each maximizes the amount of stocks that are correlated >0.90 with it.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
beebeetatter
  • 11
  • 3

1 Answers1

0

So if I understand correctly, you want to find subsets of stocks which are highly correlated with each other, and then track the averages within each group?

One way to identify such clusters is to build the correlation matrix (as you did), then pick some cutoff which you want the within-cluster pairwise correlation to be greater than (say $0.8$). Use this cutoff to convert your correlation matrix $C$ to a binary matrix $B$ where: $$ B_{ij} = \left\{ \begin{array}{ll} 1 & \mbox{if } C_{ij} \geq \text{cutoff} \\ 0 & \mbox{if } C_{ij} < \text{cutoff} \end{array} \right. $$

You can then identify highly-correlated subsets of stocks by treating this binary matrix as the adjacency matrix of a graph, and partitioning it into its maximal cliques.

If you are using R or Python, then you can use the igraph package to perform this clique partitioning.

UPDATE:

You could also find the clusters from $C$ directly using the similarity-based approaches described here

jon_simon
  • 1,679
  • 1
  • 14
  • 26
  • Thanks for a great answer. I will try to tear into this in the next few days. – beebeetatter Sep 17 '17 at 18:19
  • I have absolutely zero background in group theory, but from 10 minutes of reading what you wrote seems to make sense. I would then take the stocks comprising each maximal clique and average them to my desired variables X_i. I will then do this for several cutoffs and compare. It is not clear to me that the maximal clique will maximize the average correlation of the stocks in the clique with X_i, but perhaps this is provable? – beebeetatter Sep 17 '17 at 18:27