2

I have created a algorithm that signals when to buy a certain stock. When that signals is generated a buy the stock and hold it for lets say 10 days. So the log returns for that period will be ln(p(t+10)/p(t)). So lets say that I get like 3000 log returns from this and I can plot it as a histogram and compute the mean and the standard deviation. How can I test if the mean of these returns are statistically significant higher than 0. Can I use a one sided t-test for this or would that be misleading since we are not sure about the distribution of the 10 day returns?

I appreciate any advice. Thank you!

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
bjorks
  • 21
  • 2

1 Answers1

4

You want a 2-sided test becuase the returns can be negative. But why the interest in statistical significance ? The sample mean obviously will be different from zero, so just adjust your significance level accordingly and hey presto you have statistical significance. Or just make the sample size bigger and just like magic you can get a significant result. But how useful is statisitcal significance ? In my opinion it is not useful at all, because all it tells you is, if the algorithm actually generates a zero mean return, the probability of obtaining these returns, or even higher (if they are positive) or lower (if they are negative). More important is the effect size (the mean return itself)

Robert Long
  • 53,316
  • 10
  • 84
  • 148
  • Thank you for the answer! I think my question was incomplete. I actually want to answer the question if the returns for my holding period is statistically significant higher than the risk free rate(unadjusted for the risk).So for each time I take a trade I evaluate the returns by ln(p(t+10)/p(t)) minus the compounded one day risk free rate under the same holding period and I want to be able to give an answer like: "The null hypothesis that the true means for excess returns are zero is rejected by the asymptotic 95% confidence intervals for 20, 40 and 60 trading days".Does that make for sense? – bjorks May 22 '21 at 11:51
  • Since my distribution of the returns will be a bit skewed in some directions and I don't know if the distribution of the returns follow a normal distribution can i still use a t-test? – bjorks May 22 '21 at 11:53
  • If they are not plausibly nomally distributed (nothing is perfectly normally distributed) then a t test is inappropriate. [See here](https://stats.stackexchange.com/questions/9573) for details. – Robert Long May 22 '21 at 11:58
  • if you are worried about the histogram not being normal or symmetric, you can always used some non-parametric test. But, as bjorks said, even if you reject the null of zero ( or whatever mean you're testing ), you have to worry about slippage, transaction cost and diversification so you'll still have a ways to go to see if the strategy is viable. – mlofton May 22 '21 at 13:39
  • 1
    Actually, the effect size is *not* more important. The reason for this is that risk-averse investors must be compensated for taking on risk; they do not just want to maximize expected returns at any cost. Performing the numerical operations equivalent to a t-test and using that as a decision rule, even if it is not statistically valid or meaningful, actually comes closer to the investor's actual portfolio optimization objective because you're effectively crudely downweighing very risky assets by dividing by the variance. – Chris Haug May 22 '21 at 13:44
  • 1
    @ChrisHaug Interesting, thanks. So a "significant" p-value of 0.04999 and an associated effect size of 0.1% higher than 0 (or higher than the risk free rate) is better than a "non significant" p-value of 0.05001 with an associated effect size of 10% higher ? That seems quite strange to me. – Robert Long May 22 '21 at 13:50
  • Thanks for interesting comments. My approach is that I am testing several different strategies. The first test is to see if they even can generate statistically significant higher returns than the risk free rate. Once that is done I can see that some strategies can't even do that but some can. I then take the strategies that can generate higher returns than the risk free rate and risk adjust these returns and see if they can generate alpha according to the CAPM. I know these might not be the case for how you test algorithmic trading strategies in general, – bjorks May 22 '21 at 14:33
  • But it is for a school project so I have to relate it so some known models in finance. Like the CAPM. – bjorks May 22 '21 at 14:34
  • @bjorks perhaps you can post some plots for the assessment of normality ? QQ plot and histogram would be a good place to start. Maybe there is a simple transformation you can apply to make them normal) or perhaps the departure from normality isn't too great in which case you are good with a t-test. – Robert Long May 22 '21 at 14:44
  • 2
    @RobertLong For the same sample size, for the p-value to be almost the same with a much bigger effect size, the variance of the returns has to be much higher. How investors trade off between risk and reward depends on their utility function, but yes some investors will definitely prefer a less risky but smaller reward (e.g. sovereign bonds). – Chris Haug May 22 '21 at 14:53
  • 1
    @RobertLong You can also view what the OP is doing as computing the Sharpe ratio and setting a minimum threshold below which assets won't be further considered. They arbitrarily set this threshold by appealing to t-test-like machinery. – Chris Haug May 22 '21 at 15:08
  • 1
    @ChrisHaug now you're making my brain ache. The last time I heard about the Sharpe ratio was when I was an undergrad (30+ years ago) :D – Robert Long May 22 '21 at 15:12