Decoding Stock Market

Now that I have become bored of IT and data work (and have reached some milestones), I am going to take a few days break and get my creative juices flowing. One of the ways is decoding the stock market.

There are different ways of looking at stock market. One is where does it go from here. This is where machine learning algorithms come into play.  Let's take an example. An algo that incorporates HMM matches the currently observed  sequence of states with ones encountered during training. It then applies that change to the existing prices.  But the accuracy is not great. Why? Randomness element for sure. But can we improve it further??

I am not a big fan of interpretability of models. If the thing can be proved to be statistically significant, it should be good enough. The black swan event needs to be considered ofcourse. But apart from that, whatever works fine should be considered. I guess Renaissance is doing the same.

Factor 1: I guess qualitative factors can improve the results. A lot of algorithms may be using the qualitative input of what caused the move (which is not always apparent  as often prices move first and fundamentals comes later. Courtesy, really smart people, insider trading. Will write a  separate article on why sometimes prices move first). The sequence of states thing captures this to some extent. Don't think it does this really well.

Things like Twitter chatter can also be fed into the model.  Feeding in this extra bit of information could be helpful. The more the variables with better predictive power, the better are the results.

One can follow the smart moneyl

I have previously written a blog post breaking down the reasons of price movements.

I think the real value will come from better data. Data needs to filtered for noise and then modelling done. Models can do the filtering to some extent but not much (due to many factors). This should improve the results significantly.

Ultimately, neural networks will come to the rescue. You don't need to understand everything about the markets.