We're going to predict the closing price of the S&P 500 using a special type of recurrent neural network called an LSTM network. I'll explain why we use recurrent nets for time series data, and why LSTMs boost our network's memory power.
Coding challenge for this video:
Vishal's winning code:
Jie's runner up code:
More Learning Resources:
Please subscribe! And like. And comment. That's what keeps me going.
Join other Wizards in our Slack channel:
And please support me on Patreon:
music in the intro is chambermaid swing by parov stelar
Facebook: https://www.facebook.com/sirajology Instagram: https://www.instagram.com/sirajraval/ Instagram: https://www.instagram.com/sirajraval/
Signup for my newsletter for exciting updates in the field of AI:
Stock prediction is very versatile, you cant predict with mathematical calculations only there is no formula for that. There are lots of factors matters for stock prices going up and low. Its just probabilistic values
I looked at the challenge code, and here is my analysis.
The code is in Python2, so there are some issues there. The two print statements must be changed. The xrange() function is deprecated in Python3 so must changed to range(). If the argument is a float, it must be cast to int. Other than that, it ran well (I copied the code in the Notebook and put in a separate .py program). It ran with the supplied sp500 data, and I created an AAPL (Apple) data file from Yahoo Finance. It ran well, but to get anything meaningful, it should cover at least ten years price history. Right now, I'm analyzing the outputs and see if they are meaningful and really useful for stock trading. As an exercise, I created a virtualenv and installed the TensorFlow and Keras there. It worked great except for the Notebook because I only had Python3 in the environment.
I have had my share of ups and downs when it comes to trading Forex and stock. Now, all I do is win, I mean I make thousands of dollars daily using The *Blended Model Strategy* if you have not heard about it all you have to do is find out about it on google +, The creator is *Dmitry Vladislav* and remember to save this link to thank me later.
Andrew, thank you for the reply. I'm a little rusty at Python and was trying a lot of things quickly. Maybe I'll go back and investigate. For now, I'm just happy everything works. When I was researching range(), I found that in Python2, range() creates a list, but xrange() creates a generator that provides values as they are needed (better if the range is large). Python3 range() adapts to the application so xrange() is deprecated. BTW, I'm running on Ubuntu 18.04 "Bionic Beaver".
This is more just an example of how LSTM recurrent networks work, for any real trading you would want to build a much more complex network and feed it a lot more relevant data such as market sentiment, etc in order to get a better prediction.
Stock prices are determined by expected future value of a company. This kind of pattern detection by looking at backwards data is not a method that actually predicts anything other than trends in larger economic cycles.
Do not get me wrong, the tool itself is great, but anyone serious about trying to predict stock prices should look into the reasons stocks fluctuate and model those variable into a time series.
can i use the output from 2 different independent LSTM networks to feed into a 3rd LSTM network to create the overall network? would it be the same 1 LSTM network which uses all inputs from the 2 LSTMs?
Siraj: Great stuff thank you! Question, could you do a video that is more like "day trading?" Instead of just a single closing price each day, how to use multiple prices each day? Example data set for one month would be 20 (trading days) with 80 price samples (10 per hour).
Hi Siraj, 1st up ....wow!! Love the logic flow of the application but have zip coding skill (VBa does not count) and not even familiar with the interfaces shown. So how would a simpleton engineer apply this using an excel spreadsheet? could Excel even handle this?
Can't call up the functions that you did in your code so would need to write the equations and functions needed.
I want to replicate the logic process utilising the underlying equations and functions to analysis a data set in excel ?? Possible??
New H.W.A Phoenix 5.0 trading tool is Rated for the outstanding performance capacity towards the previous Version Phoenix 4.5, I trade on High & Low and I am using this amazing trading tool with over 93.4% performance rating, (that's a good score). I recommend HEADWAYANALYSIS from my personal trading experience.
What I find a little odd is that trading was previously done by humans, so it had human emotion (hope, fear, greed) as factors in how stocks were traded. But what we're now doing is training computer models to mimic past human behaviour. If just a few people were doing this, then okay they probably have a big advantage, but when everyone is doing it then it seems to kinda detach from reality.
Also, as time goes on and fewer humans are making trading decisions then we'll have to train computer models to react to how other computer models are behaving, which are themselves behaving to how other computer models are behaving. I dunno... something doesn't sit right.
That's pretty much a mumbo jumbo model. The "cheat" that makes the model seem to perform that good, is in the normalization (see the lstm.py from the source https://github.com/jaungiers/LSTM-Neural-Network-for-Time-Series-Prediction). The author is dividing each 50 day slice from the S&P500 by the first 50 day slice....also in the test set. And that is not the same as dividing every value by the very first one. So the stock price movement from the first 50 training days is also partly included in each test example. As soon as you apply more common (valid) normalization methods, the model in not doing better than the monkey throwing darts.
Perhaps you could make a video using attention?
Such as described in this paper as a better alternative to RNN and LSTM:
What they found was that RNN and LSTM have issues with long data strings due to their fixed length internal representation. The subject was explored further here:
In my opinion, Siraj Raval is a pure hype, and someone just copies code from some sources even without giving proper references. He just uses words such as "model" to wrap stuff up which he seems to have NO intuition or idea about. Building a neural net in 4 minutes, Some other deep concept in 30 minutes and such other stuff publish by Siraj Raval made me really dislike him. Those video stuff provide no proper intuition on what's happening inside them. Deep Learning is a DEEP subject that dives into some cool and DEEP mathematical concepts. So something like "Building a neural net in 4 minutes" with proper intuition and understanding of neural networks and deep learning is absolutely ridiculous and nuts. Sentdex's ML series is far better than this comparatively, he-at least provide a decent background to what's going on, even for the math. Just remember this is my opinion.
Can't believe he didn't include how to insert a date to output a price. Would be nice also to have a number of the accuracy of the model. Why else to even include test data?
Anyone knows how to output this?
*Forex trading is not magic.... You are not going to be a millionaire overnight, but its a good background to raise your finances at a steady growth..... You need a proper understanding on its steps by a SUPERB FOREX TEACHER to trade for you, So go with now and make your balance double within To night from here >> **https://t.co/p84BYGCqbZ** .*
*When strategics in the binary world, some people say risk is inevitable, 'YES IT IS TRUE', but i guarantee you that risk is inevitable when it comes with unimagined profit. This Strategy will make you benefited, make a change and rule your world. Start your Trading from here and make your successful forex life >> **https://t.co/pbFk7a6mEJ** .*
*My advice is not to enter the market without basic knowledge. Forex trading is a zero sum game. It’s complex, so you need math skills to understand the market trends and different algorithm. What a party wins, the other loses. So the competition is fierce here. But a good system well developed with high tech algorithm, So trade with the Top Secret Strategy by and make your balance double from here **https://t.co/nxrVq1ATMa** .*
*I am so much impressed with this creative binary trading app (Link Here >> **https://t.co/2nQjFFyekI** ). Backed with expertise, this robot works splendid and if not much, brings in decent amount of profits. Thanks so much the team for creating such innovative robot. It’s like a boon to the traders in earning consistent profits .*
It seems the algorithm does not give you the correct indication of the future trend most of time. Each time I train the model and run the prediction model, it will give you a different prediction result. I run the code given by the link: https://github.com/llSourcell/How-to-Predict-Stock-Prices-Easily-Demo. Maybe the result given by the author is the best result ? Just curious, I don't know. Anyone who tried this model is welcomed to discuss this please.
How do we know an intrinsic value of a product? Combo of all resources that make the product possible. Its market price? Any other sales expenses + profit margin(roi). Anything else is entertaining speculation: taking money from someone to give it to someone else without actually selling the actual product. I mean gambling
I have had my share of ups and downs when it comes to trading Forex. Now, all I do is win, I mean I make thousands of dollars daily using The *Blended Model Strategy* if you have not heard about it all you have to do is find out about it on google +, The creator is *Dmitry Vladislav* and remember to save this link to thank me later.
Actually it makes sense in a weird way if I just let my mind cruise with the flow and allow it self assemble instead of doing the work letting you do it. Now if I can get my LSTM working I will be jaked.
Do you have a video on how to set up the python development environment for this video? I know the readme tells how to do it, but it seems like I would need to be a Python developer already to understand what you have there.
Amazon knows the sales of each manufacturer even before their quarterly reports.
The rest is machine learning and picking them on the stock market. Amazon maps Stock to products-sold using predictive machine learning. Amazon has the data and nobody can get it. It is the ultimate "insider trading". Untouchable by the law.
Aristotle Onassis spied by tapping phone conversations to become rich. What makes you think Bezos is not doing the same, spying on manufactures' sales rates?
I agree that something strange is embedded in the page but I don't think its spam. The github repo of the original of the code is here: https://github.com/jaungiers/LSTM-Neural-Network-for-Time-Series-Prediction
You are very good at researching and reciting material , however you have no insights into ML. I think you are very funny though and I hope you consider teaching stuff that you actually practice and can give deep insights.
Yeah nah. It takes a solid level of understanding to make it look easy.
Plus, when you realise this channel was conceived and designed for a specific niche, it becomes obvious why the charimsa, memes and style was chosen.
This is the very first video I watch of yours. Instant subscriber. You take time to make your videos gripping. Thanks. (It might just be your personality but that's fine). But as well your technical expertise ;)
Hello Fellow lstm noobs, So, this worked decently with up to 5000 data points, but then I scaled it to 35k points and all prediction lines are the same and point negatively. Does anyone know why? I thought it was supposed to improve with data. And before you say overfitting, this is very volatile data
Uploading contracts to an online database should not take too long, and with the right solution, there should be a way to quickly drag and drop them into folders. Of course, the contract management team may want to give some thought as to how those folders are categorized. In some industries, it may make sense to classify them by agreement type, whereas in others they may need to be grouped by timeframe or date. It is obviously important to do what makes sense for your company and to ensure everyone understands the classification system that is instituted. With this sort of well-oiled system in place, it is a lot easier to keep a handle on things.
Divide and Conquer.
This is another area that is very industry-dependent, but it is highly unlikely that any company can afford to have an entire contract team devoted to managing one portfolio. More than likely, it is more realistic to divvy up the team and the contracts so that there is a leader for each relevant sphere. The entire team will obviously have to coordinate and communicate, but resources must be allocated in the most efficient manner possible. In turn, this will allow for several individuals to keep an eye on a smaller batch of contracts, thereby facilitating those periodic reviews.
Outsource the Tedium to Technology.