Time Series Forecasting of Stock Data Using LSTM Deep Learning

The five steps

Forecasting stock data with an LSTM network means moving data through five stages: collecting and pre-processing historical prices, building a recurrent neural network, training it on that data, testing it on a held-out split, and finally using it to generate predictions on new, unseen data.

1. Collect and pre-process the data: We will first need to collect the stock data for the time period we want to forecast. This can be done by accessing financial databases or by manually collecting the data from sources such as stock exchange websites. Next, we will pre-process the data by cleaning and normalizing it. This may include removing any missing or corrupted data, as well as scaling the data to make it easier for the model to process.

2. Build the deep learning model: Once the data has been pre-processed, we will build the deep learning model using a neural network architecture. This may include selecting the type of model (such as a recurrent neural network or a convolutional neural network) and determining the number and size of the layers. We will also need to determine the optimal hyperparameters for the model, such as the learning rate and the number of epochs. LSTM is a good default for this kind of sequence problem, but it isn’t the only option — classical statistical models are worth a look too, and ARIMA is a common baseline to compare against before committing to a neural network.

3. Train the model: Once the model has been built, we will train it using the pre-processed data. This will involve feeding the data into the model and adjusting the weights and biases to optimize the model’s performance.

4. Test the model: After the model has been trained, we will need to test its performance on a separate dataset to ensure that it is able to accurately predict future stock prices.

5. Make predictions: Once the model has been trained and tested, we can use it to make predictions on future stock data. This may involve inputting new data into the model and using the output to make informed decisions about buying and selling stocks.

Walking through an example

As an example, let’s say we want to forecast the stock price of Company X for the next month using a deep learning model. Here are the steps we would follow:

1. Collect and pre-process the data: We collect the stock data for Company X for the past year and pre-process it by cleaning and normalizing the data.

2. Build the deep learning model: We decide to use a recurrent neural network as our model, with two hidden layers and a learning rate of 0.001. We also determine that we will train the model for 50 epochs.

3. Train the model: We feed the pre-processed data into the model and train it using the specified hyperparameters.

4. Test the model: We test the model’s performance on a separate dataset and find that it can accurately predict stock prices with an error rate of 2%.

5. Make predictions: We input new data into the model and use the output to make informed decisions about buying and selling Company X stocks in the next month.

The code

Here is an example of code that can be used to forecast stock data using a deep-learning model with CSV data:

First, we will import the necessary libraries and read the CSV data:

You will find this data in the Kaggle dataset in the following link Stock Market daily data. If you’re new to the surrounding Python data-science stack, our NumPy fundamentals guide covers the array operations that pandas and Keras build on.

import pandas as pd
import numpy as np
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import Dense, LSTM

# Read in the CSV data
df = pd.read_csv('stock_data.csv')

Pre-processing the data

Next, we will pre-process the data by cleaning and normalizing it:

# Convert the 'Date' column to datetime objects
df['Date'] = pd.to_datetime(df['Date'])

# Extract the year, month, and day as separate columns
df['Year'] = df['Date'].dt.year
df['Month'] = df['Date'].dt.month
df['Day'] = df['Date'].dt.day

# Drop the original 'Date' column
df = df.drop(columns=['Date'])

# Scale the data
scaler = MinMaxScaler(feature_range=(0, 1))
df_scaled = scaler.fit_transform(df)

# Split the data into training and testing sets
train_size = int(len(df_scaled) * 0.8)
test_size = len(df_scaled) - train_size
train, test = df_scaled[0:train_size,:], df_scaled[train_size:len(df_scaled),:]

# Convert the data into a 3D array (a sequence with t timesteps and d dimensions)
def create_sequences(data, t, d):
    X, y = [], []
    for i in range(len(data)-t-1):
        a = data[i:(i+t), :]
        X.append(a)
        y.append(data[i + t, :])
    return np.array(X), np.array(y)

# Create sequences of t timesteps with d dimensions
t = 10 # timesteps
d = 9 # dimensions (including year, month, and day)
X_train, y_train = create_sequences(train, t, d)
X_test, y_test = create_sequences(test, t, d)

Building and training the model

Then, we will build and train the deep learning model:

# Build the model
model = Sequential()
model.add(LSTM(50, input_shape=(t, d)))
model.add(Dense(d))
model.compile(loss='mean_squared_error', optimizer='adam')

# Train the model
history = model.fit(X_train,
                    y_train,
                    epochs=50,
                    batch_size=1,
                    verbose=1
                    )

Testing and predicting

Finally, we will test the model and make predictions:

# Test the model
test_error = model.evaluate(X_test, y_test, verbose=2)
print(f'Test error: {test_error}')
print(f'Accuracy: {(1-test_error) * 100}%')

Limitations and practical caveats

A few things are worth knowing before you treat this kind of model as more than a learning exercise:

• Date fields aren’t really continuous values. Scaling Year, Month, and Day alongside price data with the same MinMaxScaler treats them as if they behave like price — a bigger month number doesn’t mean “more” of anything meaningful, and December (12) wrapping back to January (1) can confuse a model trained to expect smooth numeric progressions. Cyclical encodings (sine/cosine of month and day) or dropping raw date columns entirely are common fixes.
• A single train/test split can overstate accuracy. Because the sequence-splitting code above cuts the data once at 80%, the reported test error reflects one specific historical window. Rolling-window or walk-forward validation gives a more honest picture of how the model performs across different market conditions, and the general overfitting and underfitting patterns covered in our guide to fixing model overfitting apply directly here.
• Batch size 1 and 50 epochs is a starting point, not a tuned configuration. In practice you’d sweep the timestep window, batch size, and number of LSTM units, and compare against the kind of systematic tuning practices in our techniques for improving machine learning models. Larger batch sizes generally train faster on modern hardware, at some cost to convergence behavior.
• Stock prices are noisy and only partly predictable from their own history. Price movements are driven by news, earnings, and macro events that never appear in a CSV of past prices, so even a well-tuned LSTM is fundamentally limited by what’s in the input data. Treat a low test error as evidence the model fit the historical pattern, not as a guarantee of future trading performance.