Tesla Stock forecasting feature analysis Austin Li (jl3273), Lang Lei (ll674), Shichen Qi (sq89)

1. Introduction In the modern quantitative research industry,

innovation and the use of previously

neglected data sources are what push the

industry forward. Apart from traditional

methods of stock price forecasting, which

heavily associate with time series analysis,

we’d like to explore ways of effectively

performing such analysis and predictions

without regarding the time series

properties(moving average etc.) In this

project, we investigate the influence of

external factors as additional features to

traditional forecasting models, comparing

the performance of any combination of them

as well. For instance, we will operate the

sentiment analysis of Elon Musk’s Twitter to

see if it fluctuates Tesla’s closing price..

State-of-the-art forecasting techniques such

as recurrent networks and deep learning are

referenced and modified in this project so

that it is technically relevant in 2021.

2. Dataset 2.1 Dataset Description

In order to analyze the relationship between

Elon Musk’s Twitter content and Tesla

stocking price, we subtracted Elon Musk’s

Twitter data from the Twitter API, as well as

stock pricing data of Tesla and its main

competitors (Volkswagen, General Motors

and Ford) from Yahoo Finance.

Competitors Stock Price dataset:

In this dataset, we collect three of Tesla's

major competitors (Volkswagen, General

Motors and Ford) open and close stock

prices from Jan 1, 2019 to Sep 31, 2021.

Tesla Stock Price dataset:

In this dataset, we collect Tesla’s open and

close stock prices from Jan 1, 2019 to Sep

31, 2021 shown below.

To get a better understanding of the

relationship between each brand, we

combined two datasets and utilized two line

graphs to show the change of stock price

from 2019 to 2021 shown below.

It is noticeable that the trend of General

Motors looks more similar to the trend of

Tesla, implying that principal component

analysis may be conducted in the future

topic of interest. Additionally, we add stock

price difference columns for each brand to

get more features to our model. One of the

benefits by doing so is that we can easily tell

the relationship of the stock price for the

same brand.

Twitter dataset:

We extracted the content of Musk’s Twitter

from Sep 31, 2020 to Sep 31, 2021 using

Twitter API.

2.2 Feature Engineering Twitter dataset:

1) Cleaning:

The main datasets that we extracted from the

internet are Twitter. For the Twitter dataset,

we used the regular expression to delete the

user name (for instance, “@xxx”), the image

url and the reference url, and then collected

all the content in the same format. There

would be multiple tweets per day, but we

only need one measurement connected to

each day. Thus, we decided to aggregate the

tweets of the same day, and then conduct the

sentimental analysis.

2) Sentiment Analysis:

We imported Textblob to implement the

sentiment analysis and plot with the

information of Tesla stock going up or

down.

It is shown from the graph above that if Elon

Musk’s tweets have positive attitude or tone,

the scatter points are above zero while if

they are negative, scatter points are below

zero. Points lying on the horizontal line of

zero represent the neutral tone. For the stock

price, green points mean price going up

while red points represent price going down.

We also performed the sentiment analysis of

Musk’s tweets and Tesla stock price with

one day delay.

The graph shown above illustrates that if

Elon Musk’s tweets one day before have

some influence on the second-day stock

price. It is noticeable that both graphs show

some trend between tones and stock price,

which is reasonable to conduct random

forest in our model.

3. Model 3.1 Avoid overfitting

In order to find the best degree in different

polynomial transformations, we compared

the different performances according to the

Mean Square Error (MSE). We used

PolynomialFeatures from sklearn to

construct the transformations with different

exponents. Since the function generates all

polynomial and interaction features

including all polynomial combinations of

the features with degrees less than or equal

to the given degree, the number of features

increases exponentially which means it is

easy to cause overfitting. Therefore, we tried

1, 2, 3 as the given degree and compared the

loss values. The data were divided into

training and testing sets and calculated the

MSE separately.

From the plot, when the degree increases

from 1 to 3, the MSE of training sets

decreases, while the MSEs of testing sets

decrease and then increase, which means the

model starts to be overfitting when the

degree is larger than 2.

3.2 Model Performance Analysis

3.2.1 logistic regression In the first part, we used the all the history

stock prices, including both Tesla and other

stocks, to predict the stock price of Tesla.

From the plot below, we can see that the

prediction and true data are almost identical

and fit the linear regression model.

In the second part, we excluded Tesla from

the data set and only used other stocks to

predict the stock price of Tesla. According

to the plot below, much more outliers are

apart from the linear regression line. This is

a reasonable result since our prediction only

depends on the market performance without

Tesla itself, which might cause larger error

from the actual price.

Lastly, we still used the historical stock

prices excluding Tesla as our data set. This

time we tested the performance of the

polynomial transformation of degree 2, since

in the previous Avoiding overfitting part we

found 2 is the best degree in the model.

Specifically, less outliers are presented in

the plot, which means the polynomial

transformation model fits the data set better

than the linear regression.

3.3.2 Neural Network After we did the feature engineering for the

twitter dataset in the data processing part,

we found out that there is a correlation

between price and tweets tone. In the last

models, we predicted the stock price with a

polynomial transformation model and

decreased the number of outliers to the

actual price. Therefore, we decided to find

the deep underlying relationships between

the price trend and other factors. We used

stock prices of the three competitors (GM, F,

VWAGY), the sentiment scores and the

Tesla’s trend (True or False) the day before

as predictors to predict the price trend the

next day.

The code chunk shown above expresses the

definition of the architecture of our neural

network as well as its forward function. In

the training step, we used 100 epochs and a

constant learning rate of 1*10^-4 without

learning rate decay. The loss values were

calculated with a binary cross entropy loss

function because our output is binary.

This accuracy, though below 0.5, has been

the best we could obtain from modifying the

parameters.

However, we aimed to get a better

prediction of our analysis. Thus, we decided

to get rid of several inputs and change to a

new model to see if it can improve the

performance.

3.3.3 Random Forest

In the previous feature engineering part, we

conducted sentiment analysis of Elon

Musk’s tweets with the performance of Tesla

stock price on the same day. The

visualization suggests that there is a

correlation between two variables.

So for the third model, we are interested in

using a random forest model to separate the

tweets that potentially have positive

influence on the market and otherwise.

Compared with the traditional time series

model, the feature of this random forest

model is to see the connection between price

and content of Musk’s tweets. We can only

tell if the price is going up or down based on

Elon Musk tweets. For this model, we both

calculated the accuracy of the same-day

input and the one-day delay input response.

They are around 0.655 and 0.431 separately,

which means for the same-day model, about

65.6% of the predictions are correct while

for the one-day delay model only 43.1

percent of price trend predictions are

correct.

The graph presented above shows the

accuracy for the random forest model of

the same-day input. Red scatter points

represent the right prediction while black

points show the wrong prediction.

4. Conclusion It is obvious that the history stock price data

set containing Tesla predicts the stock price

best. If we exclude Tesla, the polynomial

transaction model with degree of 2 improves

the prediction compared to the linear logistic

model. Then we calculated the accuracy of

predicting price trends with different

models. In the next model, we used 3

competitors' price, sentiment scores and

Tesla price to predict the price trend, but the

accuracy is lower than 0.5 which is

meaningless. So we decided to reduce inputs

and the random forest model proves to be

effective. The accuracy increases from 0.431

to 0.655. Though our training set is small

and it produces higher variance in the

prediction, the model still shows a

correlation between the price and Twitter

content.

4.1 Weapon of Math Destruction

In the logistic regression model, our output

is the predicted stock price based on

different stock combinations. Thus, the

outputs are floats that are easy to measure.

In the neural network model, we calculated

if the Tesla price will go up or down and

classified it as True and False. These binary

outputs are also measurable. The output of

the random forest is the same as the previous

one.In all, all our response variables can be

measured quantitatively.

Our models use the stock prices and Twitter

from Musk to predict Tesla’s stock price and

if the price will go up or not. All the

resources and references are open to the

public which means people are free to use

them to build their own models and predict

the prices. Thus, our model will not harm

anyone.

Finally, since we only used history stock

prices and Twitter content to predict the

price, it will not create a feedback loop since

we do not use predicted values as our

features.

In conclusion, our project might not produce

a Weapon of Math Destruction.

4.2 Fairness

Our team do not think fairness is very

important to our models. We used all the

stock prices and relevant Twitter

information as our data set, so there is no

discrimination and bias in the data set. Also

since we try to predict Tesla’s stock price, a

small error to actual price is acceptable since

a company will never invest in a single stock

and a portfolio can also decrease the risk.

Finally, predicting stock price will not affect

legal status. Thus, fairness is not an

important factor in the models.

5. Limitation and future improvements 5.1 Limitation

The largest limitation is lack of data. The

first limitation is due to API. We are only

allowed to extract one year of Twitter data

and this insufficient data will increase

variance and cause bias. Also we can only

get the content of tweets and there is no

number of likes and repos, which might also

influence the prediction. Moreover, we

planned to add the information of successful

launch in SpaceX as our feature, but it was

hard to get from the website. Finally, we

only got open and close prices from the

stock information, but there’s insufficient

information about the stock, such as volume,

market capital, high and low prices.

5.2 Improvement

We can also use positive and negative news

on Tesla correlated company to make

important database and stock performance to

better predict the stock price.

6. Appendix

1. SG:pub.10.1007/978-1-4614-9372-3

– springer nature scigraph. (n.d.).

Retrieved December 5, 2021, from

https://scigraph.springernature.com/p

ub.10.1007/978-1-4614-9372-3.

2. Sentiment analysis of Twitter data –

ACL member portal. (n.d.).

Retrieved December 5, 2021, from

https://aclanthology.org/W11-0705.p

df.

3. Pedregosa, F., Varoquaux, G.,

Gramfort, A., Michel, V., Thirion,

B., Grisel, O., Blondel, M., Müller,

A., Nothman, J., Louppe, G.,

Prettenhofer, P., Weiss, R., Dubourg,

V., Vanderplas, J., Passos, A.,

Cournapeau, D., Brucher, M., Perrot,

M., & Duchesnay, É. (2018, June 5).

Scikit-Learn: Machine learning in

Python. arXiv.org. Retrieved

December 5, 2021, from

https://arxiv.org/abs/1201.0490.

4. Pedregosa, F., Varoquaux, G.,

Gramfort, A., Michel, V., Thirion,

B., Grisel, O., Blondel, M., Müller,

A., Nothman, J., Louppe, G.,

Prettenhofer, P., Weiss, R., Dubourg,

V., Vanderplas, J., Passos, A.,

Cournapeau, D., Brucher, M., Perrot,

M., & Duchesnay, É. (2018, June 5).

Scikit-Learn: Machine learning in

Python. arXiv.org. Retrieved

December 5, 2021, from

https://arxiv.org/abs/1201.0490.