Predicting S&P 500 based on its constituents and their social media derived sentiment

Olaniyan, Rapheal; Stamate, Daniel; Pu, Ida; Zamyatin, Alexander; Vashkel, Anna and Marechal, Frederic. 2019. 'Predicting S&P 500 based on its constituents and their social media derived sentiment'. In: 11th International Conference on Computational Collective Intelligence ICCCI 2019. Hendaye, France 4-6 September 2019. [Conference or Workshop Item]

iccci2019_v2.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

Download (416kB) | Preview

Abstract or Description

Collective intelligence, represented as sentiment extracted from social media mining, is encountered in various applications. Numerous studies involving machine learning modelling have demonstrated that such sentiment information may or may not have predictive power on the stock market trend, depending on the application and the data used. This work proposes, for the first time, an approach to predicting S&P 500 based on the closing stock prices and sentiment data of the S&P 500 constituents. One of the significant complexities of our framework is due to the high dimensionality of the dataset to analyse, which is based on a large number of constituents and their sentiments, and their lagging. Another significant complexity is due to the fact that the relationship between the response and the explanatory variables is time-varying in the highly volatile stock market data, and it is difficult to capture. We propose a predictive modelling approach based on a methodology specifically designed to effectively address the above challenges and to devise efficient predictive models based on Jordan and Elman recurrent neural networks. We further propose a hybrid trading model that incorporates a technical analysis, and the application of machine learning and evolutionary optimisation techniques. We prove that our unprecedented and innovative constituent and sentiment based approach is efficient in predicting S&P 500, and thus may be used to maximise investment portfolios regardless of whether the market is bullish or bearish.

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):


Collective intelligence, Sentiment analysis, Stock market prediction, Feature selection , Feature clustering, PCA, Jordan and Elman Neural Networks, Evolutionary computing, Statistical tests, Granger causality

Departments, Centres and Research Units:



21 May 2019Accepted
9 August 2019Published

Event Location:

Hendaye, France

Date range:

4-6 September 2019

Item ID:


Date Deposited:

29 May 2019 14:06

Last Modified:

09 Jun 2021 13:35


View statistics for this item...

Edit Record Edit Record (login required)