How to Perform Sentiment Analysis on Earnings Call of Companies

In this project, we will make use of Python scripting, the AssemblyAI platform, and the streamlit framework to develop a website for the deployment of the project. We will make use of three separate files for this project. In the first file, we will store our configuration API key obtained from the AssemblyAI website. We will create a second file for downloading and saving a specific YouTube file. Finally, we will develop the main website and computation of sentiment analysis in the final Python file.

Configuration of your API key:

To get started with the project, I would recommend checking out the AssemblyAI platform. Here, you can obtain an API key through which we can perform the project of sentiment analysis on the earnings call of companies. You can follow a simple sign-up process if you don’t already have an account. As soon as you log in, you can access your free API key on the right side of your AssemblyAI account. Copy this and place it in a Python file configure.py.

auth_key = "Enter Your API Key Here"

Saving the audio data:

In the next Python file, we will save the audio data downloaded from YouTube. For this section, we will import the youtube_dl library, which can be installed with a simple pip install command if you don’t have it already. Once the library is imported, we will create a variable that will hold all the essential parameters. We will download the audio data in an mp3 format and save it in the best audio format. This action can be done as shown in the code block below.

import youtube_dlydl_opts = {
'format': 'bestaudio/best',
'postprocessors': [{
'key': 'FFmpegExtractAudio',
'preferredcodec': 'mp3',
'preferredquality': '192',
}],
'ffmpeg-location': './',
'outtmpl': "./%(id)s.%(ext)s",
}

Once we declare the necessary parameters, we can proceed to create a function that will help us to save the desired YouTube video in the best audio file mp3 format. We will obtain the idea and strip any spaces and pass the id for extracting the information. Once the video file is downloaded, it will be stored in the present working directory. You can change the paths as you desire. The code below represents the following function.

def save_audio(link):
_id = link.strip() def get_vid(_id):
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
return ydl.extract_info(_id) # download the audio of the YouTube video locally
meta = get_vid(_id)
save_location = meta['id'] + ".mp3" print('Saved mp3 to', save_location) return save_location

We can save this file as save_audio.py and continue building the main application from the next section.

Import the essential libraries:

After the creation of the two primary Python files, we can proceed to create the main file called app.py for developing the website interface and computing the sentiment analysis accordingly. We will firstly import all the necessary requirements for our project, including the two Python files that we previously created.

import streamlit as st
from save_audio import save_audio
from configure import auth_key
import pandas as pd
from time import sleep
import urllib.request
import plotly.express as px
import plotly.graph_objects as go
from urllib.request import urlopen
from bs4 import BeautifulSoup
import json
import requests

Each of these libraries will be utilized for creating the website with statistical graphs for analyzing the various parameters of the company’s report.

Setting up all the required elements:

In this section, we will set up all the necessary requirements that will be necessary to view the various graphs for sentiment analysis, including the initialization of the desired variables. Firstly, let us create the endpoints and initialize the headers accordingly. The transcript endpoint contains the location to the AssemblyAI platform where the transcription of the audio data is available. Similarly, we also have an upload endpoint and the headers assignment with the AssemblyAI authorization API key.

## AssemblyAI endpoints and headers
transcript_endpoint = "https://api.assemblyai.com/v2/transcript"
upload_endpoint = 'https://api.assemblyai.com/v2/upload'headers_auth_only = {'authorization': auth_key}
headers = {
"authorization": auth_key,
"content-type": "application/json"
}

Once we initialize the required parameters, it would be best to set up some of the explanations related to the purpose of the sentiment analysis. We can make use of titles, captions, and subheaders available in the streamlit library to create some of the primary elements of the website.

## App explanation
st.title('Sentiment analysis of earning calls')
st.caption('With this app you can analyse the sentiment of earnings calls by providing a YouTube link to its recording.')
st.subheader('Submit a video link or choose one of the pre-determined ones to analyse.')st.subheader('Submit a video link or choose one of the pre-determined ones to analyse.')

In the next step, we will create a text input bar in which the user can input a specific link referring to a particular company’s earnings call. The data of the provided link (usually a YouTube video) would be downloaded in the best audio format and saved locally in the working directory. Once the data is downloaded, we will upload the data in terms of chunks to the AssemblyAI website and obtain the uploaded audio URL. We have also specified a default link that grants us access to one of the earnings call of Amazon.

# Get link from user
video_url = st.text_input(label='Earnings call link', value="https://www.youtube.com/watch?v=UA-ISgpgGsk")# Save audio locally
save_location = save_audio(video_url)## Upload audio to AssemblyAI
CHUNK_SIZE = 5242880def read_file(filename):
with open(filename, 'rb') as _file:
while True:
data = _file.read(CHUNK_SIZE)
if not data:
break
yield dataupload_response = requests.post(
upload_endpoint,
headers=headers_auth_only, data=read_file(save_location)
)audio_url = upload_response.json()['upload_url']
print('Uploaded to', audio_url)

Now that we have finished uploading our audio data to the AssemblyAI website, we can start with the transcription of the audio file. We will provide the uploaded audio URL and specify the condition of sentiment analysis as True. When these parameters are set, we can proceed to receive a transcript response and the transcript id. Finally, set up the polling endpoint for performing the transcription.

## Start transcription job of audio file
data = {
'audio_url': audio_url,
'sentiment_analysis': 'True',
}transcript_response = requests.post(transcript_endpoint, json=data, headers=headers)
print(transcript_response)transcript_id = transcript_response.json()['id']
polling_endpoint = transcript_endpoint + "/" + transcript_idprint("Transcribing at", polling_endpoint)Image By Author

We can check the completion status of the program and wait for the transcription process to be completed with the help of a While loop. Once the transcription process is completed, we can obtain the transcript that we can display for the user to view. But for this step, we will just store the text of the received transcript response.

## Waiting for transcription to be done
status = 'submitted'
while status != 'completed':
print('not ready yet')
sleep(1)
polling_response = requests.get(polling_endpoint, headers=headers)
transcript = polling_response.json()['text']
status = polling_response.json()['status']

Once we have completed all the steps mentioned in this section, we can proceed to the next section for visualizing the results obtained by performing sentiment analysis on the uploaded data.

Analyzing the numerous components through visualizations:

Image By Author

Once the transcript is created from the transcription of the audio file, we obtain the text data that we can display. We can display them in the sidebar of the website. We will also obtain the sentiment analysis results and convert them into a pandas data frame to make it easier to perform analysis on it.

# Display transcript
print('creating transcript')
st.sidebar.header('Transcript of the earnings call')
st.sidebar.markdown(transcript)print(json.dumps(polling_response.json(), indent=4, sort_keys=True))## Sentiment analysis response
sar = polling_response.json()['sentiment_analysis_results']## Save to a dataframe for ease of visualization
sen_df = pd.DataFrame(sar)
print(sen_df.head())

We can obtain the title of the video with the help of the beautiful soup library and display it on the website. We will also print the total number of sentences and display the information accordingly.

## Get the title of this video
with urlopen(video_url) as url:
s = url.read()
soup = BeautifulSoup(s)
title = soup.title.stringst.header(title)## Visualizations
st.markdown("### Number of sentences: " + str(sen_df.shape[0]))grouped = pd.DataFrame(sen_df['sentiment'].value_counts()).reset_index()
grouped.columns = ['sentiment','count']
print(grouped)col1, col2 = st.columns(2)

Let us create our first bar graph for understanding the amount of positive, neutral, or negative sentiments that are involved in the sentences from the earnings call of the data. You can plot the graph with either the specified parameters in the code block below or choose to make use of a different color scheme and other similar features.

# Display number of positive, negative and neutral sentiments
fig = px.bar(grouped, x='sentiment', y='count', color='sentiment', color_discrete_map={"NEGATIVE":"firebrick","NEUTRAL":"navajowhite","POSITIVE":"darkgreen"})fig.update_layout(
showlegend=False,
autosize=False,
width=400,
height=500,
margin=dict(
l=50,
r=50,
b=50,
t=50,
pad=4
)
)col1.plotly_chart(fig)

A bar graph might not always be indicative of the true perception required to analyze any problem. Hence, we will make use of an indicator that will signify if we have more positive or negative responses. We will calculate the sentiment score as shown in the below code block and analyze the indicator. The up arrow signifies a positive indication, whereas the negative indication will be represented through the down indicator. Check the above image for a more intuitive understanding.

## Display sentiment score
pos_perc = grouped[grouped['sentiment']=='POSITIVE']['count'].iloc[0]*100/sen_df.shape[0]
neg_perc = grouped[grouped['sentiment']=='NEGATIVE']['count'].iloc[0]*100/sen_df.shape[0]
neu_perc = grouped[grouped['sentiment']=='NEUTRAL']['count'].iloc[0]*100/sen_df.shape[0]sentiment_score = neu_perc+pos_perc-neg_percfig = go.Figure()fig.add_trace(go.Indicator(
mode = "delta",
value = sentiment_score,
domain = {'row': 1, 'column': 1}))fig.update_layout(
template = {'data' : {'indicator': [{
'title': {'text': "Sentiment score"},
'mode' : "number+delta+gauge",
'delta' : {'reference': 50}}]
}},
autosize=False,
width=400,
height=500,
margin=dict(
l=20,
r=50,
b=50,
pad=4
)
)col2.plotly_chart(fig)

Finally, we will also create a scatter plot for better visualization of the sentiments. This visualization can be created as shown in the below code snippet.

## Display negative sentence locations
fig = px.scatter(sar, y='sentiment', color='sentiment', size='confidence', hover_data=['text'], color_discrete_map={"NEGATIVE":"firebrick","NEUTRAL":"navajowhite","POSITIVE":"darkgreen"})fig.update_layout(
showlegend=False,
autosize=False,
width=800,
height=300,
margin=dict(
l=50,
r=50,
b=50,
t=50,
pad=4
)
)st.plotly_chart(fig)

With all these visualizations completed, we now have a brief understanding of the sentiment behind the earnings call of a company. We can predict how optimistic or not they are about the future.

Final Setup:

Image By Author

Note that the run times for the following project depend on numerous factors. The speed of the results obtained might vary for each user depending on the type of system, the upload speed, and other factors, usually taking about a few minutes.

For the final setup, I would recommend checking out the following GitHub link that covers the entire code and the requirements that are necessary to successfully deploy the following project. I would also suggest checking out the following video tutorial, which covers the following topic in a concise manner covering most of the intricate details related to this project.