Build for free

Python, sentiment analysis, R and Shiny

• 13 min read
Jump to section
Author: Blag aka Alvaro Tejada Galindo
Blag aka Alvaro Tejada Galindo
Tags:
Featured articles
API v3 Calendar API Scheduler
Introducing the new Nylas Scheduler: Experience the future of in-app scheduling
• 4 min read
Read more
API v3 Platform
Signing off Nylas API v2: Welcome Nylas API v3
• 4 min read
Read more
Email API TinyMCE Text Editor
Send emails using TinyMCE Rich Text Editor
• 5 min read
Read more
Jump to section

Knowing how your customers feel about your product is as important as creating a good product. Wouldn’t it be great if we could determine in a simple way how our customers feel by analyzing the feedback they provide via email messages?  With Nylas Neural API’s Sentiment Analysis, we can do that. Sentiment Analysis is a way of computationally analyzing a writer’s work to determine how they feel about a selected topic. With basic sentiment analysis, you can check if the writer feels positive, negative, or neutral about something.

For this post, we’re going to create a simple feedback web page using Python. The sentiment information we gather will be analyzed and stored locally in a SQLite Database to be later picked up and displayed in a Shiny Dashboard using the R Programming Language. (Shiny is an R package that lets you easily display data with a variety of charts and graphs in your own dashboard.) We’re keeping everything local just for the sake of simplicity.

Is your system ready?

If you don’t have the Nylas Python SDK installed and your environment isn’t configured, I would recommend you to read the post How to Send Emails with the Nylas Python SDK where everything is clearly explained.

To complete this project you will need:

  • Nylas developer account and credentials
  • Nylas Python SDK installed
  • Gmail account (we will use Gmail to label and sort emails)
  • We will install R and Shiny later in this post, so you don’t need to worry about it now

What are we going to do?

Let’s pretend that we are the owners of a small company that produces vegan eggs called “VeggiEggs”. We want to build a Flask application that will collect feedback from our customers and send it to us via email, so that we can store the information and also reply back as a follow up. Then we want a Python script that will read and analyze every email using Sentiment Analysis and finally create an R Shiny Script that will display a dashboard with the information we gathered.

Working on the application

What we need to do first is set up a Label so that the emails are easy to find.

Creating a label

On Gmail, we can do the following:

On the search bar, press the “Show Search Options” button.

Search all conversations

Then, use the “veggieggs” and “feedback” words.

Add keywords to search

When we create the filter, we can assign the messages to an existing label or create a new one.

Create new label

We don’t need to nest it, so just enter the label name.

Name the new label

Once we create it, we can apply some extra filters

Add properties to the new label

Now, every email that we receive containing those words “VeggieEggs” and “Feedback” will go into our label and skip the inbox.

Installing the Flask package

As we want to create a Python web application, our best option is to use Flask, one of the most popular Micro Frameworks in the Python world.

To install it, we can type the following command on the terminal:

$ pip3 install flask

Once installed, we’re ready to go.

Creating the Flask application

As we’re going to develop this application locally and not deploy it anywhere else. We’re going to create a file called config.json which will hold our Nylas Tokens information. This comes with Flask, so why not use it?

{
  "NYLAS_OAUTH_CLIENT_ID": "<YOUR_CLIENT_ID>",
  "NYLAS_OAUTH_CLIENT_SECRET": "<YOUR_CLIENT_SECRET>",
  "SECRET_KEY": "<YOUR_SECRET_KEY>"
 }

Now, we’re going to create a file called FeedbackForm.py which will handle getting the feedback and sending the email.

# Import your dependencies
from flask import Flask,render_template,json,request,flash,redirect,url_for
from nylas import APIClient

# Create the app and read the configuration file
app = Flask(__name__)
app.config.from_file("config.json", json.load)

# Initialize your Nylas API client
def load_nylas():
    nylas = APIClient(
        app.config["NYLAS_OAUTH_CLIENT_ID"],
        app.config["NYLAS_OAUTH_CLIENT_SECRET"],
        app.config["SECRET_KEY"]
    )
    return nylas
	
# This the landing page
@app.route("/", methods=['GET','POST'])
def index():
# If we are reading the page, show the feedback form
    if request.method == 'GET':
        return render_template('FeedbackForm.html')
# If we are submitting feedback, read the form
    else:
        name = request.form['name']
        email = request.form['email']
        rating = request.form['rating']
        comments = request.form['comments']
		
# Check for required fields
        if not name:
            flash('Name is required!')
            return redirect(url_for('index'))
        elif not email:
            flash('Email is required!')
            return redirect(url_for('index'))
        elif not comments:
            flash('Comments are required!')
            return redirect(url_for('index'))			
        else:
# If everything is Ok, send the email with the feedback information
            nylas = load_nylas()
            draft = nylas.drafts.create()
            draft.subject = "VeggiEggs Feedback - {} - {} - {}".format(name, 
                            email, rating)
            draft.body = comments
	draft.to = [{"name": "<YOUR_NAME>", "email": "<YOUR_EMAIL>"}]

	draft.send()			
	return render_template('ConfirmationForm.html', 
				   name = name, email = email, 
				   rating = rating, comments=comments), {"Refresh":"5;url=/"}

# Run our application		
if __name__ == "__main__":
	app.run()

For this to work properly, we need to create a file called FeedbackForm.html and we’re going to use TailwindCSS to provide a quick look and feel.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <script src="https://cdn.tailwindcss.com"></script>
    <title>Nylas</title>
</head>
<body>
	<div class="bg-green-300 border-green-600 border-b p-4 m-4 rounded">
	{% block content %}
	{% with messages = get_flashed_messages() %}
		{% if messages %}
			{% for message in messages %}
				{{ message }}
			{% endfor %}
		{% endif %}
	{% endwith %}
	<h1 class="font-black">We hope you enjoyed VeggiEggs. Please take some time to leave a review</h1>
	<br>
	<form method="post">
        <label for="name" class="font-bold">* Name</label>
        <input type="text" name="name"
               placeholder="Your name"
               value="{{ request.form['name'] }}"></input>
        <br><br>
        <label for="email" class="font-bold">* Email</label>
        <input type="email" name="email"
               placeholder="Your email"></input>
        <br><br>
		<label for="rating" class="font-bold">Choose a rating:</label>
		<select name="rating">
			<option value="1">Poor</option>
			<option value="2">Fair</option>
			<option value="3">Ok</option>
			<option value="4">Good</option>
			<option value="5" selected>Great</option>
		</select><br><br>
        <p class="font-bold">* Comments</p>
        <textarea name="comments"
               placeholder="Your comments"
               rows=5
               cols=50></textarea>        
        <br><br>
        <button type="submit" class="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded-full">Submit</button>
    </form>
    <p class="font-bold">* Required</p>
	{% endblock %}
	</div>
</body>
</html>

And we’re going to need a confirmation page as well, so that people know that their responses were recorded correctly.

We’re going to create a page called ConfirmationForm.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <script src="https://cdn.tailwindcss.com"></script>
    <title>Nylas</title>
</head>
<body>
	<div class="bg-green-300 border-green-600 border-b p-4 m-4 rounded">
	{% block content %}
	<h2 class="font-semibold">Thanks {{name}} - {{email}}</h2>
	<h2 class="font-semibold">Your rating was: {{rating}}</h2>
	<h2 class="font-semibold">Your comments were:<br><br> {{comments}}</h2>
	<p class="font-semibold">You will be automatically redirected. 
	If it doesn't work, follow the <a href="{{url_for('index')}}">link</a></p>
	{% endblock %}
	</div>
</body>
</html>

We can execute this script from the terminal by typing:

$ python3 FeedbackForm.py
Running Flask

Now, we need to open localhost:5000 on our favourite browser.

NOTE: If you are using Flask on a Mac that has Monterey or higher, you may have trouble accessing localhost. You can solve this by disabling the Airplay Receiver in your Sharing folder. (Airplay Receiver uses port 5000.) Follow these instructions: How to disable Airplay Receiver.

Feedback form that will provide us with Sentiment Analysis data

Once we hit submit, we will get instant feedback.

Feedback Form Response

We use redirection, so it will go back to the main page after 5 seconds. And here we’re displaying the information entered by the user.

If we check on our mailbox, we will find this:

Feedback form via Email

We have all the information we need on this email. So we’re ready for the next stage.

Creating a Python script

In this step, we’re going to read all the emails with the label “VeggiEggs”, in order to analyze them with the Sentiment Analysis endpoint as well as to gather important information like the rating and the date the email was sent.

We’re going to create a file called NeuralFeeback.py. It looks like this:

# Load your env variables
from dotenv import load_dotenv
load_dotenv()

# Import your dependencies
import os
from nylas import APIClient
import subprocess

# Initialize your Nylas API client
nylas = APIClient(
    os.environ.get("CLIENT_ID"),
    os.environ.get("CLIENT_SECRET"),
    os.environ.get("ACCESS_TOKEN"),
)

# Gather all messages from the label “VeggiEggs”
messages = nylas.messages.where(in_="VeggiEggs")
# Create variables to hold the information to be passed to R
rating = []
sentiment = []
score = []
date = []
sRating = ""
sSentiment = ""
sScore = ""
sDate = ""

# Read all messages
for message in messages:
# Split the title in order to get the rating value
    line = message.subject.split(" - ")
    rating.append(line[3])
# Send each message to the Sentiment Analysis endpoint
    message_analysis = nylas.neural.sentiment_analysis_message([message.id])
# Extract the Sentiment Analysis result
    sentiment.append(message_analysis[0].sentiment)
    score.append(str(message_analysis[0].sentiment_score))
    date.append(str(message.received_at.date()))	
		
# Turn the information into a long string separated by commas
sRating = ','.join(rating)
sSentiment = ','.join(sentiment)
sScore = ','.join(score)
sDate = ','.join(date)

# We want to know when R is getting called
print("Calling R")
# Call R as a subprocess and launch Shiny in the browser
subprocess.run(["RScript", "-e", "shiny::runApp('<path_to_your_python_env>', launch.browser = TRUE)", sRating, sSentiment, sScore, sDate])

Setting up R

Installing R

As we’re going to use Shiny to create our dashboard, we need to create a R script. If you don’t have R installed on your computer you can do it like this.

Here’s the latest version of R for the Mac R-4.1.3. Also, while optional, we need to install XQuartz as it is not bundled with Mac by default.

Latest R version

Installing RStudio

While it’s not mandatory, we should install RStudio which is by far the best R IDE.

Installing the necessary packages

We will need to install some packages in order to have our Shiny application running. Open up RStudio and click on “Tools → Install packages”. Type the following packages separated by a comma:

shiny → Makes it incredibly easy to build interactive web applications with R

tidyverse → The tidyverse is an opinionated collection of R packages designed for data science

treemap → Displays hierarchical data as a set of nested rectangles

imager → Fast image processing for images in up to 4 dimensions

plyr → Tools for Splitting, Applying and Combining Data

Installing R Packages for Sentiment Analysis

We need to make sure to leave “Install dependencies” checked and press Install.

Once everything is installed, we can create the script.

Creating the Shiny script

We need to press File → New → R Script and save the file as App.r then type the following

# Load packages
library("shiny")
library("tidyverse")
library("treemap")
library("imager")
library("plyr")

# UI Section of our Shiny app
ui <- fluidPage(
# Dashboard headline
  tags$div(
    HTML("<h1 style='color:blue;text-align:center;'>VeggiEggs Dashboard!</h1>")
  ),  
# Dashboard title
  title = "VeggiEggs Dashboard!",
# Define how our plots are going to be laid off
                 fluidRow(
                   column(width=5, plotOutput(outputId = "distPlot")),
                   column(width=5,offset=2, plotOutput(outputId = "distPlot2"))
                 ),
                 fluidRow(
                   column(width=5, plotOutput(outputId = "distPlot3")),
                   column(width=5,offset=2, plotOutput(outputId = "distPlot4"))
                 )) 

# Server part, here is where the plots are defined
server <- function(input, output) {

# We receive the information coming from Python
  args = commandArgs(trailingOnly=TRUE)
  ratings<-as.integer(strsplit(args[1], ",")[[1]])
  sentiments<-strsplit(args[2], ",")[[1]]
  scores<-as.integer(strsplit(args[3], ",")[[1]])    
  dates<-as.Date(strsplit(args[4], ",")[[1]])
  
# We create a Data.Frame to better handle the data
  entries = data.frame(ratings, sentiments, scores, dates=as.Date(dates))  
  
# First plot. Summarize the ratings
  output$distPlot <- renderPlot({
    Ratings <- count(entries, 'ratings')
    names(Ratings)<-c("Ratings","Count")
    Count <- Ratings$Count
    Ratings %>%
      ggplot( aes(x = Ratings, y = Count, fill = Count)) + 
        geom_bar(stat="identity")
  })
  
# Second plot. How many feedbacks per day
  output$distPlot2 <- renderPlot({
    Freq <- count(entries, 'dates')
    names(Freq)<-c("Dates","Entries")
    Freq %>% 
      ggplot( aes(x=Dates, y=Entries)) +
      geom_line(color="green") +
      geom_point()
  })
  
# Third plot. Frequency of sentiment analysis
  output$distPlot3 <- renderPlot({
    Sentiment <- count(sentiments)
    group <- paste(Sentiment$x,Sentiment$freq)
    sentiment<-Sentiment$freq
    data <- data.frame(group,sentiment)
    treemap(data,
            index="group",
            vSize="sentiment",
            type="index"
    )
  })

# Fourth Plot. Display an image for each score range  
  output$distPlot4 <- renderPlot({
    Score<-mean(entries$scores)
    if (Score < -0.5){
      image_filename<-"angry-emoji.png"
    }
    if (Score > -0.5 && Score < 0.5){
      image_filename<-"neutral-emoji.png"
    }
    if (Score > 0.5){
      image_filename<-"happy-emoji.jpeg"
    }
    
    image <- load.image(image_filename)
    plot(image)
  })
  
}

shinyApp(ui = ui, server = server)

As we are calling this script from Python, there’s nothing more to do here.

Happy Emoji Sentiment
Neutral Emoji Sentiment
Sad Emoji Sentiment

Here are the images we’re using, and they need to be in the same folder as our script.

Calling the Dashboard

In order to call the Dashboard, the only thing we need to do is run our Python script by typing this into the terminal window:

$ python3 NeuralFeedback.py

The Dashboard will be opened automatically on the default browser.

VeggiEggs Sentiment Analysis Dashboard

That’s it. Hope you like this post! You can get more information on Sentiment Analysis on the Nylas Documentation page.

You can sign up Nylas for free and start building!

Don’t miss the action, watch our LiveStream Coding with Nylas:

Join our technical demo to see the Nylas Platform in action

Related resources

Google Ruby Webhooks
How to create and read Google Webhooks using Ruby

Create and read your Google webhooks using Ruby and Sinatra, and publish them using Koyeb. Here’s the full guide.

• 8 min read
Read more
hero banner
Email API
Build mail merge and email templates using TinyMCE Rich Text Editor

Learn how to build Mail Merge and Email Template functionality in your email workflow using TinyMCE Rich Text Editor.

• 9 min read
Read more
Email API TinyMCE Text Editor
Send emails using TinyMCE Rich Text Editor

Learn how to improve your email workflow using TinyMCE Rich Text Editor.

• 5 min read
Read more