Testing webhooks using the Nylas CLI and PHP

Testing webhooks using the Nylas CLI and PHP

13 min read
Tags:

The Nylas CLI is an amazing tool that can be used for a lot of different things, but did you know we can even test webhooks without the need of an external web server? Before we get started, let’s review what webhooks are, and why they are important.

What are webhooks?

Webhooks are notifications triggered by certain events like receiving an email, opening a link inside an email, when an event has been created or deleted and more. Webhooks are important because they are submitted to our applications when something important happens, without the need of our application having the need of pulling information every “x” amount of time.

Why are webhooks important?

A webhook is triggered by the server and it’s delivered to your application without your application having to request it. Instead of having to make multiple requests, your application can wait until it receives a webhook. This makes applications more efficient and most important, faster.

When sending an email or creating an event, it’s important to know if the message was opened, a click inside the messages was clicked or if an event was modified or deleted. Having all this information will help us to make better decisions.

Using the Nylas CLI

Before getting started with the Nylas CLI, we need to install it.

For details on how to install and use the CLI, we can read the blog post Working with the Nylas CLI.

Setting up a Tunnel

In order to test webhooks on the CLI, we need to create a webhook Tunnel. This will register a temporary webhook for us and will keep it open for around 2 hours. It will also provide us with a web server and will manage all the webhook details so we don’t have to worry about anything.

We need to type (The print flag is just so that we can see the output on the CLI):

$ nylas webhook tunnel --print
Nylas Webhooks Tunnel

And just like that, our webhook is created, our web Server is ready and we only need to trigger some events and see what happens. We can just send an email to ourselves:

Draft email

We can confirm that the email was received by us:

Checkin our inbox

And check the CLI for the webhook notification:

Printing the Webhooks response in the CLI

Awesome, huh? We got one notification because we got a new email, and another one because we opened the email. What will happen if we delete the email? Yes, we will get another notification.

But, the information is a little bit hard to read, especially if we get a lot of notifications. What we can do is get some help from a PHP web application.

Using the Nylas CLI and PHP

Is your system ready?

If you already have PHP installed and configured please keep reading. Otherwise install PHP and the Apache web server. I’m using a Mac and PHP installation is long and with some extra troubles.

Basically, you can use:

$ brew install php

But this will be installed without a signature, so it’s not going to work out of the box at least on MacOS Monterrey.

This post explains how to get PHP up and running.

Installing Composer

Composer is a package manager for PHP, similar to npm, pip or gem.

To install it we need to first download the installer rename it to composer-setup.php and the run it on the terminal window:

$ php composer-setup.php --install-dir=~/.local/bin --filename=composer

Now we can call it as:

$ composer

We will use composer on the next section.

Creating the PHP webpage

We’re going to create a small script just to print out some information coming from the CLI and call it index.php:

<?php
# Read the incoming webhook
$data = file_get_contents("php://input");
# Turn the information into JSON
$events = json_decode($data, true);

# Loop the event
foreach ($events as $event) {
# Print out all the information
    var_dump($event);
}
?>

When we run this script, it will receive input from the Nylas CLI and print it in a more readable way.

First, we need to be inside the folder where we created our index.php file, then we can run this script from the terminal by using:

$ php -S 127.0.0.1:8000

This will launch an internal PHP webserver:

Launching the PHP Server

Our web server is alive on http://127.0.0.1:8000/ but it doesn’t have anything to show and it might give us an error if we open it on the browser as we build it in order to work with the CLI.

Calling our PHP app from the Nylas CLI

Now, we have everything ready and we can call our PHP Application from the Nylas CLI.

We can type:

$ nylas webhook tunnel -f http://127.0.0.1:8000/
Linking the Nylas CLI with the PHP Server

The webhook tunnel will now forward the results to our PHP web application console.

If we send a delete message, we will see this on the CLI, which looks better and more organized.

Getting a Trigger message

Using PHP

We may be wondering, “Haven’t we used PHP already?” and the answer would be “Yes, we did. But we did it locally”. Now it’s time to move to the outside world.

What we are going to do is create a PHP webpage and upload it to Heroku, so it’s freely accessible. This web application is going to use SQLite to store the webhook information and display it on the browser. While using SQLite on Heroku is not recommended because it will reset itself after a day or every time there’s a new deployment, it’s the easiest way to do some testing and it can be migrated later on to Postgresql if needed.

Setting up a Heroku account and CLI

If we don’t have a Heroku account, we can sign up for one here. Once that’s done, we will need to install the Heroku CLI. I found that the best option is to use Brew:

$ brew tap heroku/brew && brew install heroku

Once that’s done, simply log into the Heroku CLI:

$ heroku login

Creating the PHP webpage

As we’re going to deploy our application to Heroku, we need to perform some additional steps.

First, we’re going to download a repository from Github called php-getting-started:

$ git clone [email protected]:heroku/php-getting-started.git

Once cloned, we change its name to webhooks and modify what we need.

Inside the web folder, there’s a file called index.php that we need to replace with our own code:

<?php
# Import your dependencies
require('../vendor/autoload.php');
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;

# Create a new Silex application
$app = new Silex\Application();

# Specify the location of the views folder
$app->register(new Silex\Provider\TwigServiceProvider(), array(
    'twig.path' => __DIR__.'/views',
));

# This is the main page
$app->get('/', function() use($app){
    $conn = get_db_connection();
    $webhooks = $conn -> query('SELECT * FROM webhooks');
    return $app['twig']->render('index.twig', ['webhooks' => $webhooks]);
});

# This will be called to validate our webhook
$app->get('/webhook', function(Request $request) {
    if(!empty($request->query->get('challenge')))
    {
        return $request->query->get('challenge');
    }
});

# Page for the Webhook to send the information to
$app->post('/webhook', function(Request $request) use($app) {	
    $is_genuine = verify_signature(
        file_get_contents('php://input'),
        utf8_encode(getenv('CLIENT_SECRET')),
        $request->headers->get('X-Nylas-Signature')
    );

# Is it really coming from Nylas?	
    if(!$is_genuine){
        return new Response('Signature verification failed!', 401);
    }
	
# Read the webhook information and store it on the database
    $json = file_get_contents('php://input');
    $data = json_decode($json, true);
    $conn = get_db_connection();
    $email_to = "";

# Get information about the message
    foreach ($data as $delta) {
        if($delta[0]["type"] == "message.created" or 
           $delta[0]["type"] == "message.updated"){
            $message = get_message($delta[0]["object_data"]["id"]);
        }else if ($delta[0]["type"] == "message.opened" or 
                  $delta[0]["type"] == "message.link_clicked"){
	$message = get_message($delta[0]["object_data"]["metadata"]["message_id"]);
    } 
		
    foreach ($message["to"] as $email){
        $email_to = $email_to . ", " . $email["email"]; 
    }
		
    $email_to = ltrim($email_to, ", ");
		
    $date = date('r', $delta[0]["date"]);
    $stmt = $conn -> prepare ('INSERT INTO webhooks values 
                    (:id, :date, :title, :from_, :to_, :type)');
    $stmt -> bindParam(':id', $delta[0]["object_data"]["id"]);
    $stmt -> bindParam(':date', $date);
    $stmt -> bindParam(':title', $message["subject"]);
    $stmt -> bindParam(':from_', $message["from"][0]["email"]);
    $stmt -> bindParam(':to_', $email_to);
    $stmt -> bindParam(':type', $delta[0]["type"]);
    $stmt -> execute();
}

    return new Response('Webhook received', 200);	
});	

# Get message information
function get_message($id){
    $curl = curl_init();
	
    $access_token = getenv('ACCESS_TOKEN');
	
    curl_setopt_array($curl, array(
	CURLOPT_URL => 'https://api.nylas.com/messages/' . $id,
	CURLOPT_RETURNTRANSFER => true,
	CURLOPT_ENCODING => '',
	CURLOPT_MAXREDIRS => 10,
	CURLOPT_TIMEOUT => 0,
	CURLOPT_FOLLOWLOCATION => true,
	CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
	CURLOPT_CUSTOMREQUEST => 'GET',
	CURLOPT_HTTPHEADER => array(
		'Accept: application/json',
		'Content-Type: application/json',
		'Authorization: Bearer ' . $access_token 
	),
	));

    $response = curl_exec($curl);
    $d = json_decode($response, true);
    return $d;
}

# Verify the signature of the webhook sent by Nylas
function verify_signature($message, $key, $signature){
    $digest = hash_hmac('sha256', $message, $key);
    return(hash_equals($digest, $signature));
}

# Connect to the Database
function get_db_connection(){
    $db = new PDO("sqlite:database.db");
    return $db;
}

# Execute the application
$app->run();

We will notice right away that the code that we are using is more complex than our initial application, and that’s because when Nylas sent us a webhook, it expects our application to send a response back in order to validate that the service is working properly. Also, we want to validate the signature to make sure that it’s coming from Nylas and not someone else.

On the views folder, we can safely delete the files header.html, layout.html and nav.html, then we can create a new file called base.html and replace the contents of index.twig:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>{% block title %} {% endblock %}- Nylas Webhooks</title>
    <style>
        .webhook {
            padding: 10px;
            margin: 5px;
            background-color: #f3f3f3
        }
    </style>
</head>
<body>
    <div class="content">
        {% block content %} {% endblock %}
    </div>
</body>
</html>

This should be code for index.twig:

{% extends 'base.html' %}

{% block content %}
    <h1>{% block title %} Webhooks {% endblock %}</h1>
    {% for webhook in webhooks %}
        <div class='webhook'>
            <p><b>Id:</b> {{ webhook.id }} | 
               <b>Date:</b> {{ webhook.date }} |
	       <b>Title:</b> {{ webhook.title }} | 
               <b>From:</b> {{ webhook.from_ }} |
	       <b>To:</b> {{ webhook.to_ }} | 
         <b>Event:</b> {{ webhook.type }}
	</p>
        </div>
    {% endfor %}
{% endblock %}

As we’re going to use SQLite we need to create a Schema and a Table. We can call this file schema.sql:

DROP TABLE IF EXISTS webhooks;

CREATE TABLE webhooks (
    id TEXT PRIMARY KEY,
    date TEXT NOT NULL,
    title TEXT NOT NULL,
    from_ TEXT NOT NULL,
    to_ TEXT NOT NULL,
    type TEXT NOT NULL
);

Running this script using php init_db.php will generate the database.db file we need for our application.

<?php
$db = new SQLite3('database.db');
$myfile = fopen("schema.sql", "r") or die("Unable to open file!");
$db->exec(fread($myfile,filesize("schema.sql")));
fclose($myfile);
?>

Once database.db gets generated, we need to move it into the web folder.

Deploying our application to Heroku

The first thing we need to do is to initialize our git repository and add our files:

$ git init
$ echo vendor >> .gitignore
$ git add .
$ git commit -m “Initializing Git repository” 

We need to add support for SQLite, so we can open our composer.json file and add the following:

{
  "require" : {
    "silex/silex": "^2.0.4",
    "monolog/monolog": "^1.22",
    "twig/twig": "^2.0",
    "symfony/twig-bridge": "^3",
    "ext-pdo_sqlite": "*"
  },
  "require-dev": {
    "heroku/heroku-buildpack-php": "*"
  }
}

Running the command composer install will update the composer.lock file.

We must not forget to commit our changes. Otherwise, they will not be taken into account:

$ git add composer.lock
$ git commit -m “Add deployment files”

Finally, we’re going to create our Heroku application:

$ heroku create blag-php-webhooks

Application names in Heroku must be unique, so you can change blag with your own name.

Since we’re going to need to access our Nylas Tokens, we’re going to use Heroku Environment Variables.

We need to go to our Heroku Dashboard and select our application. Then go to Settings:

Heroku Settings

And select Reveal Config Vars:

Reveal Heroku Config Vars
Setup Heroku Config Vars

This information can be found on our Nylas Dashboard.

With all that ready, we can push our changes and deploy our application:

$ git push heroku main
Pushing changes to Heroku

We can now open our web application by typing:

$ heroku open
Launching our app

It looks somehow empty, but that’s because we don’t have any webhook notifications yet. For that, we need to create a webhook first.

Creating a webhook

If we go to our Nylas Dashboard and choose webhooks:

Dashboard Webhooks

We can press Create a webhook:

Create a Webhook

Here is the most important piece, as we need to provide the URL of our Heroku PHP Server and choose the Triggers that we want to use.

Passing the Heroku URL to create a Webhook

For this example we’re going to choose the first three Message Triggers.

Choosing message triggers

Notice that message.link_clicked only works when an email is sent from a paid account, so the free trial will not work.

When we press Create Webhook, we can see our webhook listed.

Webhook created

Now, if we send an email to our connected account, we will see that the event is saved and displayed on our PHP application. Notice that I haven’t implemented auto refresh, so we need to manually refresh the page.

Sending a test email

We will see the first webhook reflected on the webpage.

Checking our first Webhook response

Awesome! A new email was received.

So the next step is to send an email from a paid account so that we can track when the email is opened and more important when a link is clicked. For this, we’re going to use the Nylas Python SDK. More information on how to set up and send emails using Python can be found on How to Send Emails with the Nylas Python SDK.

We’re going to create a file called SendEmailTracking.py:

# Load your env variables
from dotenv import load_dotenv
load_dotenv()

# Import your dependencies
import os
from nylas import APIClient

# Initialize your Nylas API client
nylas = APIClient(
    os.environ.get("CLIENT_ID"),
    os.environ.get("CLIENT_SECRET"),
    os.environ.get("ACCESS_TOKEN"),
)

# Add tracking capabilities
tracking = {
        "links": "true",
        "opens": "true",
        "thread_replies": "true",
        "payload": "new-payload",
}

# Create the draft email
draft = nylas.drafts.create()
draft.subject = "With Love, from Nylas"
draft.body = "This email was sent using the Python SDK for the 
              Nylas Email API. 
              Visit <a href='https://www.nylas.com'>Nylas</a> for details."
draft.to = [{'name': 'Blag', 'email': [email protected]'}]
draft.tracking = tracking

# Send the email!
draft.send()

Notice that the link needs to be between link tags.

Here’s the email that we received.

Checking our inbox for incoming email

We can open it:

Opening up the message with a link

And click on the link:

Checking all upcoming webhooks

All events are correctly logged and displayed.

You can sign up Nylas for free and start building!

If you want to learn more about webhooks, please go to our webhooks documentation.

Related resources

How to Solve Webhook Integration Challenges with PubSub Notification Channel

Key Takeaways This article addresses the challenges of webhook integration and introduces the PubSub Notification…

How to Send Emails Using an API

Key Takeaways This post will provide a complete walkthrough for integrating an email API focused…

How to build a CRM in 3 sprints with Nylas

What is a CRM? CRM stands for Customer Relationship Management, and it’s basically a way…