Build for free

How to build an AI-powered meeting notes generator

• 9 min read
Jump to section
Author: Ram Bansal
Ram Bansal
Tags:
Featured articles
Nylas Launches Notetaker API: Setting a New Standard for Embedded Meeting Intelligence
• 3 min read
Read more
Notetaker API
Introducing the Nylas Notetaker API
• 5 min read
Read more
Notetaker API
Integrating Nylas Notetaker into your meetings: a step-by-step guide for developers
• 10 min read
Read more
Jump to section

Introduction

This post explores building an AI-powered meeting notes generator using the Nylas Notetaker API and AI coding tools. The goal is to automate recording, transcribing, analyzing, and formatting meeting notes. While software to do this exists, building with the Notetaker API enables a custom solution and offers a tailored integration for your user workflows.

This post demonstrates how developers can create a robust meeting note system using AI to increase productivity.

What We’ll Build

We’re going to build a tool that generates structured, Markdown-formatted meeting notes. The workflow looks like this:

  1. Transcription – Use Nylas Notetaker API to record and transcribe your meeting. This gives us a detailed JSON transcript to build with.
  2. Template + AI Summarization – Share the transcript with AI (i.e. OpenAI’s API) to summarize and organize the content.
  3. Polished Notes – Generate readable meeting notes (complete with sections like agenda items, discussion topics, action items, etc.) that you can easily share and revisit.

By the end, we will build a tool to support creating formatted meeting notes for all types of meetings — all generated for you!

Tools Used

To build the AI-powered meeting notes generator, we use the following tools:

  • Nylas Notetaker API: For meeting transcription and metadata. Notetaker joins your meetings and produces a transcript with speaker labels, timestamps, and more.
  • Node.js: JavaScript runtime to execute the meeting generator script.
  • OpenAI API: For AI-powered summarization of meeting content. Feel free to bring your own AI model, or use a local one.
  • AI Coding Assistant (e.g., Claude Code, Cursor, Replit): We will use Claude Code to build the meeting notes generator from scratch.

Step 1: Retrieving Meeting Data with Nylas Notetaker

Meeting transcripts can be retrieved as JSON using the Nylas Notetaker API. The transcript contains an array of meeting data, including a transcript with speaker name, timestamps, and metadata like meeting title, date, and participants.
Here is an example transcript from the transcript_content array:

{
    ...more details...
    "transcript_content": [
        { "speaker": "Joel", "text": "Hello, how's it going?" },
        { "speaker": "Ram", "text": "Hi Joel! It's going well, how about you?" }
    ]
    ...more details...
}

In a fullstack or production environment, transcripts are retrieved via a webhook or API call after a meeting finishes. This JSON structure includes all necessary data: meeting attendees, date, time, and the detailed transcript with speaker labels. Check out our recent post on integrating Nylas Notetaker to learn more.

Step 2: Setting Up Meeting Note Templates

Next let’s create Markdown templates with placeholders to dynamically generate meeting notes:

1-on-1 Meeting Template Example

# One-on-One Meeting Summary: {date}

## Participants
- {participant1}
- {participant2}

## Meeting Agenda
{agenda}

## Discussion Topics
{discussion_topics}

## Action Items
{action_items}

## Next Steps
{next_steps}

## Notes
{additional_notes}

Templates use placeholders like {date}, {participant1}, and {action_items} enclosed in curly braces. These placeholders are replaced with information from the meeting transcript using AI. Templates can be created for various meeting types:

  • One-on-one meetings
  • Team meetings
  • Sales calls

Now we’ve created meeting templates to use when generating meeting summaries.

Step 3: Build an AI Powered Meeting Notes Generator

The script, meeting_notes_generator.js , automates the creation of structured meeting notes using Notetaker produced transcripts. To create the script, we can use Claude Code to accelerate software development. In a new folder, /meeting-notes-generator, let’s first start with a specification before we run Claude in our CLI:

// in spec.md file
Generate a script in Node.js that will allow the user to run the following command:

`node meeting_notes_generator.js --input ./meetingTranscript/meeting-transcript.json --output ./meetingSummary/1-on-1-Summary.md`

To generate a meeting summary that will take the template meeting format from `templates` folder and the meeting transcript from `meetingTranscript` to output a meeting Summary into `meetingSummary` using OpenAI API.

So the script will:

1. Read the --input argument
2. Provide options to the user to select which template they like to use from the options listed (i.e. 1-on-1, Team Meeting, Project Meeting, Sales Call) using the templates from `./templates` folder
3. Extract the transcript content
4. Generate LLM prompts to analyze the transcript
5. Call the OpenAI API to process the transcript
5. Format the response into meeting notes using the specified template
6. Save the formatted notes to --output argument

Using this specification, Claude Code will analyze our directory structure and propose what to build out for our review before the code is generated. 

The script loads the transcript, identifies participants, and determines the meeting type. The script then utilizes the OpenAI API to summarize the content based on a specific prompt tailored to the meeting type. The AI-generated summary is used to fill a Markdown template, resulting in meeting notes. Let’s go over key functions from the script generated by Claude next. Key functions include:

  • extractTranscriptContent(transcriptData): Extracts usable content from the transcript such as dialog between participants, unique speaker names, meeting date.
// Extract transcript content for analysis
const extractTranscriptContent = (transcriptData) => {
  try {
    // Handle case where the data has transcript_content property
    if (Array.isArray(transcriptData) && transcriptData.length > 0 && transcriptData[0].transcript_content) {
      const firstItem = transcriptData[0];
      
      // Extract participants
      const speakers = new Set();
      firstItem.transcript_content.forEach(item => {
        if (item.speaker && item.speaker !== 'Speaker C') {
          speakers.add(item.speaker);
        }
      });
      
      // Extract meeting date
      const meetingDate = new Date(firstItem.created_at).toLocaleDateString();
      
      // Create a dialogue string from the transcript
      const dialogue = firstItem.transcript_content
        .map(item => `${item.speaker}: ${item.text}`)
        .join('\n');
      
      return {
        dialogue,
        participants: Array.from(speakers),
        date: meetingDate
      };
    }
    
    // Handle other formats if needed
    console.error('Unrecognized transcript format');
    process.exit(1);
  } catch (error) {
    console.error(`Error extracting transcript content: ${error.message}`);
    process.exit(1);
  }
};
  • generateAIPrompt(transcriptContent): Constructs a natural-language prompt for the LLM to summarize the meeting and fill out predefined sections like check-In, action items, and feedback.
// Generate AI prompt for meeting analysis
const generateAIPrompt = (transcriptContent) => {
  return `
    You are an AI assistant that summarizes meeting transcripts. 
    Analyze the following meeting transcript between        
    ${transcriptContent.participants.join(' and ')} 
    and generate the following sections:

    1. Check-in: Brief summary of how participants were feeling or what they mentioned at     
    the start of the meeting
    2. Progress Updates: Summary of any project updates or progress reports mentioned 
    during the meeting
    3. Discussion Topics: Main points discussed during the meeting, organized by topic
    4. Action Items: Specific tasks that participants agreed to do, with assignees if     
    mentioned
    5. Next Steps: What participants agreed would happen after the meeting
    6. Feedback: Any feedback that was exchanged during the meeting
    7. Additional Notes: Any other important information that doesn't fit into the above   
    categories
    
    Meeting Transcript: ${transcriptContent.dialogue}
`;
};
  • callOpenAI(prompt): Sends the prompt to OpenAI and retrieves a structured, plain-text summary of the meeting.
// Call OpenAI API to process the transcript
const callOpenAI = async (prompt) => {
  try {
    const openai = initializeOpenAI();
    const response = await openai.createChatCompletion({
      model: "gpt-3.5-turbo",
      messages: [
        {
          role: "system",
          content: "You are a helpful assistant that summarizes meeting transcripts in a concise and organized way."
        },
        {
          role: "user",
          content: prompt
        }
      ],
      max_tokens: 1500,
      temperature: 0.7,
    });

    return response.data.choices[0].message.content.trim();
  } catch (error) {
    console.error('Error calling OpenAI API:', error.response ? error.response.data : error.message);
    process.exit(1);
  }
};
  • formatSummary(template, aiResponse, transcriptContent): Parses the AI response by detecting section headers and mapping them to template placeholders, and injects meeting metadata like date and participant names.
// Format the AI response using the selected template
const formatSummary = (template, aiResponse, transcriptContent) => {
  try {
    let summary = template;

    // Replace date and participants
    summary = summary.replace('{date}', transcriptContent.date);
    
    if (transcriptContent.participants.length >= 1) {
      summary = summary.replace('{participant1}', transcriptContent.participants[0]);
    }
    
    if (transcriptContent.participants.length >= 2) {
      summary = summary.replace('{participant2}', transcriptContent.participants[1]);
    }

    // Extract sections from AI response
    const sections = {
      check_in: '',
      progress_updates: '',
      discussion_topics: '',
      action_items: '',
      next_steps: '',
      feedback: '',
      additional_notes: ''
    };

    // Simple parsing of AI response sections
    const lines = aiResponse.split('\n');
    let currentSection = '';

    for (const line of lines) {
      if (line.toLowerCase().includes('check-in:')) {
        currentSection = 'check_in';
        continue;
      } else if (line.toLowerCase().includes('progress updates:')) {
        currentSection = 'progress_updates';
        continue;
      } else if (line.toLowerCase().includes('discussion topics:')) {
        currentSection = 'discussion_topics';
        continue;
      } else if (line.toLowerCase().includes('action items:')) {
        currentSection = 'action_items';
        continue;
      } else if (line.toLowerCase().includes('next steps:')) {
        currentSection = 'next_steps';
        continue;
      } else if (line.toLowerCase().includes('feedback:')) {
        currentSection = 'feedback';
        continue;
      } else if (line.toLowerCase().includes('additional notes:') || line.toLowerCase().includes('notes:')) {
        currentSection = 'additional_notes';
        continue;
      }

      if (currentSection && line.trim()) {
        sections[currentSection] += line + '\n';
      }
    }

    // Replace template placeholders with content
    Object.entries(sections).forEach(([key, value]) => {
      summary = summary.replace(`{${key}}`, value.trim());
    });

    return summary;
  } catch (error) {
    console.error(`Error formatting summary: ${error.message}`);
    process.exit(1);
  }
};

You can pull down the entire code base on GitHub. In this section we went over some of the key functionality from meeting_notes_generator.

Step 4: Running the Notes Generator

To run the meeting notes generator, ensure you install all packages (npm install) and an OpenAI API key in the .env file if using OpenAI. Use the following CLI command to run the script:

node meeting_notes_generator.js <path_to_transcript_json> --api-key <YOUR_OPENAI_API_KEY> --output <output_notes.md>

The script reads the JSON transcript, processes it with the OpenAI API, and saves the formatted notes. The CLI output includes a success message and output file name:

Please select a template to use:
1. 1-on-1
2. project-meeting
3. sales-call
4. team-meeting
Enter your selection (number): 1
Meeting summary successfully saved to ./meetingSummary/1-on-1-Summary.md

Now we have a meeting summary created with AI using transcripts from Notetaker.

Build Time!

In this blog post, we went over the steps to integrate the Nylas Notetaker API into your meetings to automatically record and transcribe audio from meetings.

You can sign up for Nylas for free and start building! Continue building with Nylas by exploring different quickstart guides or by visiting our developer documentation.

Related resources

Notetaker API
Integrating Nylas Notetaker into your meetings: a step-by-step guide for developers

Learn how to integrate Nylas Notetaker into your meetings in this step-by-step guide.

• 10 min read
Read more
Hero Banner - How to Integrate Gmail with Cursor
Email API
How to Integrate Gmail with Cursor

Learn how to build a gmail integration fast with Cursor Agent and Nylas APIs in this step-by-step guide—perfect for developers and small teams.

• 7 min read
Read more
Screen shot of Replit AI Agent with prompt input
Calendar API Email API
How to Build a CRM with Replit AI Agent: A Step-by-Step Guide

Learn how to build a CRM fast with Replit AI Agent and Nylas APIs in this step-by-step guide—perfect for developers and small teams.

• 17 min read
Read more