The Gatsby Migration, pt.3 - Smart Pages

15 March 2020

Updated: 03 September 2023

Introduction

So far we’ve created the initial react application as with a few routes for our Home, Blog, and 404 pages. In this post we’ll look at how we can set up our Post component to render our pages dynamically based on the JSON data we have. We’ll also extend this so that we can have some more content in a markdown file that we’ll parse and add to our Gatsby data

Creating the initial React App
Rendering the “Dumb” pages with Gatsby
Rendering the “Smart” page with Gatsby (This post)

Setting Up

We’re going to make our data a little more complex by creating two additional markdown files in our static/posts directory to enable us to have more content with each post

Create the following markdown files in the application and align the names with our post-1.json and post-2.json files:

static/posts/post-1.md
static/posts/post-2.md

Gatsby Plugins

To read the data from our files we’re going to do the following:

Use the gatsby-source-filesystem to read our files into the Gatsby Data Layer
Define our own plugin that can read the file content, parse the markdown, and add it into the data layer

Reading the File Metadata

To read our file data we will need to first install the gatsby-source-filesystem plugin. Plugins in Gatsby enable us to ingest or transform data in our application. We then make use of GraphQL to query the data from the relevant component

Install the gatsby-source-filesystem plugin with:

1
yarn add gatsby-source-filesystem

And then add the plugin configuration to the gatsby-node.js file into the plugins array:

gatsby-node.js

1
{
2
    resolve: `gatsby-source-filesystem`,
3
    options: {
4
        name: `content`,
5
        path: `${__dirname}/static/posts`,
6
    },
7
}

This will read all the data from our posts directory into the filesystem. We can now start the application back up with yarn start and navigate to http://localhost:8000/__graphql in our browser to view the GraphQL data. We should be able to see the GraphiQL interface

From the GraphiQL interface run the following query to see the data from the files in our directory:

1
query PostData {
2
  allFile {
3
    nodes {
4
      name
5
      extension
6
      absolutePath
7
    }
8
  }
9
}

This should yield the following JSON with our file meta data in it:

1
{
2
  "data": {
3
    "allFile": {
4
      "nodes": [
5
        {
6
          "name": "post-1",
7
          "extension": "json",
8
          "absolutePath": "C:/repos/cra-to-gatsby/static/posts/post-1.json"
9
        },
10
        {
11
          "name": "1",
12
          "extension": "jpg",
13
          "absolutePath": "C:/repos/cra-to-gatsby/static/posts/1.jpg"
14
        },
15
        {
16
          "name": "post-1",
17
          "extension": "md",
18
          "absolutePath": "C:/repos/cra-to-gatsby/static/posts/post-1.md"
19
        }
20
        // file 2 data
21
      ]
22
    }
23
  }
24
}

Processing the Files

Now that we have our metadata for each file in the file system, we’re going to create a plugin that will allow us to read the file data and add it the GraphQL data layer

In order to do this, create a plugins directory in the root folder. Inside of the plugins directory create a folder and folder for our plugin.

Create a new folder in the plugins directory with another folder called gatsby-transformer-postdata

From this directory run the following commands to initialize and link the yarn package:

plugins/gatsby-transformer-postdata

1
yarn init -y
2
yarn link

We’ll also add the showdown package which will allow us to convert the markdown into the HTML so we can render it with our Post component

1
yarn add showdown

And then create an gatsby-node.js file in this directory with the following content:

/plugins/gatsby-transformer-postdata/gatsby-node.js

1
const fs = require('fs')
2
const crypto = require('crypto')
3
const showdown = require('showdown')
4

5
exports.onCreateNode = async ({ node, getNode, actions }) => {
6
  const { createNodeField, createNode } = actions
7

8
  // we'll process the node data here
9
}

This exposes the onCreateNode Gatsby API for our plugin. This is what Gatsby calls when creating nodes and we will be able to hook into this to create new nodes with all the data for each respective post based on the created file nodes

From the onCreateNode function we’ll do the following to create the new nodes:

Check if it is a markdown node
Check if the JSON file exists
Read file content
Parse the metadata into an object
Convert the markdown to HTML
Get the name of the node
Define the data for our node
Create the new node using the createNode function

gatsby-transformer-postdata/gatsby-node.js

1
const fs = require('fs')
2
const crypto = require('crypto')
3
const showdown = require('showdown')
4

5
exports.onCreateNode = async ({ node, actions, loadNodeContent }) => {
6
  const { createNodeField, createNode } = actions
7

8
  // 1. Check if it is a markdown node
9
  if (node.internal.mediaType == 'text/markdown') {
10
    const jsonFilePath = `${node.absolutePath.slice(0, -3)}.json`
11

12
    console.log(jsonFilePath)
13

14
    // 2. Check if the JSON file exists
15
    if (fs.existsSync(jsonFilePath)) {
16
      // 3. Read file content
17
      const markdownFilePath = node.absolutePath
18
      const markdownContent = fs.readFileSync(markdownFilePath, 'utf8')
19
      const jsonContent = fs.readFileSync(jsonFilePath, 'utf8')
20

21
      // 4. Parse the metadata into an object
22
      const metaData = JSON.parse(jsonContent)
23

24
      // 5. Convert the markdown to HTML
25
      const converter = new showdown.Converter()
26
      const html = converter.makeHtml(markdownContent)
27

28
      // 6. Get the name of the node
29
      const name = node.name
30

31
      // 7. Define the data for our node
32
      const nodeData = {
33
        name,
34
        html,
35
        metaData,
36
        slug: `/blog/${name}`,
37
      }
38

39
      // 8. Create the new node using the `createNode` function
40
      const newNode = {
41
        // Node data
42
        ...nodeData,
43

44
        // Required fields.
45
        id: `RenderedMarkdownPost-${name}`,
46
        children: [],
47
        internal: {
48
          type: `RenderedMarkdownPost`,
49
          contentDigest: crypto
50
            .createHash(`md5`)
51
            .update(JSON.stringify(nodeData))
52
            .digest(`hex`),
53
        },
54
      }
55

56
      createNode(newNode)
57
    }
58
  }
59
}

From the root directory you can clean and rerun the application:

1
yarn clean
2
yarn start

Now, reload the GraphiQL at http://localhost:8000/__graphql and run the following query to extract the data we just pushed into the node:

1
query AllPostData {
2
  allRenderedMarkdownPost {
3
    nodes {
4
      html
5
      name
6
      slug
7
      metaData {
8
        body
9
        image
10
        title
11
      }
12
    }
13
  }
14
}

This should give us the relevant post data:

1
{
2
  "data": {
3
    "allRenderedMarkdownPost": {
4
      "nodes": [
5
        {
6
          "html": "<p>Hello here is some content for Post 1</p>\n<ol>\n<li>Hello</li>\n<li>World</li>\n</ol>",
7
          "name": "post-1",
8
          "slug": "/blog/post-1",
9
          "metaData": {
10
            "body": "Hello world, how are you",
11
            "image": "/posts/1.jpg",
12
            "title": "Post 1"
13
          }
14
        },
15
        {
16
          "html": "<p>Hello here is some content for Post 2</p>\n<ol>\n<li>Hello</li>\n<li>World</li>\n</ol>",
17
          "name": "post-2",
18
          "slug": "/blog/post-2",
19
          "metaData": {
20
            "body": "Hello world, I am fine",
21
            "image": "/posts/2.jpg",
22
            "title": "Post 2"
23
          }
24
        }
25
      ]
26
    }
27
  }
28
}

Create Pages

Now that we’ve got all our data for the pages in one place we can use the onCreatePages API to create our posts, and the Post component to render the pages

Setting Up

Before we really do anything we need to rename the Blog.js file to blog.js as well as create the src/components directory and move the Post.js file into it, you may need to restart your application again using yarn start

Create Pages Dynamically

In our site root create a gatsby-node.js file which exposes an onCreatePages function:

gatsby-node.js

1
const path = require('path')
2

3
exports.createPages = async ({ graphql, actions }) => {
4
  const { createPage } = actions
5
}

From this function we need to do the following:

Query for the PostData using the graphql function
Create a page for each renderedMarkdownPost

gatsby-node.js

1
const path = require('path')
2

3
exports.createPages = async ({ graphql, actions }) => {
4
  const { createPage } = actions
5

6
  // 1. Query for the PostData using the `graphql` function
7
  const result = await graphql(`
8
    query AllPostData {
9
      allRenderedMarkdownPost {
10
        nodes {
11
          html
12
          name
13
          slug
14
          metaData {
15
            body
16
            image
17
            title
18
          }
19
        }
20
      }
21
    }
22
  `)
23

24
  result.data.allRenderedMarkdownPost.nodes.forEach((node) => {
25
    // 2. Create a page for each `renderedMarkdownPost`
26
    createPage({
27
      path: node.slug,
28
      component: path.resolve(`./src/components/Post.js`),
29
      context: node,
30
    })
31
  })
32
}

Render the Page Data

From the Post component we need to:

Export the query for the data
Get the data for the Post
Render the data

components/post.js

1
import { graphql } from 'gatsby'
2
import App from '../App'
3

4
const Post = ({ data }) => {
5
  // 2. Get the data for the Post
6
  const postData = data.renderedMarkdownPost
7

8
  // 3. Render the data
9
  return (
10
    <App>
11
      <div className="Post">
12
        <p>
13
          This is the <code>{postData.slug}</code> page
14
        </p>
15
        <h1>{postData.metaData.title}</h1>
16
        <div
17
          className="markdown"
18
          dangerouslySetInnerHTML={{ __html: postData.html }}
19
        ></div>
20
      </div>
21
    </App>
22
  )
23
}
24

25
export default Post
26

27
// 1. Export the `query` for the data
28
export const query = graphql`
29
  query PostData($slug: String!) {
30
    renderedMarkdownPost(slug: { eq: $slug }) {
31
      html
32
      name
33
      slug
34
      metaData {
35
        body
36
        image
37
        title
38
      }
39
    }
40
  }
41
`

From the Post component above we use the slug to determine which page to render, we also set the HTML content for the markdown element above using the HTML we generated. We also now have our pages dynamically created based on the data in our static directory

You can also see that we have significantly reduced the complexity in the Post component now that we don’t need to handle the data fetching from the component

If you look at the site now you should be able to navigate through all the pages as you’d expect to be able to

Summary

By now we have completed the the entire migration process - converting our static and dynamic pages to use Gatsby. In order to bring the dynamic page generation functionality to our site we’ve done the following:

Used the gatsby-source-filesystem plugin to read our file data
Created a local plugin to get the data for each post and convert the markdown to HTML
Use the onCreatePages API to dynamically create pages based on the post data
Update the Post component to render from the data supplied by the graphql query

And that’s about it, through this series we’ve covered most of the basics on building a Gatsby site and handling a few scenarios for processing data using plugins and rendering content using Gatsby’s available APIs

Nabeel Valley