ianspivey dot com

Static Site with Gatsby, Cloudflare Workers, KV, and No Origin Server

June 06, 2019

At Cloudflare, we’ve been talking a lot about the potential of originless services that live entirely at the edge. With the Cloudflare Workers KV storage service going GA, I wanted to play around with the tools and see how easy it was.

tl;dr: this site is a React app that’s entirely stored and served from Cloudflare’s edge. It took about 100 lines of code and config to set up, I can publish with “yarn deploy”, and the whole thing loads (i.e. paints content) in 100-200ms.

It took five steps:

  1. Create a new Gatsby application.
  2. Create a Workers KV Namespace.
  3. Write a script to upload static assets to Workers KV as part of the publish process.
  4. Write a Worker to intercept requests to your domain, fetch static assets from KV based on their path, and send those assets back to the browser.
  5. Use the wrangler CLI to deploy the Worker.

Why bother?

  • Static sites can be lightning-fast because there’s no server-side processing that has to happen between the first client request and the DOM render. They can also be aggressively cached at the browser and CDN because response content at a given URL never needs to change.
  • Gatsby is a powerful tool for building static sites. It can be driven from numerous data sources, so you can use it to build a static site using content from sources like Ghost, Contentful, your local filesystem, a Wordpress blog, or even all of those at the same time.
  • Using Cloudflare Workers and KV means your static site can be served from data centers in 180+ cities around the world at low cost and low latency.

If you have a reasonably-trafficked website whose primary purpose is delivering content (like a blog, help center, or API documentation) then this is a no-brainer way to give your customers a great experience.

Create a new Gatsby application

Follow the Gatsby Quick Start guide. You can stop once you’re able to run gatsby develop and preview your site in your browser.

Create a Workers KV Namespace

With Workers KV, your data is organized into key-value pairs contained within a Namespace.

Follow the Workers KV docs to create a Namespace via the API, or simply log in to the Dashboard, navigate to Workers, and create one interactively in the UI.

Write a script to upload static assets to Workers KV

When you run gatsby build, it creates static assets ready for publishing in the public/ directory in the root of your project. We want to take all of those files and write them to Workers KV.

To make things easy on ourselves, we’ll use the path of each file as a key, and the contents of the file as a value.

Create a file in your project root named upload-static.js:

var fs = require("fs")
var glob = require("glob")
var request = require("request")

const CLOUDFLARE_ACCOUNT_ID = process.env.CLOUDFLARE_ACCOUNT_ID
const CLOUDFLARE_AUTH_EMAIL = process.env.CLOUDFLARE_AUTH_EMAIL
const CLOUDFLARE_AUTH_KEY = process.env.CLOUDFLARE_AUTH_KEY
const CLOUDFLARE_NAMESPACE = process.env.CLOUDFLARE_NAMESPACE

glob("public/**/*", {nodir: true}, function (er, files) {

  files.forEach(filename => {
    file = fs.readFile(filename, (err, data) => {
        const url = `https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/storage/kv/namespaces/${CLOUDFLARE_NAMESPACE}/values/${filename}`
        request.put(url, {
            body: data,
            headers: {
                "X-Auth-Email": CLOUDFLARE_AUTH_EMAIL,
                "X-Auth-Key": CLOUDFLARE_AUTH_KEY
            },
        }, (err, response, body) => {
            if (err) {
              console.log(err)
            }
        })
    })  
  })
})

Next, wire this into your deploy step by adding a script target in the scripts dictionary in your project’s package.json:

   "scripts": {
     "build": "gatsby build",
+    "deploy-static": "node upload-static.js",
     "develop": "gatsby develop",

Make sure to export the required environment variables listed above:

CLOUDFLARE_ACCOUNT_ID
CLOUDFLARE_AUTH_EMAIL
CLOUDFLARE_AUTH_KEY
CLOUDFLARE_NAMESPACE

Other than your email address, these should all be alphanumeric uids that you can find in the Cloudflare dashboard.

Test it out by running from the project root:

yarn deploy-static

If you want to verify that everything was uploaded, you can easily do that by fetching from the Cloudflare API:

request(`https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/storage/kv/namespaces/${CLOUDFLARE_NAMESPACE}/keys`, {
    headers: {
        "X-Auth-Email": CLOUDFLARE_AUTH_EMAIL,
        "X-Auth-Key": CLOUDFLARE_AUTH_KEY
    }
}, (err, response, body) => {
    console.log(body)
}) 

Write a Worker to route static assets

Our Worker needs to do a few things:

  • Fetch static asset values from Workers KV, using the request path as the key
  • Set an appropriate Content-Type header on the Response
  • Cache Responses that we want to cache (pretty much everything)
  • Do a little special-case path-munging to map / to /index.html and prepend the public/ prefix to match our keys.

In your project root, create a file worker-fetch-static.js:

addEventListener('fetch', event => {
    event.respondWith(handleRequest(event))
})

const _types = {
    'css': 'text/css',
    'html': 'text/html',
    'json': 'application/json',
    'js': 'application/javascript',
    'png': 'image/png'
}

getType = function(path) {
    path = String(path);
    var last = path.replace(/^.*[/\\]/, '').toLowerCase();
    var ext = last.replace(/^.*\./, '').toLowerCase();
  
    var hasPath = last.length < path.length;
    var hasDot = ext.length < last.length - 1;
  
    return (hasDot || !hasPath) && _types[ext] || null;
};
  

async function handleRequest(event) {
    let request = event.request
    let cache = caches.default
    let response = await cache.match(request)

    if (response) {
        return response
    }

    const parsedUrl = new URL(request.url)
    let path = parsedUrl.pathname

    var cacheMe = true
    let lastSegment = path.substring(path.lastIndexOf('/'))
    if (lastSegment.indexOf('.') === -1) {
        path += 'index.html'
        cacheMe = false
    }

    path = 'public' + path
    const value = await static.get(path, "stream")
    const mime_type = getType(path)

    response = new Response(value, { headers: { "Content-Type": mime_type } })
    if (cacheMe) {
        event.waitUntil(cache.put(request, response.clone()))
    }

    return response
}

Use wrangler to deploy your worker

You could log in to the Cloudflare Dashboard and upload your Worker via the Editor UI, but it’s even more convenient if you can deploy from the command line (or a continuous deployment pipeline), so let’s set that up.

Install wrangler, if you haven’t already, by following the installation instructions.

To generate wrangler config in your existing project, run:

wrangler init .

Now modify that config as appropriate for your project in wrangler.toml:

name = "worker-fetch-static"
type = "javascript"
zone_id = "YOUR_ZONE_ID_HERE"
private = false
account_id = "YOUR_ACCOUNT_ID_HERE"
route = "example.com/*"

We’re using the javascript type because we only need to upload our single JS file, worker-fetch-static.js, and don’t need to worry about bundling anything else with it.

Next, wrangler looks to deploy your package’s main entrypoint as a Worker, so you’ll need to edit the package.json of your application:

   "license": "MIT",
-  "main": "n/a",
+  "main": "worker-fetch-static.js",
   "repository": {

Finally, you can update your package.json to add a few script shortcuts to make deployment easier:

   "scripts": {
     "build": "gatsby build",
+    "clean": "gatsby clean",
+    "deploy": "yarn run deploy-static && wrangler publish",
+    "deploy-static": "node upload-static.js",
     "develop": "gatsby develop",

Now try it out:

$ yarn deploy

Navigate to the path you deployed your worker to (in my case, ianspivey.com) and check out your new Gatsby site!


Ian Spivey

Written by Ian Spivey. You should follow him on Twitter