Tag: Fallback

Static First: Pre-Generated JAMstack Sites with Serverless Rendering as a Fallback

You might be seeing the term JAMstack popping up more and more frequently. I’ve been a fan of it as an approach for some time.

One of the principles of JAMstack is that of pre-rendering. In other words, it generates your site into a collection of static assets in advance, so that it can be served to your visitors with maximum speed and minimum overhead from a CDN or other optimized static hosting environment.

But if we are going to pre-generate our sites ahead of time, how do we make them feel dynamic? How do we build sites that need to change often? How do we work with things like user generated content?

As it happens, this can be a great use case for serverless functions. JAMstack and serverless are the best of friends. They complement each other wonderfully.

In this article, we’ll look at a pattern of using serverless functions as a fallback for pre-generated pages in a site that is comprised almost entirely of user generated content. We’ll use a technique of optimistic URL routing where the 404 page is a serverless function to add serverless rendering on the fly.

Buzzwordy? Perhaps. Effective? Most certainly!

You can go and have a play with the demo site to help you imagine this use case. But only if you promise to come back.

https://vlolly.net

Is that you? You came back? Great. Let’s dig in.

The idea behind this little example site is that it lets you create a nice, happy message and virtual pick-me-up to send to a friend. You can write a message, customize a lollipop (or a popsicle, for my American friends) and get a URL to share with your intended recipient. And just like that, you’ve brightened up their day. What’s not to love?

Traditionally, we’d build this site using some server-side scripting to handle the form submissions, add new lollies (our user generated content) to a database and generate a unique URL. Then we’d use some more server-side logic to parse requests for these pages, query the database to get the data needed to populate a page view, render it with a suitable template, and return it to the user.

That all seems logical.

But how much will it cost to scale?

Technical architects and tech leads often get this question when scoping a project. They need to plan, pay for, and provision enough horsepower in case of success.

This virtual lollipop site is no mere trinket. This thing is going to make me a gazillionaire due to all the positive messages we all want to send each other! Traffic levels are going to spike as the word gets out. I had better have a good strategy of ensuring that the servers can handle the hefty load. I might add some caching layers, some load balancers, and I’ll design my database and database servers to be able to share the load without groaning from the demand to make and serve all these lollies.

Except… I don’t know how to do that stuff.

And I don’t know how much it would cost to add that infrastructure and keep it all humming. It’s complicated.

This is why I love to simplify my hosting by pre-rendering as much as I can.

Serving static pages is significantly simpler and cheaper than serving pages dynamically from a web server which needs to perform some logic to generate views on demand for every visitor.

Since we are working with lots of user generated content, it still makes sense to use a database, but I’m not going to manage that myself. Instead, I’ll choose one of the many database options available as a service. And I’ll talk to it via its APIs.

I might choose Firebase, or MongoDB, or any number of others. Chris compiled a few of these on an excellent site about serverless resources which is well worth exploring.

In this case, I selected Fauna to use as my data store. Fauna has a nice API for stashing and querying data. It is a no-SQL flavored data store and gives me just what I need.

https://fauna.com

Critically, Fauna have made an entire business out of providing database services. They have the deep domain knowledge that I’ll never have. By using a database-as-a-service provider, I just inherited an expert data service team for my project, complete with high availability infrastructure, capacity and compliance peace of mind, skilled support engineers, and rich documentation.

Such are the advantages of using a third-party service like this rather than rolling your own.

Architecture TL;DR

I often find myself doodling the logical flow of things when I’m working on a proof of concept. Here’s my doodle for this site:

And a little explanation:

  1. A user creates a new lollipop by completing a regular old HTML form.
  2. The new content is saved in a database, and its submission triggers a new site generation and deployment.
  3. Once the site deployment is complete, the new lollipop will be available on a unique URL. It will be a static page served very rapidly from the CDN with no dependency on a database query or a server.
  4. Until the site generation is complete, any new lollipops will not be available as static pages. Unsuccessful requests for lollipop pages fall back to a page which dynamically generates the lollipop page by querying the database API on the fly.

This kind of approach, which first assumes static/pre-generated assets, only then falling back to a dynamic render when a static view is not available was usefully described by Markus Schork of Unilever as “Static First” which I rather like.

In a little more detail

You could just dive into the code for this site, which is open source and available for you to explore, or we could talk some more.

You want to dig in a little further, and explore the implementation of this example? OK, I’ll explain in some more details:

  • Getting data from the database to generate each page
  • Posting data to a database API with a serverless function
  • Triggering a full site re-generation
  • Rendering on demand when pages are yet to be generated

Generating pages from a database

In a moment, we’ll talk about how we post data into the database, but first, let’s assume that there are some entries in the database already. We are going to want to generate a site which includes a page for each and every one of those.

Static site generators are great at this. They chomp through data, apply it to templates, and output HTML files ready to be served. We could use any generator for this example. I chose Eleventy due to it’s relative simplicity and the speed of its site generation.

To feed Eleventy some data, we have a number of options. One is to give it some JavaScript which returns structured data. This is perfect for querying a database API.

Our Eleventy data file will look something like this:

// Set up a connection with the Fauna database. // Use an environment variable to authenticate // and get access to the database. const faunadb = require('faunadb'); const q = faunadb.query; const client = new faunadb.Client({   secret: process.env.FAUNADB_SERVER_SECRET });  module.exports = () => {   return new Promise((resolve, reject) => {     // get the most recent 100,000 entries (for the sake of our example)     client.query(       q.Paginate(q.Match(q.Ref("indexes/all_lollies")),{size:100000})     ).then((response) => {       // get all data for each entry       const lollies = response.data;       const getAllDataQuery = lollies.map((ref) => {         return q.Get(ref);       });       return client.query(getAllDataQuery).then((ret) => {         // send the data back to Eleventy for use in the site build         resolve(ret);       });     }).catch((error) => {       console.log("error", error);       reject(error);     });   }) }

I named this file lollies.js which will make all the data it returns available to Eleventy in a collection called lollies.

We can now use that data in our templates. If you’d like to see the code which takes that and generates a page for each item, you can see it in the code repository.

Submitting and storing data without a server

When we create a new lolly page we need to capture user content in the database so that it can be used to populate a page at a given URL in the future. For this, we are using a traditional HTML form which posts data to a suitable form handler.

The form looks something like this (or see the full code in the repo):

<form name="new-lolly" action="/new" method="POST">    <!-- Default "flavors": 3 bands of colors with color pickers -->   <input type="color" id="flavourTop" name="flavourTop" value="#d52358" />   <input type="color" id="flavourMiddle" name="flavourMiddle" value="#e95946" />   <input type="color" id="flavourBottom" name="flavourBottom" value="#deaa43" />    <!-- Message fields -->   <label for="recipientName">To</label>   <input type="text" id="recipientName" name="recipientName" />    <label for="message">Say something nice</label>   <textarea name="message" id="message" cols="30" rows="10"></textarea>    <label for="sendersName">From</label>   <input type="text" id="sendersName" name="sendersName" />    <!-- A descriptive submit button -->   <input type="submit" value="Freeze this lolly and get a link">  </form>

We have no web servers in our hosting scenario, so we will need to devise somewhere to handle the HTTP posts being submitted from this form. This is a perfect use case for a serverless function. I’m using Netlify Functions for this. You could use AWS Lambda, Google Cloud, or Azure Functions if you prefer, but I like the simplicity of the workflow with Netlify Functions, and the fact that this will keep my serverless API and my UI all together in one code repository.

It is good practice to avoid leaking back-end implementation details into your front-end. A clear separation helps to keep things more portable and tidy. Take a look at the action attribute of the form element above. It posts data to a path on my site called /new which doesn’t really hint at what service this will be talking to.

We can use redirects to route that to any service we like. I’ll send it to a serverless function which I’ll be provisioning as part of this project, but it could easily be customized to send the data elsewhere if we wished. Netlify gives us a simple and highly optimized redirects engine which directs our traffic out at the CDN level, so users are very quickly routed to the correct place.

The redirect rule below (which lives in my project’s netlify.toml file) will proxy requests to /new through to a serverless function hosted by Netlify Functions called newLolly.js.

# resolve the "new" URL to a function [[redirects]]   from = "/new"   to = "/.netlify/functions/newLolly"   status = 200

Let’s look at that serverless function which:

  • stores the new data in the database,
  • creates a new URL for the new page and
  • redirects the user to the newly created page so that they can see the result.

First, we’ll require the various utilities we’ll need to parse the form data, connect to the Fauna database and create readably short unique IDs for new lollies.

const faunadb = require('faunadb');          // For accessing FaunaDB const shortid = require('shortid');          // Generate short unique URLs const querystring = require('querystring');  // Help us parse the form data  // First we set up a new connection with our database. // An environment variable helps us connect securely // to the correct database. const q = faunadb.query const client = new faunadb.Client({   secret: process.env.FAUNADB_SERVER_SECRET })

Now we’ll add some code to the handle requests to the serverless function. The handler function will parse the request to get the data we need from the form submission, then generate a unique ID for the new lolly, and then create it as a new record in the database.

// Handle requests to our serverless function exports.handler = (event, context, callback) => {    // get the form data   const data = querystring.parse(event.body);   // add a unique path id. And make a note of it - we'll send the user to it later   const uniquePath = shortid.generate();   data.lollyPath = uniquePath;    // assemble the data ready to send to our database   const lolly = {     data: data   };    // Create the lolly entry in the fauna db   client.query(q.Create(q.Ref('classes/lollies'), lolly))     .then((response) => {       // Success! Redirect the user to the unique URL for this new lolly page       return callback(null, {         statusCode: 302,         headers: {           Location: `/lolly/$ {uniquePath}`,         }       });     }).catch((error) => {       console.log('error', error);       // Error! Return the error with statusCode 400       return callback(null, {         statusCode: 400,         body: JSON.stringify(error)       });     });  }

Let’s check our progress. We have a way to create new lolly pages in the database. And we’ve got an automated build which generates a page for every one of our lollies.

To ensure that there is a complete set of pre-generated pages for every lolly, we should trigger a rebuild whenever a new one is successfully added to the database. That is delightfully simple to do. Our build is already automated thanks to our static site generator. We just need a way to trigger it. With Netlify, we can define as many build hooks as we like. They are webhooks which will rebuild and deploy our site of they receive an HTTP POST request. Here’s the one I created in the site’s admin console in Netlify:

Netlify build hook

To regenerate the site, including a page for each lolly recorded in the database, we can make an HTTP POST request to this build hook as soon as we have saved our new data to the database.

This is the code to do that:

const axios = require('axios'); // Simplify making HTTP POST requests  // Trigger a new build to freeze this lolly forever axios.post('https://api.netlify.com/build_hooks/5d46fa20da4a1b70XXXXXXXXX') .then(function (response) {   // Report back in the serverless function's logs   console.log(response); }) .catch(function (error) {   // Describe any errors in the serverless function's logs   console.log(error); });

You can see it in context, added to the success handler for the database insertion in the full code.

This is all great if we are happy to wait for the build and deployment to complete before we share the URL of our new lolly with its intended recipient. But we are not a patient lot, and when we get that nice new URL for the lolly we just created, we’ll want to share it right away.

Sadly, if we hit that URL before the site has finished regenerating to include the new page, we’ll get a 404. But happily, we can use that 404 to our advantage.

Optimistic URL routing and serverless fallbacks

With custom 404 routing, we can choose to send every failed request for a lolly page to a page which will can look for the lolly data directly in the database. We could do that in with client-side JavaScript if we wanted, but even better would be to generate a ready-to-view page dynamically from a serverless function.

Here’s how:

Firstly, we need to tell all those hopeful requests for a lolly page that come back empty to go instead to our serverless function. We do that with another rule in our Netlify redirects configuration:

# unfound lollies should proxy to the API directly [[redirects]]   from = "/lolly/*"   to = "/.netlify/functions/showLolly?id=:splat"   status = 302

This rule will only be applied if the request for a lolly page did not find a static page ready to be served. It creates a temporary redirect (HTTP 302) to our serverless function, which looks something like this:

const faunadb = require('faunadb');                  // For accessing FaunaDB const pageTemplate = require('./lollyTemplate.js');  // A JS template litereal   // setup and auth the Fauna DB client const q = faunadb.query; const client = new faunadb.Client({   secret: process.env.FAUNADB_SERVER_SECRET });  exports.handler = (event, context, callback) => {    // get the lolly ID from the request   const path = event.queryStringParameters.id.replace("/", "");    // find the lolly data in the DB   client.query(     q.Get(q.Match(q.Index("lolly_by_path"), path))   ).then((response) => {     // if found return a view     return callback(null, {       statusCode: 200,       body: pageTemplate(response.data)     });    }).catch((error) => {     // not found or an error, send the sad user to the generic error page     console.log('Error:', error);     return callback(null, {       body: JSON.stringify(error),       statusCode: 301,       headers: {         Location: `/melted/index.html`,       }     });   }); }

If a request for any other page (not within the /lolly/ path of the site) should 404, we won’t send that request to our serverless function to check for a lolly. We can just send the user directly to a 404 page. Our netlify.toml config lets us define as many level of 404 routing as we’d like, by adding fallback rules further down in the file. The first successful match in the file will be honored.

# unfound lollies should proxy to the API directly [[redirects]]   from = "/lolly/*"   to = "/.netlify/functions/showLolly?id=:splat"   status = 302  # Real 404s can just go directly here: [[redirects]]   from = "/*"   to = "/melted/index.html"   status = 404

And we’re done! We’ve now got a site which is static first, and which will try to render content on the fly with a serverless function if a URL has not yet been generated as a static file.

Pretty snappy!

Supporting larger scale

Our technique of triggering a build to regenerate the lollipop pages every single time a new entry is created might not be optimal forever. While it’s true that the automation of the build means it is trivial to redeploy the site, we might want to start throttling and optimizing things when we start to get very popular. (Which can only be a matter of time, right?)

That’s fine. Here are a couple of things to consider when we have very many pages to create, and more frequent additions to the database:

  • Instead of triggering a rebuild for each new entry, we could rebuild the site as a scheduled job. Perhaps this could happen once an hour or once a day.
  • If building once per day, we might decide to only generate the pages for new lollies submitted in the last day, and cache the pages generated each day for future use. This kind of logic in the build would help us support massive numbers of lolly pages without the build getting prohibitively long. But I’ll not go into intra-build caching here. If you are curious, you could ask about it over in the Netlify Community forum.

By combining both static, pre-generated assets, with serverless fallbacks which give dynamic rendering, we can satisfy a surprisingly broad set of use cases — all while avoiding the need to provision and maintain lots of dynamic infrastructure.

What other use cases might you be able to satisfy with this “static first” approach?

The post Static First: Pre-Generated JAMstack Sites with Serverless Rendering as a Fallback appeared first on CSS-Tricks.

CSS-Tricks

, , , , , , ,

React’s Experimental Suspense API Will Rock for Fallback UI During Data Fetches

Most web applications built today receive data from an API. When fetching that data, we have to take certain situations into consideration where the data might not have been received. Perhaps it was a lost connection. Maybe it was the endpoint was changed. Who knows. Whatever the issue, it’s the end user who winds up with a big bag of nothing on the front end.

So we ought to account for that!

The common way of handling this is to have something like an isLoading state in the app. The value of isLoading is dependent on the data we want to receive. For example, it could be a simple boolean where a returned true (meaning we’re still waiting on the data), we display a loading spinner to indicate that the app is churning. Otherwise, wee’ll show the data.

Oh god, no!
📷 Credit: Jian Wei

While this isn‘t entirely bad, the awesome folks working on React have implemented (and are continuing to work on) a baked-in solution to handle this using a feature called Suspense.

Suspense sorta does what its name implies

You may have guessed it from the name, but Suspense tells a component to hold off from rendering until a condition has been met. Just like we discussed with isLoading, the rendering of the data is postponed until the API fetches the data and isLoading is set to false. Think of it like a component is standing in an elevator waiting for the right floor before stepping out.

At the moment, Suspense can only be used to conditionally load components that use React.lazy() to render dynamically, without a page reload. So, say we have a map that takes a bit of time to load when the user selects a location. We can wrap that map component with Suspense and call something like the Apple beachball of death to display while we’re waiting on the map. then, once the map loads, we kick the ball away.

// Import the Map component const Map = React.lazy(() => import('./Map'));  function AwesomeComponent() [   return (     // Show the <Beachball> component until the <Map> is ready     <React.Suspense fallback={<Beachball />}>       <div>         <Map />       </div>     </React.Suspense>   ); }

Right on. Pretty straightforward so far, I hope.

But what if we want the fallback beachball, not for a component that has loaded, but when waiting for data to be returned from an API. Well, that’s a situation Suspense seems perfectly suited for, but unfortunately, does not handle that quite yet. But it will.

In the meantime, we can put an experimental feature called react-cache (the package previously known as simple-cache-provider) to demonstrate how Suspense ought to work with API fetching down the road.

Let’s use Suspense with API data anyway

OK, enough suspense (sorry, couldn‘t resist). Let’s get to a working example where we define and display a component as a fallback while we’re waiting for an API to spit data back at us.

Remember, react-cache is experimental. When I say experimental, I mean just that. Even the package description urges us to refrain from using it in production.

Here’s what we’re going to build: a list of users fetched from an API.

Get Source Code

Alright, let’s begin!

First, spin up a new project

Let’s start by generating a new React application using create-react-app.

## Could be any project name create-react-app csstricks-react-suspense

This will bootstrap your React application. Because the Suspense API is still a work in progress, we will make use of a different React version. Open the package.json file in the project’s root directory, edit the React and React-DOM version numbers, and add the simple-cache-provider package (we’ll look into that later). Here’s what that looks like:

"dependencies": {   "react": "16.4.0-alpha.0911da3",   "react-dom": "16.4.0-alpha.0911da3",   "simple-cache-provider": "0.3.0-alpha.0911da3" }

Install the packages by running yarn install.

In this tutorial, we will build the functionality to fetch data from an API. We can use the createResource() function from simple-cache-provider to do that in the src/fetcher.js file:

import { createResource } from 'simple-cache-provider';  const sleep = (duration) => {   return new Promise((resolve) => {     setTimeout(() => {       resolve()     }, duration)   }) }  const loadProfiles = createResource(async () => {   sleep(3000)   const res = await fetch(`https://randomuser.me/api/?results=15`);   return await res.json(); });  export default loadProfiles

So, here’s what’s happening there. The sleep() function blocks the execution context for a specific duration, which will be passed as an argument. The sleep() function is then called in the loadProfiles() function to stimulate a delay of three seconds (3,000ms). By using createResource() to make the API call, we either return the resolved value (which is the data we are expecting from the API) or throw a promise.

Next, we will create a higher-order component called withCache that enable caching on the component it wraps. We’ll do that in a new file called, creatively, withCache.js. Go ahead and place that in the project’s src directory.

import React from 'react'; import { SimpleCache } from 'simple-cache-provider';  const withCache = (Component) => {   return props => (     <SimpleCache.Consumer>       {cache => <Component cache={cache} {...props} />}     </SimpleCache.Consumer>   ); }  export default withCache;

This higher-order component uses SimpleCache from the simple-cache-provider package to enable the caching of a wrapped component. We’ll make use of this when we create our next component, I promise. In the meantime, create another new file in src called Profile.js — this is where we’ll map through the results we get from the API.

import React, { Fragment } from 'react'; import loadProfiles from './fetcher' import withCache from './withCache'  // Just a little styling const cardWidth = {   width: '20rem' }  const Profile = withCache((props) => {   const data = loadProfiles(props.cache);   return (     <Fragment>       {         data.results.map(item => (         <div key={item.login.uuid} className="card" style={cardWidth}>           <div>             <img src={item.picture.thumbnail} />           </div>             <p>{item.email}</p>           </div>         ))       }     </Fragment>   ) });  export default Profile

What we have here is a Profile component that’s wrapped in withCache the higher-order component we created earlier. Now, whatever we get back from the API (which is the resolved promise) is saved as a value to the data variable, which we’ve defined as the props for the profile data that will be passed to the components with cache (props.cache).

To handle the loading state of the app before the data is returned from the API, we’ll implement a placeholder component which will render before the API responds with the data we want.

Here’s what we want the placeholder to do: render a fallback UI (which can be a loading spinner, beach ball or what have you) before the API responds, and when the API responds, show the data. We also want to implement a delay (delayMs ) which will come in handy for scenarios where there’s almost no need to show the loading spinner. For example; if the data comes back in less than two seconds, then maybe a loader is a bit silly.

The placeholder component will look like this;

const Placeholder = ({ delayMs, fallback, children }) => {   return (     <Timeout ms={delayMs}>       {didTimeout => {         return didTimeout ? fallback : children;       }}     </Timeout>   ); }

delayMs, fallback and children will be passed to the Placeholder component from the App component which we will see shortly. The Timeout component returns a boolean value which we can use to either return the fallback UI or the children of the Placeholder component (the Profile component in this case).

Here’s the final markup of our App, piecing together all of the components we’ve covered, plus some decorative markup from Bootstrap to create a full page layout.

class App extends React.Component {   render() {     return (       <React.Fragment>         // Bootstrap Containers and Jumbotron              <div className="App container-fluid">           <div className="jumbotron">             <h1>CSS-Tricks React Suspense</h1>           </div>           <div className="container">             <div>               // Placeholder contains Suspense and wraps what needs the fallback UI               <Placeholder                 delayMs={1000}                 fallback={                   <div className="row">                     <div className="col-md">                       <div className="div__loading">                         <Loader />                       </div>                     </div>                   </div>                 }               >                 <div className="row">                   // This is what will render once the data loads                   <Profile />                 </div>               </Placeholder>             </div>           </div>         </div>       </React.Fragment>     );   } }

That’s a wrap

Pretty neat, right? It’s great that we’re in the process of getting true fallback UI support right out of the React box, without crafty tricks or extra libraries. Totally makes sense given that React is designed to manage states and loading being a common state to handle.

Remember, as awesome as Suspense is (and it is really awesome), it is important to note that it’s still in experimental phase, making it impractical in a production application. But, since there are ways to put it to use today, we can still play around with it in a development environment all we want, so experiment away!

Folks who have been working on and with Suspense have been writing up their thoughts and experience. Here are a few worth checking out:

The post React’s Experimental Suspense API Will Rock for Fallback UI During Data Fetches appeared first on CSS-Tricks.

CSS-Tricks

, , , , , , ,
[Top]