Search

React Data Layer – Part 7: Offline Writes

Josh Justice

12 min read

May 26, 2019

React Data Layer – Part 7: Offline Writes

Editor’s Note: The API used by this blog series is no longer available. As a result, you will not be able to run this application locally, but you can read the code and explanations to see how to build a robust real-world frontend app data layer.

This post is the seventh part of an 8-part series going in-depth into how to build a robust real-world frontend app data layer. See the previous parts here:

Caching read data while offline introduces some complexity into an app, and caching writes adds even more complexity. With read data, the worst that can happen is data is out of date. With cached writes, if there is an error, data a user enters can be lost. Ensuring this doesn’t happen is a significant cost, so you want to make sure it’s worth it for your app.

With that said, let’s see how we can implement some basic cached writes. The first thing we can do is keep track of failed adds and retry them.

If you like, you can download the app as of the end of the post.

Retrying Adds

First, let’s keep track of a queue of games that have not yet been saved. Make the following changes to store/games/actions.js:

 export const STORE_GAMES = 'STORE_GAMES';
 export const ADD_GAME = 'ADD_GAME';
+export const QUEUE_GAME_TO_ADD = 'QUEUE_GAME_TO_ADD';
 export const START_LOADING = 'START_LOADING';
 export const RECORD_ERROR = 'RECORD_ERROR';
...
 export const addGame = (title) => async (dispatch) => {
   const game = {
     type: 'games',
     attributes: { title },
   };
+  dispatch({
+    type: QUEUE_GAME_TO_ADD,
+    game,
+  });
   const { data: responseBody } = await api.post('/games', { data: game });
   dispatch({
     type: ADD_GAME,
     game: responseBody.data,
   });
 };

Import the new action into store/games/reducers.js:

 import {
+  QUEUE_GAME_TO_ADD,
   ADD_GAME,
   STORE_GAMES,
Next, let’s create a gamesToAdd reducer:export function gamesToAdd(state = [], action) {
  switch (action.type) {
    case QUEUE_GAME_TO_ADD:
      return [action.game, ...state];
    case ADD_GAME:
      return state.filter((game) => (
        game.attributes.title !== action.game.attributes.title
      ));
    default:
      return state;
  }
}

And add it to combineReducers:

 export default combineReducers({
   games,
+  gamesToAdd,
   loading,
   error,
 });

We start with an empty array of games to add. When QUEUE_GAME_TO_ADD is dispatched, we add the passed-in game to the array. After the game is saved to the server, when ADD_GAME is dispatched, we remove that game from the array again, filtering it by title. This implementation actually has a problem when there are two games with the same title; we’ll look at a way to fix that soon.

With these changes, the games are queued, but we don’t actually do anything with them yet. But now we can take advantage of these queued games to re-attempt any failed adds when the app restarts. Let’s add an action creator function to do so:

export const addQueuedGames = () => async (dispatch, getState) => {
  const { games: { gamesToAdd } } = getState();
  for (const game of gamesToAdd) {
    const { data: responseBody } = await api.post('/games', { data: game });
    dispatch({
      type: ADD_GAME,
      game: responseBody.data,
    });
  }
};

This action creator function retrieves the list of gamesToAdd, then attempts to post each one to the server again. If it succeeds, ADD_GAME is dispatched, which adds the game into the main game list, and removes it from the gamesToAdd so it won’t be attempted again. Note that we only send one post at a time, waiting for it to complete before beginning the next one. This is intentional to avoid overwhelming the server with lots of requests at once.

Now we just need to map the addQueuedGames action creator to the GameList:

 import {
   loadGames,
   addGame,
+  addQueuedGames,
 } from 'store/games/actions';
...
 const mapDispatchToProps = {
   loadGames,
   addGame,
+  addQueuedGames,
   logOut,
 };
And call addQueuedGames() as an effect in GameList:   useEffect(() => {
-    loadGames();
+    loadGames()
+      .then(addQueuedGames);
   }, []);

We wait for loadGames() to resolve before we call addQueuedGames(). This is because both of them affect the games state: loadGames() overwrites the list, and addQueuedGames() adds records to it. Depending on what order the responses come back in, we could get nondeterministic behavior.

So what order do we want the operations to happen in? We want to ensure that after games is overwritten by loadGames() that we start to addQueuedGames(), because that ensures they’ll be added into the most up-to-date games list.

Reload the app then go Offline, then add a new video game. Confirm in the Network tab that the POST failed. Go back Online, then reload the page. Check the Network tab to confirm there’s a successful POST, and check the video game list to confirm you see the game in the list. You can reload one more time to make sure the game is successfully pulled down from the server.

Remove Duplication

Before we move on to any more functionality, there’s a refactoring we can do here. We have some duplicated logic in addGame() and addQueuedGames(): both functions post to the server and then dispatch ADD_GAME. We want to ensure that the same logic is run in both cases, so to prevent them from getting out of sync, let’s extract that duplicated code into a helper function:

 export const addGame = (title) => async (dispatch) => {
   const game = {
     type: 'games',
     attributes: { title },
   };
   dispatch({
     type: QUEUE_GAME_TO_ADD,
     game,
   });
-  const { data: responseBody } = await api.post('/games', { data: game });
-  dispatch({
-    type: ADD_GAME,
-    game: responseBody.data,
-  });
+  await persistGame(dispatch, game);
 };

 export const addQueuedGames = () => async (dispatch, getState) => {
   const { games: { gamesToAdd } } = getState();
   for (const game of gamesToAdd) {
-    const { data: responseBody } = await api.post('/games', { data: game });
-    dispatch({
-      type: ADD_GAME,
-      game: responseBody.data,
-    });
+    await persistGame(dispatch, game);
   }
 };
+
+const persistGame = async (dispatch, game) => {
+  const { data: responseBody } = await api.post('/games', { data: game });
+  dispatch({
+    type: ADD_GAME,
+    game: responseBody.data,
+  });
+};

We don’t need to export this function from actions.js because no other part of our app needs to call it directly.

Tracking Identity Locally

Let’s revisit the bug we mentioned earlier: if two games are added with the same name, they won’t be removed correctly from the gamesToAdd list. To remove one game at a time, we need some kind of unique identifier for each one. Usually the server is in charge of providing unique IDs, but in this case these are records that haven’t yet been saved to the server.

Instead, what we can do is assign a unique ID for local tracking purposes. To do this, add the uuid library:

$ yarn add uuid

Next, when we add a game, generate a UUID and include it in the payload of the QUEUE_GAME_TO_ADD and ADD_GAME actions:

+import generateUuid from 'uuid/v4';
 import api from '../api';
...
 export const addGame = (title) => async (dispatch) => {
+  const uuid = generateUuid();
   const game = {
     type: 'games',
     attributes: { title },
   };

   dispatch({
     type: QUEUE_GAME_TO_ADD,
+    uuid,
     game,
   });
-  await persistGame(dispatch, game);
+  await persistGame(dispatch, game, uuid);
 };
...
-const persistGame = async (dispatch, game) => {
+const persistGame = async (dispatch, game, uuid) => {
   const { data: responseBody } = await api.post('/games', { data: game });
   dispatch({
     type: ADD_GAME,
+    uuid,
     game: responseBody.data,
   });
 };

We need to change addQueuedGames too, but let’s change the gamesToAdd reducer first so it’s easier to follow:

-export function gamesToAdd(state = [], action) {
+export function gamesToAdd(state = {}, action) {
  switch (action.type) {
    case QUEUE_GAME_TO_ADD:
-      return [action.game, ...state];
+      return {
+        [action.uuid]: action.game,
+        ...state,
+      };
    case ADD_GAME:
-      return state.filter((game) => (
-        game.attributes.title !== action.game.attributes.title
-      ));
+      const {
+        [action.uuid]: gameToRemove,
+        ...gamesToKeep
+      } = state;
+      return gamesToKeep;
    default:
      return state;
  }
}

We change gamesToAdd’s state to be an object instead of an array. When we QUEUE_GAME_TO_ADD, we add the game under the key that is the UUID we provide. When we ADD_GAME, we remove the game by looking it up by the UUID we provide. We are using “object rest” syntax here, which is less common than “object spread” syntax. gamesToKeep will be assigned an object containing all properties in the object other than the action.uuid property.

Now, back to updating addQueuedGames:

 export const addQueuedGames = () => async (dispatch, getState) => {
   const { gamesToAdd } = getState();
-  for (const game of gamesToAdd) {
-    await persistGame(dispatch, game);
+  for (const uuid in gamesToAdd) {
+    const game = gamesToAdd[uuid];
+    await persistGame(dispatch, game, uuid);
   }
 };

Now that gamesToAdd is an object keyed off of UUIDs, instead of using for/of to loop through array values, we use for/in to loop through object keys. Then we retrieve the game, and pass both game and uuid to persistGame.

Because we are now looking up games by UUID instead of name, we should be able to handle two games with titles just fine.

Here, we are using UUIDs as a temporary local identifier to track games that have not yet been persisted to the server. This works regardless of what kind of ID the server uses. But UUIDs can also help us create an even better user experience: an optimistic UI.

What’s Optimistic UI?

So far our UI is what is called a “pessimistic UI”: it doesn’t show records in the list until the server responds with a success. In this post, we’ve added some offline robustness to our writes, but a record still only shows up in the list when it’s been successfully saved to the server.

An alternative approach is called “optimistic UI”: you update the UI right away, assuming the request will succeed. This gives a greater sense of perceived performance to the app, and can be helpful if the user needs to work on the added data right away.

We could implement an optimistic UI purely at the UI layer by retrieving data both from the games and gamesToAdd state properties and merging them into one array to display. This would get a little complex in our case because gamesToAdd is an object rather than an array; some data transformation would need to be done to combine them. This is certainly doable, though, and it might be the right approach for your app.

There’s another way we could implement optimistic UI: by actually adding the games to the games array right away, in addition to adding them to gamesToAdd. But if we want to do this, we have to deal with a problem related to our use of IDs.

Right now our render method assumes all games have an ID, but new games don’t–not until the server response returns with the ID it’s assigned. Many apps work this way because they use auto-incrementing numeric IDs. When you’re using an auto-incrementing numeric ID, you have to let the server do the assignment, because otherwise two different clients could create new records with the same numeric ID. Because of this, auto-incrementing IDs present inherent challenges to treating “unsaved” and “saved” records the same.

A good way to take away this difference at a deep level is to use the same UUIDs we’ve been using locally on the server side as well. When a record’s ID field is a UUID, you can create a UUID on the client side and save it up to the server, and you can be assured that there is almost no chance that it will conflict. As a result, you will already know the record’s ID before you receive the server response, so presenting an optimistic UI becomes a lot simpler.

It’s important to note, though, that at this point we move beyond the realm of things you can do purely on the client side. If you don’t have access to change your backend, or if it’s an established system that can’t easily have its IDs changed, you may not be able to take this step.

Conveniently, our sandboxapi backend has been using UUIDs for its IDs all along! If you create a record without providing an ID the server will auto-assign a UUID to it, but if you create a record and provide the server with an ID, it will use that ID for the saved record. So let’s do it!

We’ll take two steps. First, we’ll save the client-created UUID in the game record we send up to the server. Once that’s working, we can rearrange our code to get an optimistic UI.

Client-Generated UUID

Let’s begin by assigning the UUID into the game before we save it to the server. Make this change in the addGame() action creator:

   const game = {
     type: 'games',
+    id: uuid,
     attributes: { title },
   };

Now that the UUID is available on the game, our gamesToAdd reducer can retrieve it from the game instead of needing a second uuid parameter on the action. Let’s update it to do so:

 export function gamesToAdd(state = {}, action) {
   switch (action.type) {
     case QUEUE_GAME_TO_ADD:
       return {
-        [action.uuid]: action.game,
+        [action.game.id]: action.game,
         ...state,
       };
     case ADD_GAME:
       const {
-        [action.uuid]: gameToRemove,
+        [action.game.id]: gameToRemove,
         ...gamesToKeep
       } = state;
       return gamesToKeep;
     default:
       return state;
   }
 }

Now we’re no longer using the action.uuid property, so we can remove it from our dispatch() calls, as well as from the persistGame() function:

 export const addGame = (title) => async (dispatch) => {
   const uuid = generateUuid();
   const game = {
     type: 'games',
     id: uuid,
     attributes: { title },
   };
   dispatch({
     type: QUEUE_GAME_TO_ADD,
-    uuid,
     game,
   });
-  await persistGame(dispatch, game, uuid);
+  await persistGame(dispatch, game);
 };

 export const addQueuedGames = () => async (dispatch, getState) => {
   const { gamesToAdd } = getState();
   for (const uuid in gamesToAdd) {
     const game = gamesToAdd[uuid];
-    await persistGame(dispatch, game, uuid);
+    await persistGame(dispatch, game);
   }
 };

-const persistGame = async (dispatch, game, uuid) => {
+const persistGame = async (dispatch, game) => {
   const { data: responseBody } = await api.post('/games', { data: game });
   dispatch({
     type: ADD_GAME,
-    uuid,
     game: responseBody.data,
   });
 };

Rerun the app and check the Network tab for POST requests; you should now see a UUID going up to the server.

Optimistic UI

Now we’re ready to update our UI to optimistically add the record to the list before the response is received.

First, instead of dispatching QUEUE_GAME_TO_ADD before we send our data to the server and ADD_GAME afterward, we want to ADD_GAME to the local list immediately, before the server request. So let’s add a new GAME_PERSISTED action, which will be dispatched after the request to the server completes.

 export const STORE_GAMES = 'STORE_GAMES';
 export const ADD_GAME = 'ADD_GAME';
 export const QUEUE_GAME_TO_ADD = 'QUEUE_GAME_TO_ADD';
+export const GAME_PERSISTED = 'GAME_PERSISTED';
 export const START_LOADING = 'START_LOADING';
 export const RECORD_ERROR = 'RECORD_ERROR';

So when exactly do these actions need to be dispatched? Well, within the addGame action creator, we want to continue running QUEUE_GAME_TO_ADD to enqueue it, then ADD_GAME to add it to the games list, then GAME_PERSISTED if the server request returns successfully.

How about for addQueuedGames? This will run after the app reloads and the data has been re-downloaded from the server. So we will need to re-ADD_GAME to get our game back into the games list, and then GAME_PERSISTED upon success.

So the ADD_GAME and GAME_PERSISTED actions should be dispatched both for addGame() and addQueuedGames(). To accomplish this, we should put both of these actions in the persistGame() helper function that is called from both addGame() and addQueuedGames():

 const persistGame = async (dispatch, game, uuid) => {
+  dispatch({
+    type: ADD_GAME,
+    game,
+  });
   const { data: responseBody } = await api.post('/games', { data: game });
   dispatch({
-    type: ADD_GAME,
+    type: GAME_PERSISTED,
     game: responseBody.data,
   });
 };

Next, let’s update the gamesToAdd reducer to react to the updated action. Now it’s no longer when we dispatch ADD_GAME that we want to remove the game, but when we dispatch GAME_PERSISTED:

 import {
   START_LOADING,
   RECORD_ERROR,
   QUEUE_GAME_TO_ADD,
+  GAME_PERSISTED,
   ADD_GAME,
   STORE_GAMES,
 } from './actions';
...
 export function gamesToAdd(state = {}, action) {
   switch (action.type) {
     case QUEUE_GAME_TO_ADD:
       return {
         ...state,
         [action.uuid]: action.game,
       };
-    case ADD_GAME:
+    case GAME_PERSISTED:
       const { [action.uuid]: gameToRemove, ...gamesToKeep } = state;
       return gamesToKeep;
     default:
       return state;
   }
 }

Reload the app and you should see the game titles as usual. Go Offline and add a new video game. You’ll see it added to the list right away. Go back online and reload the app. You should see requests go out for the games to be persisted to the server.

Overly-Optimistic UI?

This is an optimistic UI because the user sees the results of their actions right away, before the server returns. In a sense, though, this UI is a lie: these records have not yet been saved to the server permanently, but it looks as though they have been. This could cause users to assume that data is permanently saved that isn’t yet, leading to lost data.

The best way to handle prevent confusion like this is to design your UI to unobtrusively let the user know the state of their records, warning them if they haven’t yet been permanently persisted. This might be an icon next to each record, or even a single “unsaved data” icon in a corner of the screen. It would be good to inform users why that data isn’t saved yet, and to provide them an option to manually dispatch addQueuedGames.

What’s Next?

We’ve seen how to cache record creation to send them to the server. It took some work, including changing the values we use for primary keys, but it was doable. If you want more advanced offline and sync capabilities than this, though, you may want to use an off-the-shelf tool. In the final post we’ll do a survey of a few such tools.

Click here for Part 8

Josh Justice

Author Big Nerd Ranch

Josh Justice has worked as a developer since 2004 across backend, frontend, and native mobile platforms. Josh values creating maintainable systems via testing, refactoring, and evolutionary design, and mentoring others to do the same. He currently serves as the Web Platform Lead at Big Nerd Ranch.

Speak with a Nerd

Schedule a call today! Our team of Nerds are ready to help

Let's Talk

Related Posts

We are ready to discuss your needs.

Not applicable? Click here to schedule a call.

Stay in Touch WITH Big Nerd Ranch News