Data Transformation

[…]

3. using the select option

v3 introduced built-in selectors, which can also be used to transform data:

/* select-transformation */

export const useTodosQuery = () =>
  useQuery(['todos'], fetchTodos, {
    select: (data) => data.map((todo) => todo.name.toUpperCase()),
  })

selectors will only be called if data exists, so you don't have to care about undefined here. Selectors like the one above will also run on every render, because the functional identity changes (it's an inline function). If your transformation is expensive, you can memoize it either with useCallback, or by extracting it to a stable function reference:

/* select-memoizations */

const transformTodoNames = (data: Todos) =>
  data.map((todo) => todo.name.toUpperCase())
export const useTodosQuery = () =>
  useQuery(['todos'], fetchTodos, {
    // ✅ uses a stable function reference
    select: transformTodoNames,
  })

export const useTodosQuery = () =>
  useQuery(['todos'], fetchTodos, {
    // ✅ memoizes with useCallback
    select: React.useCallback(
      (data: Todos) => data.map((todo) => todo.name.toUpperCase()),
      []
    ),
  })

Further, the select option can also be used to subscribe to only parts of the data. This is what makes this approach truly unique. Consider the following example:

/* select-partial-subscriptions */

export const useTodosQuery = (select) =>
  useQuery(['todos'], fetchTodos, { select })

export const useTodosCount = () => useTodosQuery((data) => data.length)

export const useTodo = (id) =>
  useTodosQuery((data) =>
    data.find((todo) => todo.id === id))

Here, we've created a useSelector like API by passing a custom selector to our useTodosQuery. The custom hooks still works like before, as select will be undefined if you don't pass it, so the whole state will be returned.

But if you pass a selector, you are now only subscribed to the result of the selector function. This is quite powerful, because it means that even if we update the name of a todo, our component that only subscribes to the count via useTodosCount will not rerender. The count hasn't changed, so react-query can choose to not inform this observer about the update 🥳 (Please note that this is a bit simplified here and technically not entirely true - I will talk in more detail about render optimizations in Part 3).

• 🟢 best optimizations
• 🟢 allows for partial subscriptions
• 🟡 structure can be different for every observer
• 🟡 structural sharing is performed twice (I will also talk about this in more detail in Part 3)

Data Transformation

[...]

2. In the render function

As advised in Part 1, if you create custom hooks, you can easily do transformations there:

/* render-transformation */

const fetchTodos = async (): Promise<Todos> => {
  const response = await axios.get('todos')
  return response.data
}

export const useTodosQuery = () => {
  const queryInfo = useQuery(['todos'], fetchTodos)
  return {
    ...queryInfo,
    data: queryInfo.data?.map((todo) =>
      todo.name.toUpperCase()),
  }
}

As it stands, this will not only run every time your fetch function runs, but actually on every render (even those that do not involve data fetching). This is likely not a problem at all, but if it is, you can optimize with useMemo. Be careful to define your dependencies as narrow as possible. data inside the queryInfo will be referentially stable unless something really changed (in which case you want to recompute your transformation), but the queryInfo itself will not. If you add queryInfo as your dependency, the transformation will again run on every render:

/* useMemo-dependencies */

export const useTodosQuery = () => {
  const queryInfo = useQuery(['todos'], fetchTodos)
  return {
    ...queryInfo,
    // 🚨 don't do this - the useMemo does nothing at all here!
    data: React.useMemo(
      () => queryInfo.data?.map((todo) => todo.name.toUpperCase()),
      [queryInfo]
    ),
    // ✅ correctly memoizes by queryInfo.data
    data: React.useMemo(
      () => queryInfo.data?.map((todo) => todo.name.toUpperCase()),
      [queryInfo.data]
    ),
  }
}

Especially if you have additional logic in your custom hook to combine with your data transformation, this is a good option. Be aware that data can be potentially undefined, so use optional chaining when working with it.

• 🟢 optimizable via useMemo
• 🟡 exact structure cannot be inspected in the devtools
• 🔴 a bit more convoluted syntax
• 🔴 data can be potentially undefined

Increasing StaleTime

React Query comes with a default staleTime of zero. This means that every query will be immediately considered as stale, which means it will refetch when a new subscriber mounts or when the user refocuses the window. It is aimed to keep your data as up-to-date as necessary.

This goal overlaps a lot with WebSockets, which update your data in real-time. Why would I need to refetch at all if I just manually invalidated because the server just told me to do so via a dedicated message?

So if you update all your data via websockets anyways, consider setting a high staleTime. In my example, I just used Infinity. This means the data will be fetched initially via useQuery, and then always come from the cache. Refetching only happens via the explicit query invalidation.

You can best achieve this by setting global query defaults when creating the QueryClient:

const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      staleTime: Infinity,
    },
  },
})

Web Workers

As of webpack 5, you can use Web Workers without worker-loader.

Syntax

new Worker(new URL('./worker.js', import.meta.url));

Currently starting to read https://github.com/substack/stream-handbook, maybe I just need a deeper understanding of streams

According to @xander76 on Twitter the code would have to use a Transform stream, which looks something like this:

let cacheEntry = "";
renderToNodeStream(<Frontend/>)
  .pipe(new Transform({
    transform(chunk, enc, callback) {
      cacheEntry += chunk; callback(chunk);
    },
    flush(callback) {
      redis.set(req.path, cacheEntry);
    }
  })
  .pipe(res);

/* ScrollToTop.js */

import { useEffect } from 'react';
import { withRouter } from 'react-router-dom';

function ScrollToTop({ history }) {
  useEffect(() => {
    const unlisten = history.listen(() => {
      window.scrollTo(0, 0);
    });
    return () => {
      unlisten();
    }
  }, []);

  return (null);
}

export default withRouter(ScrollToTop);

Globalize Selectors if Needed​

[...]

We refer to this pattern as "globalizing" selectors. A "globalized" selector is one that accepts the Redux root state as an argument, and knows how to find the relevant slice of state to perform the real logic. A "localized" selector is one that expects just a piece of the state as an argument, without knowing or caring where that is in the root state:

// "Globalized" - accepts root state, knows to find data at `state.todos`
const selectAllTodosCompletedGlobalized = state =>
  state.todos.every(todo => todo.completed)

// "Localized" - only accepts `todos` as argument, doesn't know where that came from
const selectAllTodosCompletedLocalized = todos =>
  todos.every(todo => todo.completed)

"Localized" selectors can be turned into "globalized" selectors by wrapping them in a function that knows how to retrieve the right slice of state and pass it onwards.

Redux Toolkit's createEntityAdapter API is an example of this pattern. If you call todosAdapter.getSelectors(), with no argument, it returns a set of "localized" selectors that expect the entity slice state as their argument. If you call todosAdapter.getSelectors(state => state.todos), it returns a set of "globalized" selectors that expect to be called with the Redux root state as their argument.

There may also be other benefits to having "localized" versions of selectors as well. For example, say we have an advanced scenario of keeping multiple copies of createEntityAdapter data nested in the store, such as a chatRoomsAdapter that tracks rooms, and each room definition then has a chatMessagesAdapter state to store the messages. We can't directly look up the messages for each room - we first have to retrieve the room object, then select the messages out of that. This is easier if we have a set of "localized" selectors for the messages.

Balance Selector Usage​

[...]

Similarly, don't make every single selector memoized!. Memoization is only needed if you are truly deriving results, and if the derived results would likely create new references every time. A selector function that does a direct lookup and return of a value should be a plain function, not memoized.

Some examples of when and when not to memoize:

// ❌ DO NOT memoize: will always return a consistent reference
const selectTodos = state => state.todos
const selectNestedValue = state => state.some.deeply.nested.field
const selectTodoById = (state, todoId) => state.todos[todoId]

// ❌ DO NOT memoize: deriving data, but will return a consistent result
const selectItemsTotal = state => {
  return state.items.reduce((result, item) => {
    return result + item.total
  }, 0)
}
const selectAllCompleted = state => state.todos.every(todo => todo.completed)

// ✅ SHOULD memoize: returns new references when called
const selectTodoDescriptions = state => state.todos.map(todo => todo.text)

Creating Unique Selector Instances​

There are many cases where a selector function needs to be reused across multiple components. If the components will all be calling the selector with different arguments, it will break memoization - the selector never sees the same arguments multiple times in a row, and thus can never return a cached value.

The standard approach here is to create a unique instance of a memoized selector in the component, and then use that with useSelector. That allows each component to consistently pass the same arguments to its own selector instance, and that selector can correctly memoize the results.

For function components, this is normally done with useMemo or useCallback:

import { makeSelectItemsByCategory } from './categoriesSlice'

function CategoryList({ category }) {
  // Create a new memoized selector, for each component instance, on mount
  const selectItemsByCategory = useMemo(makeSelectItemsByCategory, [])

  const itemsByCategory = useSelector(state =>
    selectItemsByCategory(state, category)
  )
}

Using Selectors with React-Redux​

• Calling Selectors with Parameters​

It's common to want to pass additional arguments to a selector function. However, useSelector always calls the provided selector function with one argument - the Redux root state.

The simplest solution is to pass an anonymous selector to useSelector, and then immediately call the real selector with both state and any additional arguments:

import { selectTodoById } from './todosSlice'

function TodoListitem({ todoId }) {
  // Captures `todoId` from scope, gets `state` as an arg, and forwards both
  // to the actual selector function to extract the result
  const todo = useSelector(state => selectTodoById(state, todoId))
}

Selector Factories​

createSelector only has a default cache size of 1, and this is per each unique instance of a selector. This creates problems when a single selector function needs to get reused in multiple places with differing inputs.

One option is to create a "selector factory" - a function that runs createSelector() and generates a new unique selector instance every time it's called:

const makeSelectItemsByCategory = () => {
  const selectItemsByCategory = createSelector(
    [state => state.items, (state, category) => category],
    (items, category) => items.filter(item => item.category === category)
  )
  return selectItemsByCategory
}

This is particularly useful when multiple similar UI components need to derive different subsets of the data based on props.

Passing Input Parameters​

A Reselect-generated selector function can be called with as many arguments as you want: selectThings(a, b, c, d, e). However, what matters for re-running the output is not the number of arguments, or whether the arguments themselves have changed to be new references. Instead, it's about the "input selectors" that were defined, and whether their results have changed. Similarly, the arguments for the "output selector" are solely based on what the input selectors return.

This means that if you want to pass additional parameters through to the output selector, you must define input selectors that extract those values from the original selector arguments:

const selectItemsByCategory = createSelector(
  [
    // Usual first input - extract value from `state`
    state => state.items,
    // Take the second arg, `category`, and forward to the output selector
    (state, category) => category
  ],
  // Output selector gets (`items, category)` as args
  (items, category) => items.filter(item => item.category === category)
)

For consistency, you may want to consider passing additional parameters to a selector as a single object, such as selectThings(state, otherArgs), and then extracting values from the otherArgs object.

Reselect Usage Patterns and Limitations​

Nesting Selectors​

It's possible to take selectors generated with createSelector, and use them as inputs for other selectors as well. In this example, the selectCompletedTodos selector is used as an input to selectCompletedTodoDescriptions:

const selectTodos = state => state.todos

const selectCompletedTodos = createSelector([selectTodos], todos => todos.filter(todo => todo.completed) )

const selectCompletedTodoDescriptions = createSelector( [selectCompletedTodos], completedTodos => completedTodos.map(todo => todo.text) )

Because of this, it's important that all of the "input selectors" you provide should accept the same types of parameters. Otherwise, the selectors will break.

const selectItems = state => state.items;

// expects a number as the second argument
const selectItemId = (state, itemId) => itemId;

// expects an object as the second argument
const selectOtherField (state, someObject) => someObject.someField;

const selectItemById = createSelector(
    [selectItems, selectItemId, selectOtherField],
    (items, itemId, someField) => items[itemId]
);

In this example, selectItemId expects that its second argument will be some simple value, while selectOtherField expects that the second argument is an object. If you call selectItemById(state, 42), selectOtherField will break because it's trying to access 42.someField.

const selectItems = state => state.items
const selectItemId = (state, itemId) => itemId

const selectItemById = createSelector(
  [selectItems, selectItemId],
  (items, itemId) => items[itemId]
)

const item = selectItemById(state, 42)

/*
Internally, Reselect does something like this:

const firstArg = selectItems(state, 42);  
const secondArg = selectItemId(state, 42);  

const result = outputSelector(firstArg, secondArg);  
return result;  
*/

const a = someSelector(state, 1) // first call, not memoized
const b = someSelector(state, 1) // same inputs, memoized
const c = someSelector(state, 2) // different inputs, not memoized
const d = someSelector(state, 1) // different inputs from last time, not memoized

const state = {
  a: {
    first: 5
  },
  b: 10
}

const selectA = state => state.a
const selectB = state => state.b

const selectA1 = createSelector([selectA], a => a.first)

const selectResult = createSelector([selectA1, selectB], (a1, b) => {
  console.log('Output selector running')
  return a1 + b
})

const result = selectResult(state)
// Log: "Output selector running"
console.log(result)
// 15

const secondResult = selectResult(state)
// No log output
console.log(secondResult)
// 15

Note that the second time we called selectResult, the "output selector" didn't execute. Because the results of selectA1 and selectB were the same as the first call, selectResult was able to return the memoized result from the first call.

// ❌ BROKEN: this will not memoize correctly, and does nothing useful!
const brokenSelector = createSelector(
  state => state.todos,
  todos => todos
)

When you call the selector, Reselect will run your input selectors with all of the arguments you gave, and looks at the returned values. If any of the results are === different than before, it will re-run the output selector, and pass in those results as the arguments. If all of the results are the same as the last time, it will skip re-running the output selector, and just return the cached final result from before.

This means that "input selectors" should usually just extract and return values, and the "output selector" should do the transformation work.

const selectA = state => state.a
const selectB = state => state.b
const selectC = state => state.c

const selectABC = createSelector([selectA, selectB, selectC], (a, b, c) => {
  // do something with a, b, and c, and return a result
  return a + b + c
})

// Call the selector function and get a result
const abc = selectABC(state)

// could also be written as separate arguments, and works exactly the same
const selectABC2 = createSelector(selectA, selectB, selectC, (a, b, c) => {
  // do something with a, b, and c, and return a result
  return a + b + c
})

Memoization is a form of caching. It involves tracking inputs to a function, and storing the inputs and the results for later reference. If a function is called with the same inputs as before, the function can skip doing the actual work, and return the same result it generated the last time it received those input values. This optimizes performance by only doing work if inputs have changed, and consistently returning the same result references if the inputs are the same.

function TodoList() {
  // ❌ WARNING: this _always_ returns a new reference, so it will _always_ re-render!
  const completedTodos = useSelector(state =>
    state.todos.map(todo => todo.completed)
  )
}

Optimizing Selectors with Memoization​

• Selectors used with useSelector or mapState will be re-run after every dispatched action, regardless of what section of the Redux root state was actually updated. Re-running expensive calculations when the input state sections didn't change is a waste of CPU time, and it's very likely that the inputs won't have changed most of the time anyway.
• useSelector and mapState rely on === reference equality checks of the return values to determine if the component needs to re-render. If a selector always returns new references, it will force the component to re-render even if the derived data is effectively the same as last time. This is especially common with array operations like map() and filter(), which return new array references.

// Bad: always returning a new reference
const selectTodoDescriptions = state => {
  // This creates a new array reference!
  return state.todos.map(todo => todo.text)
}

useSelector compares its results using strict === reference comparisons, so the component will re-render any time the selector result is a new reference! This means that if you create a new reference in your selector and return it, your component could re-render every time an action has been dispatched, even if the data really isn't different.

import React, {useState, useMemo, useCallback} from 'react';

function App() {
  const [length, set_length] = useState(3);
  const [name, set_name] = useState('John Doe');

  const on_name_changed = useCallback((e) => set_name(e.target.value), []);
  const on_length_changed = useCallback((e) => set_length(Number(e.target.value)), []);

  return (
    <>
      <input value={name} onChange={on_name_changed} />
      <NameDisplay name={name}/>
      <hr />
      <input value={length} onChange={on_length_changed} />
      <FibDisplay length={length} />
    </>
  );
}

function FibDisplay({length}) {
  const numbers = useMemo(() => {
    console.log('Calculating numbers...');
    const result = [1, 1];
    for (let i = 2; i < length; i++) {
      result[i] = result[i - 1] + result[i - 2];
    }
    return result;
  }, [length]);

  return <p>{length} numbers of the fibonacci sequence: {numbers.join(', ')}</p>;
}

const NameDisplay = React.memo(function ({name}) {
  console.log('Rerendering name...');
  return <p>Your name is {name}</p>;
});

export default App;

Considering React.memo, all these concerns apply as well. But there is one more: React.memo should only be applied to pure components.

Another issue has to do with Redux/Context and hooks. Before hooks, the Redux selectors passed store values via props and React.memo would capture them. However, if you are using useSelector/useContext, React.memo will not rerender your component when those change. Because of these complexities, I would advise against React.memo, as useMemo should be sufficient in most cases.

// Memoized
import React, {useState, useMemo} from 'react';

function App() {
  const [length, set_length] = useState(3);
  const [name, set_name] = useState('John Doe');

  return (
    <>
      <input value={name} onChange={e => set_name(e.target.value)} />
      <NameDisplay name={name}/>
      <hr />
      <input value={length} onChange={e => set_length(Number(e.target.value))} />
      <FibDisplay length={length} />
    </>
  );
}

function FibDisplay({length}) {
  const numbers = useMemo(() => {
    console.log('Calculating numbers...');
    const result = [1, 1];
    for (let i = 2; i < length; i++) {
      result[i] = result[i - 1] + result[i - 2];
    }
    return result;
  }, [length]);

  return <p>{length} numbers of the fibonacci sequence: {numbers.join(', ')}</p>;
}

const NameDisplay = React.memo(function ({name}) {
  console.log('Rerendering name...');
  return <p>Your name is {name}</p>;
});

export default App;

import React, {useState} from 'react';

function App() {
  const [length, set_length] = useState(3);
  const [name, set_name] = useState('John Doe');

  return (
    <>
      <input value={name} onChange={e => set_name(e.target.value)} />
      <NameDisplay name={name}/>
      <hr />
      <input value={length} onChange={e => set_length(Number(e.target.value))} />
      <FibDisplay length={length} />
    </>
  );
}

function FibDisplay({length}) {
  console.log('Calculating numbers & rerendering...');
  const numbers = [1, 1];
  for (let i = 2; i < length; i++) {
    numbers[i] = numbers[i - 1] + numbers[i - 2];
  }

  return <p>{length} numbers of the fibonacci sequence: {numbers.join(', ')}</p>;
}

function NameDisplay({name}) {
  console.log('Rerendering name...');
  return <p>Your name is {name}</p>;
}

export default App;

const MemoizedLogout = React.memo(Logout);
function MyApp({ store, cookies }) {
  const onLogout = useCallback(
    () => cookies.clear('session'), 
    [cookies]
  );
  return (
    <div className="main">
      <header>
        <MemoizedLogout
          username={store.username}
          onLogout={onLogout}
        />
      </header>
      {store.content}
    </div>
  );
}

function MyApp({ store, cookies }) {
  return (
    <div className="main">
      <header>
        <MemoizedLogout
          username={store.username}
          onLogout={() => cookies.clear('session')}
        />
      </header>
      {store.content}
    </div>
  );
}

function Logout({ username, onLogout }) {
  return (
    <div onClick={onLogout}>
      Logout {username}
    </div>
  );
}
const MemoizedLogout = React.memo(Logout);

function MovieViewsRealtime({ title, releaseDate, views }) {
  return (
    <div>
      <MemoizedMovie title={title} releaseDate={releaseDate} />
      Movie views: {views}
    </div>
  )
}

function MovieViewsRealtime({ title, releaseDate, views }) {
  return (
    <div>
      <Movie title={title} releaseDate={releaseDate} />
      Movie views: {views}
    </div>
  );
}

function moviePropsAreEqual(prevMovie, nextMovie) {
  return prevMovie.title === nextMovie.title
    && prevMovie.releaseDate === nextMovie.releaseDate;
}
const MemoizedMovie2 = React.memo(Movie, moviePropsAreEqual);

React.memo(Component, [areEqual(prevProps, nextProps)]);

export function Movie({ title, releaseDate }) {
  return (
    <div>
      <div>Movie title: {title}</div>
      <div>Release date: {releaseDate}</div>
    </div>
  );
}
export const MemoizedMovie = React.memo(Movie);

const currentPage = // something calculated from ScrollPosition

const lastResult = usePageQuery(currentPage - 1, { skip: currentPage === 1 }) // don't fetch pages before 0
const currentResult = usePageQuery(currentPage)
const nextResult = usePageQuery(currentPage + 1)

const combined = useMemo(() => {
  const arr = new Array(pageSize * (currentPage + 1))
  for (const data of [lastResult.data, currentResult.data, nextResult.data]) {
    if (data) {
      arr.splice(data.offset, data.items.length, ...data.items)
    }
  }
  return arr
}, [pageSize, currentPage, lastResult.data, currentResult.data, nextResult.data])

// work with `combined` here