How to choose the Redux state shape for an app with list/detail views and pagination?

Please consult “real world” example from Redux repo.
It shows the solution to exactly this problem.

Your state shape should look like this:

{
  entities: {
    users: {
      1: { id: 1, name: 'Dan' },
      42: { id: 42, name: 'Mary' }
    }
  },
  visibleUsers: {
    ids: [1, 42],
    isFetching: false,
    offset: 0
  }
}

Note I’m storing entities (ID -> Object maps) and visibleUsers (description of currently visible users with pagination state and IDs) separately.

This seems similar to your “Shared data set” approach. However I don’t think the drawbacks you list are real problems inherent to this approach. Let’s take a look at them.

Now the problem I have with this approach is that when then list of users gets huge(say millions), it might take a while to download

You don’t need to download all of them! Merging all downloaded entities to entities doesn’t mean you should query all of them. The entities should contain all entities that have been downloaded so far—not all entities in the world. Instead, you’d only download those you’re currently showing according to the pagination information.

when I navigate directly to /detail/, I wouldn't yet have all of my users downloaded, so to get data for just the one, I'm gonna have to download them all. Millions of users just to edit one.

No, you’d request just one of them. The response action would fire, and reducer responsible for entities would merge this single entity into the existing state. Just because state.entities.users may contain more than one user doesn’t mean you need to download all of them. Think of entities as of a cache that doesn’t have to be filled.

---

Finally, I will direct you again to the “real world” example from Redux repo. It shows exactly how to write a reducer for pagination information and entity cache, and how to normalize JSON in your API responses with normalizr so that it’s easy for reducers to extract information from server actions in a uniform way.