Building algebraic geometry without prime ideals

Actually, you have rediscovered a nice motivation of using prime ideals as points. Indeed, your collection of points are triples $(R, k_x, \mathrm{ev}_x)$ where , $\mathrm{ev}_x \colon R \to k_x$ is a homomorphism. The collection of all such triples is a class rather a set. In any case, you should not change the universe to get the underlying topological space whose functions are given by $R$, much as you won't change the universe when you reconstruct a differential manifold from the algebra of differentiable functions.

A nice solution is to impose an equivalence relation on the set of points. Define $$(R, k_x, \mathrm{ev}_x) \sim (R, k_y, \mathrm{ev}_y)$$ whenever there are field extensions $i_1 \colon k_x \hookrightarrow K$ and $i_2 \colon k_y \hookrightarrow K$ such that $i_1 \circ \mathrm{ev}_x = i_2 \circ \mathrm{ev}_y$. After all, a point with coordinates in a field remains the same if we consider the coordinates in a bigger field. Now take the quotient set of the equivalence relation. It is clear that the triples $(R, k_x, \mathrm{ev}_x)$ are classified by $\mathrm{Im}(\mathrm{ev}_x)$, equivalently by $\mathrm{Ker}(\mathrm{ev}_x)$, that turns out to be a prime ideal. Thus, every equivalence class has a canonical representative $(R, \kappa(\mathfrak{p}), \mathrm{ev}_\mathfrak{p})$ where $\mathfrak{p}$ is a prime ideal in $R$, $\kappa(\mathfrak{p}) = R_\mathfrak{p}/\mathfrak{p}R_\mathfrak{p}$, the residue field of $\mathfrak{p}$ and $\mathrm{ev}_\mathfrak{p} \colon R \to \kappa(\mathfrak{p})$ the canonical map. So in fact points as maps to fields are classified by primes, have a canonical field where elements of $R$ may be evaluated and the collection that form the equivalence classes is clearly a set.

Of course, the next step is to define a sheaf of rings, that in some sense, might be interpreted as a sheaf of functions on $\mathrm{Spec}(R)$. This is exactly the motivation I use for the philosophy points are primes in algebraic geometry in my graduate courses under the name "the sermon of points". Of course, this point of view is well-known though it is rarely displayed in print.

This nice approach to points on schemes in fact becomes crucial once one leaves the world of schemes and travels to the galaxy of stacks.

For an algebraic stack $X$, one defines a point of $X$ to be a morphism $\mathrm{Spec}(k) \to X$, modulo the natural equivalence relation discussed above.

See in particular:

https://stacks.math.columbia.edu/tag/01J5

https://stacks.math.columbia.edu/tag/04XE