What's the difference between buildscript and allprojects in build.gradle?

The "buildscript" configuration section is for gradle itself (i.e. changes to how gradle is able to perform the build). So this section will usually include the Android Gradle plugin.

The "allprojects" section is for the modules being built by Gradle.

Oftentimes the repository section is the same for both, since both will get their dependencies from jcenter usually (or maybe maven central). But the "dependencies" section will be different.

Usually the "dependencies" section for "allprojects" is empty since the dependencies for each module are unique and will be in the "build.gradle" file within each of the modules. However, if all of the modules shared the same dependencies then they could be listed here.

TL;DR: buildscript helps find plugins, allprojects applies to all projects

https://docs.gradle.org/current/userguide/userguide_single.html#applyPluginBuildscript says

Binary plugins that have been published as external jar files can be added to a project by adding the plugin to the build script classpath and then applying the plugin.

So you need buildscript for gradle to find the plugins, as

Gradle at its core intentionally provides very little for real world automation. All of the useful features, like the ability to compile Java code, are added by plugins. Plugins add new tasks (e.g. JavaCompile), domain objects (e.g. SourceSet), conventions (e.g. Java source is located at src/main/java) as well as extending core objects and objects from other plugins.

Concerning allprojects:

The Project API provides a property allprojects which returns a list with the current project and all its subprojects underneath it. If you call allprojects with a closure, the statements of the closure are delegated to the projects associated with allprojects.