importing data from firestore into bigQuery

Firestore now allows to import / export data. The data is exported to a Cloud Storage bucket and from there it can be imported into Big Query. Here are some links that might be helpful:

  • https://firebase.google.com/docs/firestore/manage-data/export-import
  • https://cloud.google.com/bigquery/docs/loading-data-cloud-datastore (Firestore uses the same format as Datastore for imports / exports)

** Edit: Docs for BigQuery imports from Firestore are now available https://cloud.google.com/bigquery/docs/loading-data-cloud-firestore


In case anyone need it. I ended up automating scripts because current export data option only allows stream data and preserve for 30 days.

export data

gcloud beta firestore export --collection-ids=users gs://mybucket/users

load backup into bq

bq load --source_format=DATASTORE_BACKUP mydataset.users gs://gs://mybucket/users/all_namespaces/kind_users/all_namespaces_kind_users.export_metadata

You can now export data from Cloud Firestore to BigQuery with a Firebase Extension. See: https://firebase.google.com/products/extensions/firestore-bigquery-export

Also see David's answer on how to import/export data.

Outdated answer below:

There is no built-in support to import data from the Firebase Realtime Database or Cloud Firestore into BigQuery.

For now, if you want to import data, you'll have to write code to do so.