How do I make a required thrift field optional?

It is possible to do this if you control all the readers and writers of the struct in the following steps :

  1. Change the field in the thrift schema from required to optional. Sync this new schema in all the systems that refer to or use this struct. Do not stop writing this field. This is a safe change as the field will still be always present.
  2. Only after the schema change has been deployed in all the systems using this struct, you can stop writing this field in some cases. This is now safe to do as this field is marked as optional in every system, there are no "old readers" as referred to in the other answer.

You can't, because as the saying goes required is forever. The following is a quote from Diwaker Gupta's highly recommendable "Missing Guide". It pretty much nails it down why one should think twice (at least) before using required:

Required Is Forever

You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field — old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead. Some have come the conclusion that using required does more harm than good; they prefer to use only optional. However, this view is not universal.

I'm afraid that the only option is to deprecate the entire struct and create a new one.

In addition to that, there are in fact three degrees of requiredness, only two of them have keywords:

  • required: must exist on read, must be set on write
  • optional: may or may not be set, entirely optional
  • "default": may not exist on read, always written (unless it is a null pointer )

The "default" requiredness is applied implicitly when neither required nor optional are specified.

As one clearly sees, the restrictions for required are rather strict if we look at the compatibility site of things. Even adding a new required field to a struct may lead to incompatibility, e.g. if a new client is reading data from an old server (or vice versa), because the new required field is not in the data written by the old impl, but expected by the new impl.

Tags:

Thrift