TensorFlow: How to use CudnnLSTM with variable input length (like dynamic_rnn)?

Just found this:

tf.contrib.cudnn_rnn.CudnnLSTM currently does not support batches with sequences of different length, thus this is normally not an option to use.

Source: http://returnn.readthedocs.io/en/latest/tf_lstm_benchmark.html

TensorFlow will soon finally have support for variable sequence lengths: https://github.com/tensorflow/tensorflow/blob/2f672ee9562a452f8dbfa259a8ccec56367e9b17/tensorflow/contrib/cudnn_rnn/python/layers/cudnn_rnn.py#L389

It looks like it landed too late for 1.13, so it'll probably only be available on TensorFlow 1.14.

You can try it out today by installing the tf-nightly-gpu package and passing sequence_lengths=lengths where lenghts is a tf.int32 Tensor with shape [batch_size], containing the lengths of each sequence in your batch.