fastai error predicting with exported/reloaded model: "Input type and weight type should be the same"

So the answer to this ended up being relatively simple:

1) As noted in my comment, training in mixed precision mode (setting conv_learner to_fp16()) caused the error with the exported/reloaded model

2) To train in mixed precision mode (which is faster than regular training) and enable export/reload of the model without errors, simply set the model back to default precision before exporting.

...In code, simply changing the example above:

# Export Model
conv_learner.export()

to:

# Export Model (after converting back to default precision for safe export/reload
conv_learner = conv_learner.to_fp32()
conv_learner.export()

...and now the full (reproduceable) code example above runs without errors, including the prediction after model reload.

Your model is in half precision if you have .to_fp16, which would be the same if you would model.half() in PyTorch.

Actually if you trace the code .to_fp16 will call model.half() But there is a problem. If you convert the batch norm layer also to half precision you may get the convergence problem.

This is why you would typically do this in PyTorch:

model.half()  # convert to half precision
for layer in model.modules():
  if isinstance(module, torch.nn.modules.batchnorm._BatchNorm):      
    layer.float()

This will convert any layer to half precision other than batch norm.

Note that code from PyTorch forum is also OK, but just for nn.BatchNorm2d.

Then make sure your input is in half precision using to() like this:

import torch
t = torch.tensor(10.)
print(t)
print(t.dtype)
t=t.to(dtype=torch.float16)
print(t)
print(t.dtype)
# tensor(10.)
# torch.float32
# tensor(10., dtype=torch.float16)
# torch.float16