Keras Predictions and Sagemaker Predictions are different

Solution for Keras Predictions and Sagemaker Predictions are different
is Given Below:

I am Trying to deploy a tensorflow keras model using amazon sagemaker. Process finishes successfully, yet i get different prediction results when predicted directly using keras and when calling sagemaker endpoint to make predictions.

I used these steps in deploying the model to sagemaker.

Check the following example.

data = np.random.randn(1, 150, 150, 3)

# predict using amazon sagemaker
sagemaker_predict = uncompiled_predictor.predict(data)
print(sagemaker_predict)

#predict same using keras
val = model.predict(data)
print(val)


>>{'predictions': [[0.491645753]]}
[[0.]]

Is this something supposed to happen? For my knowledge it should be the same. For some reason data gets corrupted or sagemaker weights get reinitialized. Any ideas?

Not suppose to happen.
See what you get if you deloy the model directly to TensorFlow serving (which is what the SageMaker inference container wraps).
To experiment faster you can work with SageMaker inference container in local mode, so you can start/stop an endopint in seconds.

Finally found a solution. It seems to be a problem with .h5 (HDF5) weights file, for some reason sagemaker seems not to extract weight from .h5. Therefore changed the weights file to TensorFlow SavedModel format

As for tensorflow keras save and serialize documentation

There are two formats you can use to save an entire model to disk: the TensorFlow SavedModel format, and the older Keras H5 format. The recommended format is SavedModel. It is the default when you use model.save().

You can switch to the H5 format by:

Passing save_format=”h5″ to save().
Passing a filename that ends in .h5 or .keras to save()

So instead of saving weights as

model.save("my_model.h5")

save as

model.save("my_model")

And load the same weights as

keras.models.load_model("my_model")

This will save your file in TensorFlow SavedModel format which you can follow in the above documentation to load and deploy to sagemaker.