tflite-runtime

[ProPython007]

https://pypi.org/project/tflite-runtime/

[am1tyadav]

I'd like to see this as well but the TFLite RT was changed to Lite RT https://developers.googleblog.com/en/tensorflow-lite-is-now-litert/ and the corresponding package on PyPi is now: https://pypi.org/project/ai-edge-litert/

The ai-edge-litert package does not have any dependency other than numpy . Not sure if this warrants a separate question/ request.

[FeodorFitsner]

It doesn't look like it's open-source project: https://pypi.org/project/ai-edge-litert/#files

[am1tyadav]

No? it says the license is Apache 2.0 on the PyPI page. Has the same license on the source code as well: https://github.com/google-ai-edge/LiteRT

[am1tyadav]

However, it doesn't yet work with Python 3.12 it seems, so that could be a problem

[FeodorFitsner]

Nice, will take a look!

[am1tyadav]

I wanted to make sure this works in a minimal python environment. So it turns out that ai-edge-litert only depends on numpy~=1.26 (but not numpy > 2). There is no other dependency, I've still tried this just on Python 3.11

Inference

• AI Edge LiteRT 1.0.1
• NumPy 1.26.4

from ai_edge_litert.interpreter import Interpreter
import numpy as np

x = np.zeros((1, 28, 28), dtype=np.float32)

model_filepath = "model.tflite"
signature_key = "serving_default"
output_key = "output_0"

interpreter = Interpreter(model_path=model_filepath)
signature = interpreter.get_signature_runner(signature_key)
outputs = signature(keras_tensor=x)[output_key]

print(outputs) # in this case a np.ndarray of shape (1, 10)

Creating the TFLite Model

• TensorFlow 2.18
• Keras 3.7

import tensorflow as tf
import keras

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = keras.models.Sequential(
    [
        keras.layers.Flatten(input_shape=(28, 28)),
        keras.layers.Dense(64, activation="relu"),
        keras.layers.Dense(10, activation="softmax"),
    ]
)

model.compile(
    optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"]
)
model.fit(
    x_train, y_train, validation_data=(x_test, y_test), epochs=5
)

converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

with open("model.tflite", "wb") as f:
    f.write(tflite_model)