Converting and deploying a TFLite model tutorial

This tutorial demonstrates how to convert and deploy a TFLite model using the TOSA converter for TFLite and the ML SDK for Vulkan®. It walks through model conversion to TOSA MLIR, VGF file generation, and execution using the ML SDK Scenario Runner.

  1. Install the TOSA converter for TFLite:

The repo is available at: TOSA converter for TFLite, please refer to their README.md for building the tool.

After installation, run the following command to verify the tool is installed:

which tosa-converter-for-tflite

  1. Download the TFLite Model:

In this tutorial, we will use the SESR (super-efficient super resolution) model from the Arm® Model Zoo. To clone the entire Arm® Model Zoo repository into your current working directory, run:

git clone https://github.com/Arm-Examples/ML-zoo.git

The model includes both the quantized .tflite file and corresponding .npy input/output test data, which we will use in later steps.

  1. Convert the Model to TOSA MLIR Bytecode:

Run the following command to convert the SESR model from TFLite to TOSA in the form of MLIR bytecode:

tosa-converter-for-tflite ./ML-zoo/models/superresolution/SESR/tflite_int8/SESR_1080p_to_4K_withD2S_full_int8.tflite \
    -o SESR_1080p_to_4K_withD2S_full_int8.mlirbc

  1. Generate the VGF File and Scenario Template:

Similarly as for deploying the PyTorch files (Converting and deploying a PyTorch model tutorial), to generate the VGF file and Scenario Template, use the TOSA MLIR bytecode file:

model-converter --input SESR_1080p_to_4K_withD2S_full_int8.mlirbc --output SESR_1080p_to_4K_withD2S_full_int8.vgf

vgf_dump --input SESR_1080p_to_4K_withD2S_full_int8.vgf --output scenario.json --scenario-template

  1. Modify the Scenario Template:

    1. Replace TEMPLATE_PATH_TENSOR_INPUT_0 with ML-zoo/models/superresolution/SESR/tflite_int8/testing_input/net_input/0.npy.

    2. Replace TEMPLATE_PATH_TENSOR_OUTPUT_0 with output.npy.

  2. Run the ML SDK Scenario Runner on the ML Emulation Layer for Vulkan®:

scenario-runner --scenario scenario.json

After the run, the contents of ML-zoo/models/superresolution/SESR/tflite_int8/testing_output/net_output/0.npy and output.npy should be identical.

To verify the output matches the reference output, save the following script into compare_numpy.py:

#!/usr/bin/env python3
#
# SPDX-FileCopyrightText: Copyright 2023-2025 Arm Limited and/or its affiliates <open-source-office@arm.com>
# SPDX-License-Identifier: Apache-2.0
#
import argparse

try:
    import argcomplete
except:
    argcomplete = None
import numpy


def parse_arguments():
    parser = argparse.ArgumentParser()
    parser.add_argument("files", metavar="file", nargs="+", help="NumPy data file")
    if argcomplete:
        argcomplete.autocomplete(parser)
    return parser.parse_args()


def main():
    args = parse_arguments()
    if len(args.files) <= 1:
        return 0
    first = numpy.load(args.files[0])
    for f in args.files[1:]:
        other = numpy.load(f)
        if not numpy.array_equal(first, other):
            return 1
    return 0


if __name__ == "__main__":
    exit(main())

Then run the following command:

python3 compare_numpy.py ML-zoo/models/superresolution/SESR/tflite_int8/testing_output/net_output/0.npy output.npy

An exit status of 0 indicates that the contents of the two files are identical.