VGF Decoder C++ API

The VGF Library decoder enables the parsing of VGF files. The VGF Library contains different sections and the methods for each section. The important sections are the Header section, Model Sequence Table, Module Table, Model Resource Table and the Constant Section.

Header section decoding

The header decoder allows you to verify the validity of the VGF file and the file version. The header decoder also uses memory size and offset pairs to provide the location of the other sections within the VGF file.

To parse the VGF file header, you must load the file into memory and then create a header decoder. To create a header decoder, you must provide the VGF data-pointer:

    std::unique_ptr<HeaderDecoder> decoder = CreateHeaderDecoder(vgf_data.c_str());

After creating a header decoder, you can verify the validity of the VGF file and its compatibility with the library. To verify the validity of the VGF file, check if their versions match:

    ASSERT_TRUE(decoder->IsValid() == true);
    ASSERT_TRUE(decoder->CheckVersion() == true);

To further verify a potentially unsafe vgf file, use the verify functions for each section:

VerifyModuleTable(
    vgf_data.c_str() + headerDecoder->GetModuleTableOffset(),
    headerDecoder->GetModuleTableSize());

VerifyModelResourceTable(
    vgf_data.c_str() + headerDecoder->GetModelResourceTableOffset(),
    headerDecoder->GetModelResourceTableSize());

VerifyModelSequenceTable(
    vgf_data.c_str() + headerDecoder->GetModelSequenceTableOffset(),
    headerDecoder->GetModelSequenceTableSize());

VerifyConstant(
    vgf_data.c_str() + headerDecoder->GetConstantsOffset(),
    headerDecoder->GetConstantsSize());

Model Sequence Table decoding

The Model Sequence table contains all of the segments in the VGF file. Each segment entry contains the input and output resources of the segment and their exposed bindings. The Model sequence table also contains information about any constants that the segment uses and optional push constants.

For each resource, either input, output, or constant, a pointer into the Model Resource table is stored to access detailed information of the resource itself. For example, if the resource is a tensor or an image, the Model Resource table holds its shape or dimension.

With the previously created header decoder, you must create a Model Sequence Table decoder.

    ASSERT_TRUE(VerifyModelSequenceTable(vgf_data.c_str() + headerDecoder->GetModelSequenceTableOffset(),
                                         headerDecoder->GetModelSequenceTableSize()));
    std::unique_ptr<ModelSequenceTableDecoder> seqTableDecoder =
        CreateModelSequenceTableDecoder(vgf_data.c_str() + headerDecoder->GetModelSequenceTableOffset());

To extract all the relevant information of any segment, you can address each segment using the respective index. For example, you can create a binding slot decoder to extract the inputs or outputs to the segment.

Binding Slot decoding

Each segment in the Model Sequence Table (MST) corresponds to a single Vulkan® pipeline object. You can use binding slots to map how specific resources are to be bound to pipelines via one or more ‘DescriptorSet’ objects when using the Vulkan® API. Optionally, you can also use different views of these slots to determine more optimal scheduling strategies. For example, by analyzing the graph connectivity between segments some platforms may enabled overlapped execution where dependencies allow.

Most of the useful information required is contained in the Model Resource Table and each binding slot contains an index to an entry in the table. Multiple Binding slots can reference a single Model Resource Table.

The segment contains 1 or more DescriptorSetInfos. Each DescriptorSetInfos correspond to a separate Vulkan® descriptor set object required by the associated Vulkan® pipeline.

Therefore, when you translate the VGF contents to Vulkan® API calls, there are 2 distinct stages where you need the binding slot information:

  1. During creation of the descriptor set layout

  2. During writing of the descriptor set

There are several routes to getting binding slot information depending on the required view.

  1. The most common route is via DescriptorSetInfos on the ‘Segment’. Each segment has a list of descriptor sets which themselves contain a view of the binding slots. There is a 1-1 mapping between the DescriptorSetInfos in the Model Sequence Table and the Vulkan® API DescriptorSets.

  2. The ModelSequenceTable has a list of binding slots that form the inputs and outputs to the entire model. This is useful when connecting game or app resources to the model.

  3. Each segment has a list of input and output binding slots. You can use the binding slots for for graph connectivity analysis.

    ASSERT_TRUE(segmentIndex < seqTableDecoder->modelSequenceTableSize());
    ASSERT_TRUE(discriptorSetInfoIndex < seqTableDecoder->getSegmentDescriptorSetInfosSize(segmentIndex));

    BindingSlotArrayHandle bindingSlotsHandle =
        seqTableDecoder->getDescriptorBindingSlotsHandle(segmentIndex, discriptorSetInfoIndex);
    size_t numSlots = seqTableDecoder->getBindingsSize(bindingSlotsHandle);

To setup the corresponding resources for a segment, you can use the binding slot decoder together with the Model Resource table.

Push Constant decoding

Vulkan® push constants are also supported in the VGF encoding. The push constants are available in the segment info.

    mlsdk_decoder_push_constant_ranges_handle pcrHandle =
        mlsdk_decoder_model_sequence_get_segment_push_constant_range(modelSequenceDecoder, segmentIndex);

    size_t numRanges = mlsdk_decoder_get_push_constant_ranges_size(modelSequenceDecoder, pcrHandle);

    ASSERT_TRUE(numRanges > rangeIdx);

    uint32_t rangeOffset = mlsdk_decoder_get_push_constant_range_offset(modelSequenceDecoder, pcrHandle, rangeIdx);
    uint32_t rangeSize = mlsdk_decoder_get_push_constant_range_size(modelSequenceDecoder, pcrHandle, rangeIdx);
    uint32_t rangeStageFlags =
        mlsdk_decoder_get_push_constant_range_stage_flags(modelSequenceDecoder, pcrHandle, rangeIdx);

Model Resource Table decoding

The Module Resource table stores information used to build up the resources which are needed by the model encoded in the VGF. An entry in the table holds the type of the resource, which can be tensor, image, buffer, along with resource specific information. For example, a tensor entry contains its element type, shape and stride. However, an image entry contains its image encoding and its dimension.

To find the correct MRT entry, for example, binding slot ‘0’, you can use the previously acquired BindingSlotArrayHandle:

    ASSERT_TRUE(slotIndex < numSlots);

    uint32_t bindingId = seqTableDecoder->getBindingSlotBinding(bindingSlotsHandle, slotIndex);
    uint32_t mrtIndex = seqTableDecoder->getBindingSlotMrtIndex(bindingSlotsHandle, slotIndex);

After you create a Model Resource Table decoder, you can use the obtained mrtIndex to extract information for that binding:

    ASSERT_TRUE(VerifyModelResourceTable(vgf_data.c_str() + headerDecoder->GetModelResourceTableOffset(),
                                         headerDecoder->GetModelResourceTableSize()));
    std::unique_ptr<ModelResourceTableDecoder> mrtDecoder =
        CreateModelResourceTableDecoder(vgf_data.c_str() + headerDecoder->GetModelResourceTableOffset());

    size_t numEntries = mrtDecoder->size();

    ASSERT_TRUE(numEntries > mrtIndex);

    ResourceCategory category = mrtDecoder->getCategory(mrtIndex);
    DataView<int64_t> shape = mrtDecoder->getTensorShape(mrtIndex);
    std::optional<DescriptorType> type = mrtDecoder->getDescriptorType(mrtIndex);
    FormatType format = mrtDecoder->getVkFormat(mrtIndex);
    DataView<int64_t> stride = mrtDecoder->getTensorStride(mrtIndex);
    // ...

Module Table decoding

The API facilitates the extraction of each module which contains the module type, SPIR-V™ code existence and, if applicable, its corresponding code.

After you create a Module table decoder, you can extract the respective module of the segment. To extract the module, you can use the moduleIndex:

    ASSERT_TRUE(VerifyModuleTable(vgf_data.c_str() + headerDecoder->GetModuleTableOffset(),
                                  headerDecoder->GetModuleTableSize()));
    std::unique_ptr<ModuleTableDecoder> moduleDecoder =
        CreateModuleTableDecoder(vgf_data.c_str() + headerDecoder->GetModuleTableOffset());

    size_t numModules = moduleDecoder->size();

    ASSERT_TRUE(moduleIndex < numModules);

    DataView<uint32_t> moduleCode;
    if (moduleDecoder->hasSPIRV(moduleIndex)) {
        moduleCode = moduleDecoder->getModuleCode(moduleIndex);

        //...
    }

Decoder API reference

using BindingSlotArrayHandle = const BindingSlotArrayHandle_s*
using NameArrayHandle = const NameArrayHandle_s*
using PushConstantRangeHandle = const PushConstantRangeHandle_s*
size_t HeaderSize()

Returns the size of Header section in memory’.

size_t HeaderDecoderSize()

Returns the size of Header decoder in memory’.

std::unique_ptr<HeaderDecoder> CreateHeaderDecoder(const void *data)

Constructs a Header decoder.

Parameters:

data --

HeaderDecoder *CreateHeaderDecoderInPlace(const void *data, void *decoderMem)

Constructs a Header decoder in-place using pre-allocated memory.

Parameters:

  • data --

  • decoderMem --

size_t ModuleTableDecoderSize()

Returns the size of Module Table decoder in memory’.

bool VerifyModuleTable(const void *data, uint64_t size)

Returns true if input points to a valid Module Table.

Parameters:

  • data --

  • size -- Max count of read bytes

std::unique_ptr<ModuleTableDecoder> CreateModuleTableDecoder(const void *data)

Constructs a Module Table decoder.

Parameters:

data --

ModuleTableDecoder *CreateModuleTableDecoderInPlace(const void *data, void *decoderMem)

Constructs a Module Table decoder in-place using pre-allocated memory.

Parameters:

  • data --

  • decoderMem --

size_t ModelResourceTableDecoderSize()

Returns the size of Module Resource Table decoder in memory’.

bool VerifyModelResourceTable(const void *data, uint64_t size)

Returns true if input points to a valid Model Resource Table.

Parameters:

  • data --

  • size -- Max count of read bytes

std::unique_ptr<ModelResourceTableDecoder> CreateModelResourceTableDecoder(const void *data)

Constructs a Model Resource Table decoder.

Parameters:

data --

ModelResourceTableDecoder *CreateModelResourceTableDecoderInPlace(const void *data, void *decoderMem)

Constructs a Model Resource Table decoder in-place using pre-allocated memory.

Parameters:

  • data --

  • decoderMem --

size_t ConstantDecoderSize()

Returns the size of Constant decoder in memory’.

bool VerifyConstant(const void *data, uint64_t size)

Returns true if input points to a valid Constant section.

Parameters:

  • data --

  • size -- Max count of read bytes

std::unique_ptr<ConstantDecoder> CreateConstantDecoder(const void *data)

Constructs a Constant section decoder.

Parameters:

data -- Pointer to the Constants section data

ConstantDecoder *CreateConstantDecoderInPlace(const void *data, void *decoderMem)

Constructs a Constant section decoder in-place using preallocated memory.

Parameters:

  • data -- Pointer to the Constants section data

  • decoderMem -- memory in which to create the decoder

bool VerifyModelSequenceTable(const void *data, uint64_t size)

Returns true if input points to a valid Model Sequence Table.

Parameters:

  • data --

  • size -- Max count of read bytes

std::unique_ptr<ModelSequenceTableDecoder> CreateModelSequenceTableDecoder(const void *data)

Constructs a Model Sequence Table decoder.

Parameters:

data --

ModelSequenceTableDecoder *CreateModelSequenceTableDecoderInPlace(const void *data, void *decoderMem)

Constructs a Model Sequence Table decoder in-place using pre-allocated memory.

Parameters:

  • data --

  • decoderMem --

class HeaderDecoder
#include <decoder.hpp>

Public Functions

virtual bool IsValid() const = 0

Checks the header magic value of the VGF file.

virtual uint16_t GetEncoderVulkanHeadersVersion() const = 0

Get the value of VK_HEADER_VERSION used by Encoding tool.

virtual bool CheckVersion() const = 0

Checks the VGF version.

virtual uint8_t GetMajor() const = 0

Returns the Major version value.

virtual uint8_t GetMinor() const = 0

Returns the Minor version value.

virtual uint8_t GetPatch() const = 0

Returns the Patch version value.

virtual uint64_t GetModuleTableSize() const = 0

Returns the size of the Module Table in memory.

virtual uint64_t GetModuleTableOffset() const = 0

Returns the relative location of the Module Table in memory.

virtual uint64_t GetModelSequenceTableOffset() const = 0

Returns the relative location of the Model Sequence Table in memory.

virtual uint64_t GetModelSequenceTableSize() const = 0

Returns the size of the Model Sequence Table in memory.

virtual uint64_t GetModelResourceTableOffset() const = 0

Returns the relative location of the Model Resource Table in memory.

virtual uint64_t GetModelResourceTableSize() const = 0

Returns the size of the Model Resource Table in memory.

virtual uint64_t GetConstantsSize() const = 0

Returns the size of the Constant section in memory.

virtual uint64_t GetConstantsOffset() const = 0

Returns the relative location of the Constant section in memory.

class ModuleTableDecoder
#include <decoder.hpp>

Public Functions

virtual size_t size() const = 0

Retrieves the number of entries in the Module Table.

virtual ModuleType getModuleType(uint32_t idx) const = 0

Retrieves the moduleType of module ‘idx’.

Parameters:

idx --

virtual bool hasSPIRV(uint32_t idx) const = 0

Checks if the table entry ‘idx’ has SPIR-V code.

Parameters:

idx --

virtual std::string_view getModuleName(uint32_t idx) const = 0

Retrieves the module name of module ‘idx’.

Parameters:

idx --

virtual std::string_view getModuleEntryPoint(uint32_t idx) const = 0

Returns the entry point of module ‘idx’.

Parameters:

idx --

virtual DataView<uint32_t> getModuleCode(uint32_t idx) const = 0

Retrieves the SPIR-V code of the module ‘idx’ in the Module Table.

Parameters:

idx --

class ModelResourceTableDecoder
#include <decoder.hpp>

Public Functions

virtual size_t size() const = 0

Returns the number of entries in the Model Resource Table.

virtual std::optional<DescriptorType> getDescriptorType(uint32_t id) const = 0

Returns the DescriptorType of the MRT entry ‘id’.

Parameters:

id --

virtual FormatType getVkFormat(uint32_t id) const = 0

Returns the VkFormat of the MRT entry ‘id’.

Parameters:

id --

virtual ResourceCategory getCategory(uint32_t id) const = 0

Returns the ResourceCategory of the MRT entry ‘id’.

Parameters:

id --

virtual DataView<int64_t> getTensorShape(uint32_t id) const = 0

Returns the tensor shape of the MRT entry ‘id’. A “-1” value represents an unshaped dimension.

Parameters:

id --

virtual DataView<int64_t> getTensorStride(uint32_t id) const = 0

Returns the tensor stride of the MRT entry ‘id’.

Parameters:

id --

class ConstantDecoder
#include <decoder.hpp>

Public Functions

virtual size_t size() const = 0

Returns the number of constants in the Constant section.

virtual uint32_t getConstantMrtIndex(uint32_t idx) const = 0

Returns the Model Resource Table index of constant ‘idx’.

Parameters:

idx -- Index of the constant

virtual bool isSparseConstant(uint32_t idx) const = 0

Returns true if constant is 2:4 sparse on one dimension.

Parameters:

idx -- Index of the constant

virtual int64_t getConstantSparsityDimension(uint32_t idx) const = 0

Returns the dimension on which the constant is sparse.

Parameters:

idx -- Index of the constant

virtual DataView<uint8_t> getConstant(uint32_t idx) const = 0

Returns the constant at location ‘idx’ in the Constant section.

Parameters:

idx -- Index of the constant

struct BindingSlotArrayHandle_s
#include <decoder.hpp>
struct NameArrayHandle_s
#include <decoder.hpp>
struct PushConstantRangeHandle_s
#include <decoder.hpp>
class ModelSequenceTableDecoder
#include <decoder.hpp>

Public Functions

virtual size_t modelSequenceTableSize() const = 0

Returns the number of segments.

virtual size_t getSegmentDescriptorSetInfosSize(uint32_t segmentIdx) const = 0

Returns the number of descriptor set infos for segment ‘segmentIdx’ in the Model Sequence Table.

Parameters:

segmentIdx --

virtual DataView<uint32_t> getSegmentConstantIndexes(uint32_t segmentIdx) const = 0

Returns the constants that are linked to segment ‘segmentIdx’ in the Model Sequence Table.

Parameters:

segmentIdx --

virtual ModuleType getSegmentType(uint32_t segmentIdx) const = 0

Returns the ModuleType of segment ‘segmentIdx’.

Parameters:

segmentIdx --

virtual std::string_view getSegmentName(uint32_t segmentIdx) const = 0

Returns the name of segment ‘segmentIdx’.

Parameters:

segmentIdx --

virtual uint32_t getSegmentModuleIndex(uint32_t segmentIdx) const = 0

Returns the name of segment ‘segmentIdx’.

Parameters:

segmentIdx --

virtual DataView<uint32_t> getSegmentDispatchShape(uint32_t segmentIdx) const = 0

Returns the dispatch shape for segment ‘segIdx’.

Parameters:

segmentIdx --

virtual BindingSlotArrayHandle getDescriptorBindingSlotsHandle(uint32_t segmentIdx, uint32_t descIdx) const = 0

Retrieves the ‘BindingSlotArrayHandle’ corresponding to segment ‘segmentIdx’ descriptor ‘descIdx’.

Parameters:

  • segmentIdx --

  • descIdx --

Returns:

Handle to the binding slot array

virtual BindingSlotArrayHandle getSegmentInputBindingSlotsHandle(uint32_t segmentIdx) const = 0

Retrieves the inputs ‘BindingSlotArrayHandle’ corresponding to segment ‘segmentIdx’.

Parameters:

segmentIdx --

Returns:

Handle to the binding slot array

virtual BindingSlotArrayHandle getSegmentOutputBindingSlotsHandle(uint32_t segmentIdx) const = 0

Retrieves the outputs ‘BindingSlotArrayHandle’ corresponding to segment ‘segmentIdx’.

Parameters:

segmentIdx --

Returns:

Handle to the binding slot array

virtual BindingSlotArrayHandle getModelSequenceInputBindingSlotsHandle() const = 0

Retrieves the inputs ‘BindingSlotArrayHandle’ for the model sequence.

Returns:

Handle to the binding slot array

virtual BindingSlotArrayHandle getModelSequenceOutputBindingSlotsHandle() const = 0

Retrieves the outputs ‘BindingSlotArrayHandle’ for the model sequence.

Returns:

Handle to the binding slot array

virtual NameArrayHandle getModelSequenceInputNamesHandle() const = 0

Retrieves the ‘NameArrayHandle’ for the inputs of the model.

Returns:

Handle to the name array

virtual NameArrayHandle getModelSequenceOutputNamesHandle() const = 0

Retrieves the ‘NameArrayHandle’ for the outputs of the model.

Returns:

Handle to the name array

virtual size_t getNamesSize(NameArrayHandle handle) const = 0

Retrieves the number of names.

Parameters:

handle -- Opaque handle to an array of names

Returns:

The number of names in the array referenced by handle

virtual std::string_view getName(NameArrayHandle handle, uint32_t nameIdx) const = 0

Retrieves the name at a given index in the array.

Parameters:

  • handle -- Opaque handle to an array of names

  • nameIdx -- Index of the name in the array to fetch

Returns:

name

virtual size_t getBindingsSize(BindingSlotArrayHandle handle) const = 0

Retrieves the number of bindings.

Parameters:

handle -- Opaque handle to an array of BindSlots

Returns:

The number of BindingSlots in the array referenced by handle

virtual uint32_t getBindingSlotBinding(BindingSlotArrayHandle handle, uint32_t slotIdx) const = 0

Retrieves the bindings at slot ‘slotIdx’.

Parameters:

  • handle -- Opaque handle to an array of BindSlots

  • slotIdx -- index of the slot in the the array of bindingslots referenced by the handle

Returns:

The binding id of the selected BindingSlot

virtual uint32_t getBindingSlotMrtIndex(BindingSlotArrayHandle handle, uint32_t slotIdx) const = 0

Retrieves the MRT index which is associated to the binding slot ‘slotIdx’.

Parameters:

  • handle -- Opaque handle to an array of BindSlots

  • slotIdx -- index of the slot in the the array of bindingslots referenced by the handle

Returns:

The index of the entry in Model Resource Table referenced by this binding slot

virtual PushConstantRangeHandle getSegmentPushConstRange(uint32_t segmentIdx) const = 0

Retrieves the push constant ranges corresponding to segment ‘segmentIdx’ and stores them in target.

Parameters:

segmentIdx -- Index of the segment to query

Returns:

The handle to the push constants range for the segment

virtual size_t getPushConstRangesSize(PushConstantRangeHandle handle) const = 0

Returns the number of push constant ranges.

Parameters:

handle -- The handle to the push constants range

virtual uint32_t getPushConstRangeStageFlags(PushConstantRangeHandle handle, uint32_t rangeIdx) const = 0

Returns the stage flags of push constant range ‘rangeIdx’.

Parameters:

  • handle -- The handle to the push constants range

  • rangeIdx -- Index of the specific range to query

virtual uint32_t getPushConstRangeOffset(PushConstantRangeHandle handle, uint32_t rangeIdx) const = 0

Returns the offset of push constant range ‘rangeIdx’.

Parameters:

  • handle -- The handle to the push constants range

  • rangeIdx -- Index of the specific range to query

virtual uint32_t getPushConstRangeSize(PushConstantRangeHandle handle, uint32_t rangeIdx) const = 0

Returns the size of push constant range ‘rangeIdx’.

Parameters:

  • handle -- The handle to the push constants range

  • rangeIdx -- Index of the specific range to query