Scenario Runner C++ API

Warning

The Scenario Runner C++ API is currently experimental and subject to change, including incompatible changes, in future releases.

The Scenario Runner API creates and executes Vulkan® scenarios. A scenario can be constructed from an existing JSON description or defined programmatically. Both workflows return a mlsdk::scenariorunner::Scenario, which can accept new in-memory inputs and be executed repeatedly without rebuilding its resources and pipelines.

Build from JSON

Use mlsdk::scenariorunner::ScenarioJsonFactory to preserve the standalone Scenario Runner workflow in an application. Each UID defined for a memory resource (buffer, image, or tensor) in the JSON file can be resolved to the corresponding mlsdk::scenariorunner::BufferId, mlsdk::scenariorunner::ImageId, or mlsdk::scenariorunner::TensorId for in-memory data transfer. Each lookup validates that the UID exists and identifies the expected resource type:

        // JSON memory-resource UIDs remain available for lookup after construction.
        auto scenario = ScenarioJsonFactory::make(assetPath(executable, "increment.json"));
        const auto inputId = scenario->getBufferId("input");
        const auto outputId = scenario->getBufferId("output");

        const std::vector<uint32_t> input{10, 20, 30, 40};
        const std::vector<uint32_t> expected{11, 21, 31, 41};
        scenario->upload(inputId, view(input));
        scenario->run();
        const auto output = scenario->download(outputId);

Relative resource paths in the JSON file are resolved from the scenario file’s directory by default. The factory also accepts explicit working and output directories. JSON resource src fields are loaded while the scenario is created, and resource dst fields are written after each non-dry run. The typed upload() and download() operations remain available alongside this file-based input and output.

Build programmatically

Use mlsdk::scenariorunner::createScenarioBuilder() when the scenario is assembled by the application. Register resources before commands and retain the returned IDs. Commands use those IDs for their bindings, and the same IDs are used to transfer data after the scenario is built:

        // Register the shader and all resources before defining the dispatch.
        auto builder = createScenarioBuilder();

        ShaderInfo shaderInfo{};
        shaderInfo.debugName = "increment";
        shaderInfo.entry = "main";
        shaderInfo.shaderType = ShaderType::GLSL;
        shaderInfo.stage = ShaderStage::Compute;
        shaderInfo.src = readShaderCode(assetPath(executable, "increment.comp").string(), shaderInfo);
        const auto shaderId = builder->addShader(shaderInfo);

        const auto inputId = builder->addBuffer(BufferInfo{"input", 16});
        const auto outputId = builder->addBuffer(BufferInfo{"output", 16});

        DispatchComputeData dispatch{shaderId};
        dispatch.debugName = "increment";
        dispatch.bindings = {{0, 0, inputId, std::nullopt, vk::DescriptorType::eStorageBuffer},
                             {0, 1, outputId, std::nullopt, vk::DescriptorType::eStorageBuffer}};
        dispatch.computeDispatch.gwcx = 4;
        dispatch.computeDispatch.profileName = dispatch.debugName;
        builder->addDispatchCompute(std::move(dispatch));

        // The returned IDs remain valid for uploading inputs and downloading outputs.
        auto scenario = builder->build(ScenarioOptions{});
        const std::vector<uint32_t> input{1, 2, 3, 4};
        const std::vector<uint32_t> expected{2, 3, 4, 5};
        scenario->upload(inputId, view(input));
        scenario->run();
        const auto output = scenario->download(outputId);

The example has four stages:

createScenarioBuilder() creates the builder. addShader() and addBuffer() register every resource and return the IDs used below.
DispatchComputeData is configured to identify the shader, bind the input and output buffers to descriptor set 0 bindings 0 and 1, and specify four X-axis workgroups. addDispatchCompute() then appends that command to the builder’s execution sequence. Each programmatic binding explicitly provides its Vulkan® descriptor type.
build() consumes the builder and returns an executable scenario. The resource IDs remain valid for the scenario’s lifetime.
upload(), run(), and download() transfer the application input, execute the dispatch, and return the owning output data respectively.

Calling build() transfers the registered resources and commands to the scenario. The builder must not be reused afterwards. Upload operations copy the provided bytes before returning, while download operations return owning data. For programmatically-created scenarios, ShaderInfo::src contains immutable SPIR-V code and VgfInfo::src contains an immutable VGF view. readShaderCode() and loadVgfView() create these values from files; relative paths are resolved from the application’s current working directory. The JSON factory resolves relative resource paths from the scenario file’s directory and performs these conversions automatically.

API Reference

The main entry points are:

API	Purpose
`mlsdk::scenariorunner::ScenarioJsonFactory`	Build a scenario from an existing JSON scenario file.
`mlsdk::scenariorunner::createScenarioBuilder()`	Start defining a scenario programmatically.
`mlsdk::scenariorunner::ScenarioBuilder`	Register resources and commands, then build an executable scenario.
`mlsdk::scenariorunner::Scenario`	Upload inputs, execute commands, and download outputs.

Scenario construction and execution 

class ScenarioBuilder

Public interface for defining and building a scenario.

Resource registration returns stable typed IDs. Callers retain these IDs and use them to transfer data through the Scenario returned by build(). The builder copies each resource description and consumes command values. Calling build() transfers ownership of the complete definition to the returned Scenario and leaves the builder immutable.

Public Functions

virtual ~ScenarioBuilder() = default: Destroy the builder.

virtual BufferId addBuffer(const BufferInfo &info) = 0

Register a buffer resource.

Parameters:: info -- Buffer description to copy into the scenario.
Throws:: std::runtime_error -- If the builder has already been consumed.
Returns:: The ID used to refer to the buffer in commands and data transfers.

virtual ImageId addImage(const ImageInfo &info) = 0

Register an image resource.

Parameters:: info -- Image description to copy into the scenario.
Throws:: std::runtime_error -- If the builder has already been consumed.
Returns:: The ID used to refer to the image in commands and data transfers.

virtual TensorId addTensor(const TensorInfo &info) = 0

Register a tensor resource.

Parameters:: info -- Tensor description to copy into the scenario.
Throws:: std::runtime_error -- If the builder has already been consumed.
Returns:: The ID used to refer to the tensor in commands and data transfers.

virtual ShaderId addShader(const ShaderInfo &info) = 0

Register a shader resource.

Parameters:: info -- Shader description to copy into the scenario.
Throws:: std::runtime_error -- If the builder has already been consumed.
Returns:: The ID used to refer to the shader in commands.

virtual RawDataId addRawData(const RawDataInfo &info) = 0

Register raw command data.

Parameters:: info -- Raw-data description to copy into the scenario.
Throws:: std::runtime_error -- If the source path is empty or the builder was consumed.
Returns:: The ID used to refer to the data in commands.

virtual VgfId addVgf(const VgfInfo &info) = 0

Register a VGF resource.

Parameters:: info -- VGF description to copy into the scenario.
Throws:: std::runtime_error -- If the builder has already been consumed.
Returns:: The ID used to refer to the VGF in commands.

virtual GraphConstantResourceId addGraphConstant(const GraphConstantInfo &info) = 0

Register a graph constant resource.

Parameters:: info -- Graph-constant description to copy into the scenario.
Throws:: std::runtime_error -- If the builder has already been consumed.
Returns:: The ID used to refer to the constant in graph commands.

virtual ImageBarrierId addImageBarrier(const ImageBarrierInfo &info) = 0

Register an image barrier.

Parameters:: info -- Barrier description containing a previously registered image ID.
Throws:: std::runtime_error -- If the image ID is invalid or the builder was consumed.
Returns:: The ID used to refer to the barrier in a barrier command.

virtual BufferBarrierId addBufferBarrier(const BufferBarrierInfo &info) = 0

Register a buffer barrier.

Parameters:: info -- Barrier description containing a previously registered buffer ID.
Throws:: std::runtime_error -- If the buffer ID is invalid or the builder was consumed.
Returns:: The ID used to refer to the barrier in a barrier command.

virtual TensorBarrierId addTensorBarrier(const TensorBarrierInfo &info) = 0

Register a tensor barrier.

Parameters:: info -- Barrier description containing a previously registered tensor ID.
Throws:: std::runtime_error -- If the tensor ID is invalid or the builder was consumed.
Returns:: The ID used to refer to the barrier in a barrier command.

virtual MemoryBarrierId addMemoryBarrier(const MemoryBarrierInfo &info) = 0

Register a global memory barrier.

Parameters:: info -- Barrier description to copy into the scenario.
Throws:: std::runtime_error -- If the builder has already been consumed.
Returns:: The ID used to refer to the barrier in a barrier command.

virtual MemoryGroupId createMemoryGroup() = 0

Create an empty memory aliasing group.

Throws:: std::runtime_error -- If the builder has already been consumed.
Returns:: The ID of the new memory group.

virtual void addResourceToMemoryGroup(MemoryGroupId group, MemoryResourceId resource) = 0

Add a buffer, image, or tensor to a memory aliasing group.

Parameters:

group -- Existing memory group.
resource -- Previously registered memory resource.

Throws:

std::runtime_error -- If either ID is invalid or the resource is already in another group.

virtual void addDispatchCompute(DispatchComputeData command) = 0

Append a compute dispatch to the execution sequence.

Parameters:: command -- Command whose referenced resources have already been registered.
Throws:: std::runtime_error -- If a resource ID is invalid or the builder was consumed.

virtual void addDispatchFragment(DispatchFragmentData command) = 0

Append a fragment dispatch to the execution sequence.

Parameters:: command -- Command whose referenced resources have already been registered.
Throws:: std::runtime_error -- If a resource ID is invalid or the builder was consumed.

virtual void addDispatchVgf(DispatchVgfData command) = 0

Append a VGF dispatch to the execution sequence.

Parameters:: command -- Command whose referenced resources have already been registered.
Throws:: std::runtime_error -- If a resource ID is invalid or the builder was consumed.

virtual void addDispatchDataGraphBinary(DispatchDataGraphBinaryData command) = 0

Append a data graph binary dispatch to the execution sequence.

Parameters:: command -- Command whose referenced resources have already been registered.
Throws:: std::runtime_error -- If a resource ID is invalid or the builder was consumed.

virtual void addDispatchDataGraph(DispatchDataGraphData command) = 0

Append a SPIR-V™ graph dispatch to the execution sequence.

Parameters:: command -- Command whose referenced resources have already been registered.
Throws:: std::runtime_error -- If a resource ID is invalid or the builder was consumed.

virtual void addDispatchOpticalFlow(DispatchOpticalFlowData command) = 0

Append an optical flow dispatch to the execution sequence.

Parameters:: command -- Command whose referenced resources have already been registered.
Throws:: std::runtime_error -- If a resource ID is invalid or the builder was consumed.

virtual void addPipelineBarrier(PipelineBarrierData command) = 0

Append a set of resource barriers to the execution sequence.

Parameters:: command -- Command whose barrier IDs have already been registered.
Throws:: std::runtime_error -- If a barrier ID is invalid or the builder was consumed.

virtual void addFrameBoundary(FrameBoundaryData command) = 0

Append a profiling boundary marker to the execution sequence.

Parameters:: command -- Boundary whose resource IDs have already been registered.
Throws:: std::runtime_error -- If a resource ID is invalid or the builder was consumed.

virtual std::unique_ptr<Scenario> build(const ScenarioOptions &options) = 0

Consume the builder and return a ready-to-run scenario.

Parameters:: options -- Runtime and diagnostic options for the scenario.
Throws:: std::runtime_error -- If the scenario definition is invalid or the builder was already consumed.
Returns:: A scenario that owns the registered resources and commands.

std::unique_ptr<ScenarioBuilder> mlsdk::scenariorunner::createScenarioBuilder()

Create an empty scenario builder.

Returns:: A new builder with no resources or commands.

class ScenarioJsonFactory

Construct a scenario from a JSON description.

The factory preserves the standalone Scenario Runner JSON workflow while returning the same Scenario interface used by programmatically built scenarios. The returned scenario owns all parsed construction data.

Public Static Functions

static std::unique_ptr<Scenario> make(const std::filesystem::path &scenarioFile, const std::filesystem::path &workDir = {}, const std::filesystem::path &outputDir = {}, const ScenarioOptions &options = {})

Parse a JSON scenario file and return a ready-to-run scenario.

Parameters:

scenarioFile -- Path to the JSON scenario description.
workDir -- Base directory for relative input paths. The scenario file’s parent directory is used when this is empty.
outputDir -- Base directory for relative output paths.
options -- Runtime and diagnostic options for the scenario.

Throws:

std::runtime_error -- If parsing, validation, or runtime setup fails.

Returns:

A ready-to-run scenario that owns the parsed resources and commands.

class Scenario

Public interface for executing a built scenario and transferring resource data.

Implementations own the built resources and pipelines. A caller can upload new input data, execute the scenario, and download output data repeatedly without rebuilding those resources.

Public Functions

virtual void run() = 0

Execute the scenario once.

This is equivalent to run(1, false).

virtual void run(int repeatCount, bool dryRun) = 0

Execute the scenario one or more times.

Parameters:

repeatCount -- Number of executions; must be greater than zero.
dryRun -- Skip workload execution and output-resource saving.

Throws:

std::invalid_argument -- If repeatCount is not positive.

virtual BufferId getBufferId(std::string_view uid) const = 0

Get a buffer resource ID from its UID.

Note

This lookup is available for scenarios created by ScenarioJsonFactory.

Parameters:: uid -- UID declared by the JSON scenario.
Throws:: std::runtime_error -- If the UID is unknown or identifies another resource type.
Returns:: The corresponding buffer ID.

virtual ImageId getImageId(std::string_view uid) const = 0

Get an image resource ID from its UID.

Note

This lookup is available for scenarios created by ScenarioJsonFactory.

Parameters:: uid -- UID declared by the JSON scenario.
Throws:: std::runtime_error -- If the UID is unknown or identifies another resource type.
Returns:: The corresponding image ID.

virtual TensorId getTensorId(std::string_view uid) const = 0

Get a tensor resource ID from its UID.

Note

This lookup is available for scenarios created by ScenarioJsonFactory.

Parameters:: uid -- UID declared by the JSON scenario.
Throws:: std::runtime_error -- If the UID is unknown or identifies another resource type.
Returns:: The corresponding tensor ID.

virtual void upload(BufferId id, const BufferDataView &data) = 0

Upload data to an existing buffer resource.

Note

The data is copied before this function returns.

Parameters:

id -- Buffer registered in this scenario.
data -- Non-owning view of the bytes to upload.

Throws:

std::runtime_error -- If the ID is invalid or the data size does not match the resource.

virtual void upload(ImageId id, const ImageDataView &data) = 0

Upload data to an existing image resource.

Note

The data is copied before this function returns.

Note

Missing mip levels are generated using linear-filtered blits. Automatic tiling selection prefers support for this operation; explicitly configured tiling is respected.

Parameters:

id -- Image registered in this scenario.
data -- Non-owning view of the image bytes and validation metadata.

Throws:

std::runtime_error -- If the ID or supplied resource metadata is invalid.
std::runtime_error -- If mip levels are missing and the image’s format and tiling do not support linear-filtered blits. Supply the complete mip chain in this case.

virtual void upload(TensorId id, const TensorDataView &data) = 0

Upload data to an existing tensor resource.

Note

The data is copied before this function returns.

Parameters:

id -- Tensor registered in this scenario.
data -- Non-owning view of the tensor bytes and validation metadata.

Throws:

std::runtime_error -- If the ID or supplied resource metadata is invalid.

virtual BufferData download(BufferId id) const = 0

Download data from an existing buffer resource.

Parameters:: id -- Buffer registered in this scenario.
Throws:: std::runtime_error -- If the ID is invalid.
Returns:: An owning copy of the buffer contents.

virtual ImageData download(ImageId id) = 0

Download data from an existing image resource.

Parameters:: id -- Image registered in this scenario.
Throws:: std::runtime_error -- If the ID is invalid.
Returns:: An owning copy of the base image mip and its metadata.

virtual TensorData download(TensorId id) const = 0

Download data from an existing tensor resource.

Parameters:: id -- Tensor registered in this scenario.
Throws:: std::runtime_error -- If the ID is invalid.
Returns:: An owning copy of the tensor contents and its metadata.

In-memory resource data 

struct BufferDataView

Non-owning bytes used to upload a buffer.

Public Members

const void *data = {nullptr}: Address of the first byte, or nullptr when size is zero.

size_t size = {0}: Number of bytes available at data.

struct BufferData

Owning buffer contents returned by Scenario::download().

Public Members

std::vector<std::byte> data: Downloaded bytes.

struct ImageDataView

Non-owning bytes and optional metadata used to upload an image.

Public Members

const void *data = {nullptr}: Address of the first byte, or nullptr when size is zero.

size_t size = {0}: Number of bytes available at data.

std::vector<int64_t> shape: Image shape to validate; it must match the resource shape.

std::optional<vk::Format> format = {std::nullopt}: Image format to validate, or std::nullopt to omit format validation.

uint32_t mipLevels = {1}: Number of mip levels contained in data.

struct ImageData

Owning image contents and metadata returned by Scenario::download().

Public Members

std::vector<std::byte> data: Downloaded base-mip bytes in tightly packed image order.

std::vector<int64_t> shape: Logical image shape.

std::optional<vk::Format> format = {std::nullopt}: Image texel format when available.

uint32_t mipLevels = {1}: Number of mip levels represented in data.

struct TensorDataView

Non-owning bytes and optional metadata used to upload a tensor.

Public Members

const void *data = {nullptr}: Address of the first byte, or nullptr when size is zero.

size_t size = {0}: Number of bytes available at data.

std::vector<int64_t> shape

Tensor shape to validate.

An empty shape is accepted only for a rank-converted scalar tensor.

std::optional<vk::Format> format = {std::nullopt}: Tensor format to validate, or std::nullopt to omit format validation.

struct TensorData

Owning tensor contents and metadata returned by Scenario::download().

Public Members

std::vector<std::byte> data: Downloaded bytes in tightly packed tensor order.

std::vector<int64_t> shape: Logical tensor shape.

std::optional<vk::Format> format = {std::nullopt}: Tensor element format when available.

Resource identifiers 

ScenarioBuilder returns a distinct opaque ID type for each resource category. Retain these IDs to define command bindings and transfer resource data. Do not construct IDs from numeric values.

using mlsdk::scenariorunner::BufferId = ResourceId<BufferIdTag>: Buffer resource ID.

using mlsdk::scenariorunner::ImageId = ResourceId<ImageIdTag>: Image resource ID.

using mlsdk::scenariorunner::TensorId = ResourceId<TensorIdTag>: Tensor resource ID.

using mlsdk::scenariorunner::ShaderId = ResourceId<ShaderIdTag>: Shader resource ID.

using mlsdk::scenariorunner::RawDataId = ResourceId<RawDataIdTag>: Raw-data resource ID.

using mlsdk::scenariorunner::VgfId = ResourceId<VgfIdTag>: VGF resource ID.

using mlsdk::scenariorunner::GraphConstantResourceId = ResourceId<GraphConstantResourceIdTag>: Graph-constant resource ID.

using mlsdk::scenariorunner::MemoryGroupId = ResourceId<MemoryGroupIdTag>: Memory aliasing group ID.

using mlsdk::scenariorunner::ImageBarrierId = ResourceId<ImageBarrierIdTag>: Image barrier ID.

using mlsdk::scenariorunner::BufferBarrierId = ResourceId<BufferBarrierIdTag>: Buffer barrier ID.

using mlsdk::scenariorunner::TensorBarrierId = ResourceId<TensorBarrierIdTag>: Tensor barrier ID.

using mlsdk::scenariorunner::MemoryBarrierId = ResourceId<MemoryBarrierIdTag>: Global memory barrier ID.

using mlsdk::scenariorunner::MemoryResourceId = std::variant<BufferId, ImageId, TensorId>: ID of a buffer, image, or tensor that can belong to a memory group.

Memory resource descriptions 

Scenario Runner translates these descriptions to Vulkan® values during runtime setup. They intentionally provide a smaller API and do not necessarily map one-to-one to Vulkan® enums or flag combinations.

enum class mlsdk::scenariorunner::Tiling

Memory organization used for an image or tensor.

Values:

enumerator Optimal: Implementation-dependent organization optimized for device access.

enumerator Linear: Texels or tensor elements use a linear organization.

enumerator Unknown: Tiling has not been specified.

enum class mlsdk::scenariorunner::FilterMode

Filtering operation used when sampling an image.

Values:

enumerator Linear: Linearly interpolate neighboring samples.

enumerator Nearest: Select the sample nearest to the texture coordinates.

enumerator Unknown: Filter mode has not been specified.

enum class mlsdk::scenariorunner::AddressMode

Behavior for sampling outside an image’s normalized coordinate range.

Values:

enumerator ClampBorder: Use the configured border color.

enumerator ClampEdge: Extend the nearest edge texel.

enumerator Repeat: Repeat the image periodically.

enumerator MirroredRepeat: Repeat and mirror the image periodically.

enumerator Unknown: Address mode has not been specified.

enum class mlsdk::scenariorunner::BorderColor

Color returned when sampling beyond an image border.

Values:

enumerator FloatTransparentBlack: Floating-point transparent black.

enumerator FloatOpaqueBlack: Floating-point opaque black.

enumerator FloatOpaqueWhite: Floating-point opaque white.

enumerator IntTransparentBlack: Integer transparent black.

enumerator IntOpaqueBlack: Integer opaque black.

enumerator IntOpaqueWhite: Integer opaque white.

enumerator FloatCustomEXT: Application-provided floating-point border color.

enumerator IntCustomEXT: Application-provided integer border color.

enumerator Unknown: Border color has not been specified.

using mlsdk::scenariorunner::CustomColorValue = std::variant<std::array<float, 4>, std::array<int32_t, 4>>: A variant of the supported custom border color representations.

struct BufferInfo

Structure that describes 1-dimensional buffer data.

Note

We don’t account for any specialized meta-data like for example padding or stride information. We assume that the provided data are packed in a linear manner

Public Members

std::string debugName: Human-readable name used in diagnostics and profiling output.

uint32_t size: Buffer size in bytes.

uint64_t memoryOffset = {}: Byte offset within an aliased memory allocation.

struct SamplerSettings

Structure that describes the sampler of an image.

Public Members

FilterMode minFilter = FilterMode::Nearest : Minification filter.

FilterMode magFilter = FilterMode::Nearest : Magnification filter.

FilterMode mipFilter = FilterMode::Nearest : Filter used between mip levels.

AddressMode addressModeU = AddressMode::ClampEdge : Addressing mode for the U coordinate.

AddressMode addressModeV = AddressMode::ClampEdge : Addressing mode for the V coordinate.

AddressMode addressModeW = AddressMode::ClampEdge : Addressing mode for the W coordinate.

BorderColor borderColor = BorderColor::FloatTransparentBlack : Border color used by clamp-to-border addressing.

CustomColorValue customBorderColor: Value used when borderColor selects a custom color.

struct ImageInfo

Structure that describes image.

Note

We don’t account for any specialized meta-data like for example padding or stride information. We assume that the provided data are packed in a linear manner

Public Members

std::string debugName: Human-readable name used in diagnostics and profiling output.

std::vector<int64_t> shape: Logical image dimensions.

vk::Format format: Format of data supplied to the image.

vk::Format targetFormat: Format used by the Vulkan® image.

bool isInput: Whether the image is populated as an input during initialization.

SamplerSettings samplerSettings: Sampling configuration for the image.

uint32_t mips: Number of mip levels.

bool isSampled = {false}: Allow the image to be used as a sampled image.

bool isStorage = {false}: Allow the image to be used as a storage image.

bool isColorAttachment = {false}: Allow the image to be used as a color attachment.

std::optional<Tiling> tiling: Requested memory tiling, or no value to let Scenario Runner choose.

uint64_t memoryOffset = {}: Byte offset within an aliased memory allocation.

struct TensorInfo

Structure that describes N-dimensional data.

Note

We don’t account for any specialized meta-data like for example padding or stride information. We assume that the provided data are packed in a linear manner

Public Members

std::string debugName: Human-readable name used in diagnostics and profiling output.

std::vector<int64_t> shape: Logical tensor dimensions.

vk::Format format: Vulkan® format describing each tensor element.

int64_t sparsityDimension = {-1}: Sparse dimension index, or -1 for a dense tensor.

bool descriptorBufferCaptureReplay = {false}: Enable descriptor-buffer capture and replay for this tensor.

Tiling tiling = {Tiling::Linear}: Memory organization used by the tensor.

uint64_t memoryOffset = {}: Byte offset within an aliased memory allocation.

Shader and graph resource descriptions 

enum class mlsdk::scenariorunner::ShaderType

Representation of shader source or binary data.

Values:

enumerator Unknown: Shader representation has not been specified.

enumerator SPIR_V: SPIR-V™ binary.

enumerator GLSL: GLSL source code.

enumerator HLSL: HLSL source code.

enum class mlsdk::scenariorunner::ShaderStage

Pipeline stage in which a shader executes.

Values:

enumerator Unknown: Shader stage has not been specified.

enumerator Compute: Compute shader stage.

enumerator Vertex: Vertex shader stage.

enumerator Fragment: Fragment shader stage.

union Constant

#include <types.hpp>

Value of a shader specialization constant.

Used for specialization constants

Public Members

int32_t i

uint32_t ui

float f

struct SpecializationConstant

Specialization constant identifier and value.

Public Members

int id = {}: Specialization constant ID declared by the shader.

Constant value: Value supplied for the specialization constant.

struct SpecializationConstantMap

Maps specialization constants to a shader within a data graph.

Public Members

std::vector<SpecializationConstant> specializationConstants: Constants supplied to the target shader.

std::string shaderTarget: Name of the shader within the data graph.

struct RawDataInfo

Information needed to load raw data from a file.

Public Members

std::string debugName: Human-readable name used in diagnostics.

std::string src: Path to the file containing the raw bytes.

struct ShaderInfo

Shader source, compilation settings, and execution metadata.

Public Members

std::string debugName: Human-readable name used in diagnostics and profiling output.

std::string entry: Shader entry-point name.

uint32_t pushConstantsSize = {}: Number of push-constant bytes required by the shader.

std::vector<SpecializationConstant> specializationConstants: Specialization constants applied when creating the pipeline.

std::shared_ptr<const std::vector<uint32_t>> src: Immutable in-memory SPIR-V module.

ShaderType shaderType = {ShaderType::Unknown}: Representation from which src was produced.

ShaderStage stage = {ShaderStage::Unknown}: Pipeline stage that executes the shader.

std::string buildOpts: Additional options passed to the shader compiler.

std::vector<std::string> includeDirs: Directories searched for source include files.

struct VgfInfo

Information needed to load and configure a VGF.

Public Members

std::string debugName: Human-readable name used in diagnostics and profiling output.

std::shared_ptr<const VgfView> src: Immutable in-memory VGF view.

uint32_t pushConstantsSize = {}: Number of push-constant bytes required by the graph.

std::vector<SpecializationConstantMap> specializationConstantMaps: Per-shader specialization constants applied to the graph.

struct GraphConstantInfo

Data and tensor metadata for a graph constant.

Public Members

vk::Format format = {vk::Format::eUndefined}: Vulkan® format describing each constant element.

std::vector<int64_t> dims: Logical dimensions of the constant tensor.

std::vector<uint8_t> data: Constant bytes in tightly packed tensor order.

std::string debugName: Human-readable name used in diagnostics.

Bindings and commands 

struct TypedBinding

Associates a descriptor binding location with a memory resource.

Public Members

uint32_t set = {}: Descriptor set index.

uint32_t id = {}: Binding index within the descriptor set.

MemoryResourceId resource: Buffer, image, or tensor bound to the descriptor.

std::optional<uint32_t> lod: Image mip level to bind, or no value for the base level.

vk::DescriptorType vkDescriptorType = {}: Descriptor type used for this binding.

struct ComputeDispatch

Workgroup count and profiling label for a compute dispatch.

Public Members

uint32_t gwcx = {1}: Number of workgroups dispatched in the X dimension.

uint32_t gwcy = {1}: Number of workgroups dispatched in the Y dimension.

uint32_t gwcz = {1}: Number of workgroups dispatched in the Z dimension.

std::string profileName: Label used for profiling output.

struct DispatchComputeData

Command that dispatches a compute shader.

Public Functions

inline explicit DispatchComputeData(ShaderId shader)

Construct a command for a compute shader.

Parameters:: shader -- Registered compute shader to dispatch.

Public Members

std::string debugName: Human-readable name used in diagnostics.

std::vector<TypedBinding> bindings: Descriptor bindings supplied to the shader.

ComputeDispatch computeDispatch = {}: Workgroup counts and profiling label.

ShaderId shader: Compute shader to dispatch.

bool implicitBarrier = {true}: Insert implicit synchronization around the dispatch.

std::optional<RawDataId> pushData: Optional raw push-constant data.

struct DispatchFragmentData

Command that dispatches a graphics pipeline.

Public Functions

inline DispatchFragmentData(ShaderId vertexShader, ShaderId fragmentShader)

Construct a graphics command from its shader stages.

Parameters:

vertexShader -- Registered vertex shader.
fragmentShader -- Registered fragment shader.

Public Members

std::string debugName: Human-readable name used in diagnostics.

std::vector<TypedBinding> bindings: Descriptor bindings supplied to the shaders.

ShaderId vertexShader: Vertex shader to dispatch.

ShaderId fragmentShader: Fragment shader to dispatch.

std::vector<Attachment> colorAttachments: Color attachments written by the graphics pipeline.

std::optional<vk::Extent2D> renderExtent: Render area, or no value to derive it from the attachments.

bool implicitBarrier = {true}: Insert implicit synchronization around the dispatch.

std::optional<RawDataId> pushData: Optional raw push-constant data.

struct Attachment

Image and optional mip level used as a color attachment.

Public Members

ImageId resource: Image used as the attachment.

std::optional<uint32_t> lod: Image mip level, or no value for the base level.

struct ResolvedPushConstantMap

Associates push-constant data with a shader in a data graph.

Public Members

RawDataId pushData: Registered raw data containing push-constant bytes.

std::string shaderTarget: Name of the target shader within the data graph.

struct ResolvedShaderSubstitution

Replaces a named shader in a data graph with another shader resource.

Public Members

ShaderId shader: Replacement shader resource.

std::string target: Name of the shader replaced within the data graph.

struct DispatchVgfData

Command that dispatches a VGF.

Public Functions

inline explicit DispatchVgfData(VgfId vgf)

Construct a command for a VGF.

Parameters:: vgf -- Registered VGF to dispatch.

Public Members

VgfId vgf: VGF to dispatch.

std::string debugName: Human-readable name used in diagnostics.

std::vector<TypedBinding> bindings: External resources bound to graph interfaces.

std::vector<ResolvedPushConstantMap> pushConstants: Push-constant data associated with graph shaders.

std::vector<ResolvedShaderSubstitution> shaderSubstitutions: Shader replacements applied to the graph.

bool implicitBarrier = {true}: Insert implicit synchronization around the dispatch.

struct DispatchDataGraphData

Command that dispatches a SPIR-V™ data graph.

Public Functions

inline explicit DispatchDataGraphData(ShaderId graphShader)

Construct a command for a SPIR-V™ data graph.

Parameters:: graphShader -- Registered SPIR-V™ graph shader to dispatch.

Public Members

ShaderId graphShader: SPIR-V™ graph shader to dispatch.

std::string debugName: Human-readable name used in diagnostics.

std::vector<TypedBinding> bindings: External resources bound to graph interfaces.

std::vector<GraphConstantResourceId> graphConstants: Constants supplied to the graph.

bool implicitBarrier = {true}: Insert implicit synchronization around the dispatch.

enum class mlsdk::scenariorunner::OpticalFlowGridSize : uint32_t

Optical flow output grid size.

Values:

enumerator Invalid: Invalid grid size.

enumerator e1x1: One-by-one-pixel grid.

enumerator e2x2: Two-by-two-pixel grid.

enumerator e4x4: Four-by-four-pixel grid.

enumerator e8x8: Eight-by-eight-pixel grid.

enum class mlsdk::scenariorunner::OpticalFlowPerformanceLevel : uint32_t

Requested optical flow performance level.

Values:

enumerator Invalid: Invalid performance level.

enumerator Unknown: Performance level has not been specified.

enumerator Slow: Slow performance preset.

enumerator Medium: Medium performance preset.

enumerator Fast: Fast performance preset.

struct DispatchOpticalFlowData

Command that dispatches an optical flow data graph.

Public Functions

inline DispatchOpticalFlowData(TypedBinding search, TypedBinding reference, TypedBinding output)

Construct an optical-flow command from its required images.

Parameters:

search -- Search image binding.
reference -- Template image binding.
output -- Output flow image binding.

Public Members

std::string debugName: Human-readable name used in diagnostics.

TypedBinding searchImage: Search image (Vulkan REFERENCE)

TypedBinding templateImage: Template image (Vulkan INPUT)

TypedBinding outputImage: Forward (horizontal, vertical) displacement from template to search, in input-image pixels.

std::optional<TypedBinding> hintMotionVectors: Optional motion-vector hints.

std::optional<TypedBinding> outputCost: Optional output cost image.

uint32_t width = {0}: Input image width in pixels.

uint32_t height = {0}: Input image height in pixels.

OpticalFlowPerformanceLevel performanceLevel = {OpticalFlowPerformanceLevel::Medium}: Requested performance preset.

uint32_t executionFlags = {0}: Vulkan® optical-flow execution flags.

OpticalFlowGridSize gridSize = {OpticalFlowGridSize::e1x1}: Output motion-vector grid size.

uint32_t meanFlowL1NormHint = {0}: Mean-flow L1-norm hint supplied to the implementation.

bool implicitBarrier = {true}: Insert implicit synchronization around the dispatch.

struct PipelineBarrierData

Command that executes a collection of registered barriers.

Public Members

std::vector<MemoryBarrierId> memoryBarriers: Global memory barriers to execute.

std::vector<ImageBarrierId> imageBarriers: Image barriers to execute.

std::vector<TensorBarrierId> tensorBarriers: Tensor barriers to execute.

std::vector<BufferBarrierId> bufferBarriers: Buffer barriers to execute.

struct FrameBoundaryData

Command that marks resources at a profiling boundary.

Public Members

std::vector<BufferId> buffers: Buffers included in the boundary marker.

std::vector<ImageId> images: Images included in the boundary marker.

std::vector<TensorId> tensors: Tensors included in the boundary marker.

Barriers and synchronization 

enum class mlsdk::scenariorunner::MemoryAccess

Type of memory access synchronized by a barrier.

Values:

enumerator ComputeShaderWrite: Write performed by a compute shader.

enumerator MemoryWrite: Any memory write.

enumerator MemoryRead: Any memory read.

enumerator GraphWrite: Write performed by a data graph.

enumerator GraphRead: Read performed by a data graph.

enumerator ComputeShaderRead: Read performed by a compute shader.

enumerator Unknown: Memory access has not been specified.

enum class mlsdk::scenariorunner::PipelineStage

Pipeline stage synchronized by a barrier.

Values:

enumerator Graph: Data graph pipeline stage.

enumerator Compute: Compute shader pipeline stage.

enumerator Graphics: Graphics pipeline stages.

enumerator All: All supported pipeline stages.

enumerator Unknown: Pipeline stage has not been specified.

enum class mlsdk::scenariorunner::ImageLayout

Image layout used before or after an image barrier.

Values:

enumerator General: General-purpose image layout.

enumerator TensorAliasing: Layout used when image memory aliases tensor memory.

enumerator Undefined: Previous image contents need not be preserved.

enumerator Unknown: Image layout has not been specified.

struct SubresourceRange

Mip levels and array layers affected by an image operation.

Public Members

uint32_t baseMipLevel = {0}: First mip level in the range.

uint32_t levelCount = {1}: Number of mip levels in the range.

uint32_t baseArrayLayer = {0}: First array layer in the range.

uint32_t layerCount = {1}: Number of array layers in the range.

struct BaseBarrierInfo

Access and pipeline-stage dependencies shared by barrier resources.

Subclassed by mlsdk::scenariorunner::BufferBarrierInfo, mlsdk::scenariorunner::ImageBarrierInfo, mlsdk::scenariorunner::MemoryBarrierInfo, mlsdk::scenariorunner::TensorBarrierInfo

Public Members

std::string debugName: Human-readable name used in diagnostics.

MemoryAccess srcAccess = {MemoryAccess::Unknown}: Access operations that must complete before the barrier.

MemoryAccess dstAccess = {MemoryAccess::Unknown}: Access operations that wait for the barrier.

std::vector<PipelineStage> srcStages = {PipelineStage::All}: Pipeline stages that must complete before the barrier.

std::vector<PipelineStage> dstStages = {PipelineStage::All}: Pipeline stages that wait for the barrier.

struct ImageBarrierInfo : public mlsdk::scenariorunner::BaseBarrierInfo 

Barrier dependencies and layout transition for an image.

Public Members

ImageId image = {0}: Image synchronized by the barrier.

ImageLayout oldLayout = {ImageLayout::Undefined}: Expected layout before the barrier.

ImageLayout newLayout = {ImageLayout::Undefined}: Required layout after the barrier.

SubresourceRange range: Mip levels and array layers synchronized by the barrier.

struct BufferBarrierInfo : public mlsdk::scenariorunner::BaseBarrierInfo 

Barrier dependencies for a byte range within a buffer.

Public Members

BufferId buffer = {0}: Buffer synchronized by the barrier.

uint64_t offset = {}: First byte synchronized by the barrier.

uint64_t size = {}: Number of bytes synchronized by the barrier.

struct TensorBarrierInfo : public mlsdk::scenariorunner::BaseBarrierInfo 

Barrier dependencies for a tensor.

Public Members

TensorId tensor = {0}: Tensor synchronized by the barrier.

struct MemoryBarrierInfo : public mlsdk::scenariorunner::BaseBarrierInfo : Global memory barrier dependencies.

Runtime options 

struct ScenarioOptions

Options that are passed for configuring a scenario.

Public Functions

inline bool shouldDumpNeuralDebugDatabase() const: Return whether neural debug database output is enabled.

inline bool shouldDumpNeuralStatistics() const: Return whether neural accelerator statistics output is enabled.

inline bool shouldDumpGraphProfiling() const: Return whether data graph profiling output is enabled.

Public Members

bool enablePipelineCaching = {false}: Enable Vulkan® pipeline caching.

bool clearPipelineCache = {false}: Remove existing pipeline cache contents before setup.

bool failOnPipelineCacheMiss = {false}: Fail setup when a required pipeline is absent from the cache.

bool enableGPUDebugMarkers = {false}: Emit GPU debug markers when supported.

bool captureFrame = {false}: Capture a frame using the configured capture integration.

bool enableRobustnessFeatures = {false}: Enable supported Vulkan® robustness features.

std::filesystem::path pipelineCachePath: Pipeline cache input and output path.

std::filesystem::path neuralDebugDatabaseDumpDir: Directory for neural debug database output.

std::filesystem::path neuralStatisticsDumpDir: Directory for neural accelerator statistics output.

std::filesystem::path graphProfilingDumpDir: Directory for data graph profiling output.

std::filesystem::path sessionRAMsDumpDir: Directory for session RAM output.

std::filesystem::path perfCountersPath: Performance counter configuration path.

std::filesystem::path profilingPath: Scenario profiling output path.

std::vector<std::string> disabledExtensions: Vulkan® device extensions to leave disabled.

vk::NeuralAcceleratorStatisticsModeARM neuralStatisticsMode = {}: Neural accelerator statistics collection mode.