gt4py.cartesian.backend package

Submodules

gt4py.cartesian.backend.base module

class gt4py.cartesian.backend.base.Backend(builder: StencilBuilder)

Bases: abc.ABC

builder: StencilBuilder

property extra_cache_info: Dict[str, Any]

Provide additional data to be stored in cache info file (sublass hook).

property extra_cache_validation_keys: List[str]

List keys from extra_cache_info to be validated during consistency check.

classmethod filter_options_for_id(options: gt4py.cartesian.definitions.BuildOptions) → gt4py.cartesian.definitions.BuildOptions

Filter a copy for options that should not be part of the stencil ID.

abstract generate() → Type[StencilObject]

Generate the stencil class from GTScript’s internal representation.

Returns

• type – The generated stencil class after loading through python’s import API

• In case the stencil class for the given ID is found in cache, this method can avoid

• rebuilding it. Rebuilding can however be triggered through the options parameter

• if supported by the backend.

languages: ClassVar[Optional[Dict[str, Any]]] = None

Language support:

• “computation”: name of the language in which the computation is implemented.

• “bindings”: names of supported language bindings / wrappers. If a high-level language is compatible out-of-the-box, it should not be listed.

Languages should be spelled using the official spelling but lower case (“python”, “fortran”, “rust”).

abstract load() → Optional[Type[StencilObject]]

Load the stencil class from the generated python module.

type:

The generated stencil class after loading through python’s import API

This assumes that Backend.generate() has been called already on the same stencil. The python module therefore already exists and any python extensions have been compiled. In case the stencil changed since the last generate call, it will not be rebuilt.

name: ClassVar[str]

Backend name

options: ClassVar[Dict[str, Any]]

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

storage_info: ClassVar[gt4py.storage.layout.LayoutInfo]

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.base.BaseBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.base.Backend

MODULE_GENERATOR_CLASS: ClassVar[Type[BaseModuleGenerator]]

check_options(options: gt4py.cartesian.definitions.BuildOptions) → None

generate() → Type[StencilObject]

Generate the stencil class from GTScript’s internal representation.

Returns

• type – The generated stencil class after loading through python’s import API

• In case the stencil class for the given ID is found in cache, this method can avoid

• rebuilding it. Rebuilding can however be triggered through the options parameter

• if supported by the backend.

load() → Optional[Type[StencilObject]]

Load the stencil class from the generated python module.

type:

The generated stencil class after loading through python’s import API

This assumes that Backend.generate() has been called already on the same stencil. The python module therefore already exists and any python extensions have been compiled. In case the stencil changed since the last generate call, it will not be rebuilt.

make_module(**kwargs: Any) → Type[StencilObject]

make_module_source(*, args_data: Optional[gt4py.cartesian.backend.module_generator.ModuleData] = None, **kwargs: Any) → str

Generate the module source code with or without stencil id.

class gt4py.cartesian.backend.base.BasePyExtBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.base.BaseBackend

MODULE_GENERATOR_CLASS: ClassVar[Type[BaseModuleGenerator]]

build_extension_module(pyext_sources: Dict[str, Any], pyext_build_opts: Dict[str, str], *, uses_cuda: bool = False) → Tuple[str, str]

builder: StencilBuilder

property extra_cache_info: Dict[str, Any]

Provide additional data to be stored in cache info file (sublass hook).

property extra_cache_validation_keys: List[str]

List keys from extra_cache_info to be validated during consistency check.

abstract generate() → Type[StencilObject]

Generate the stencil class from GTScript’s internal representation.

Returns

• type – The generated stencil class after loading through python’s import API

• In case the stencil class for the given ID is found in cache, this method can avoid

• rebuilding it. Rebuilding can however be triggered through the options parameter

• if supported by the backend.

name: ClassVar[str]

Backend name

options: ClassVar[Dict[str, Any]]

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

property pyext_build_dir_path: pathlib.Path

property pyext_class_name: str

property pyext_module_name: str

property pyext_module_path: str

storage_info: ClassVar[gt_storage.layout.LayoutInfo]

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.base.CLIBackendMixin(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.base.Backend

builder: StencilBuilder

abstract generate_bindings(language_name: str) → Dict[str, Union[str, Dict]]

Generate bindings source code from language_name to the target language of the backend.

Returns

• Analog to generate_computation() but containing bindings source code, The

• dictionary contains a tree of directories with leaves being a mapping from filename to

• source code pairs, relative to the build directory.

Raises

RuntimeError – If the backend does not support the bindings language

abstract generate_computation() → Dict[str, Union[str, Dict]]

Generate the computation source code in a way agnostic of the way it is going to be used.

Returns

If a key’s value is a string it is interpreted as a file name and the value as the source code of that file If a key’s value is a Dict, it is interpreted as a directory name and it’s value as a nested file hierarchy to which the same rules are applied recursively. The root path is relative to the build directory.

Return type

Dict[str, str | Dict] of source file names / directories -> contents

Raises

NotImplementedError – If the backend does not support usage outside of JIT compilation / generation.

Example

def mystencil(...):
    ...

options = BuildOptions(name="mystencil", ...)
ir = frontend.generate(mystencil, {}, options)
stencil_src = backend.generate_computation(ir, options)

print(stencil_src)

# this might be output from a fictional backend:
{
    "mystencil_project": {
        "src": {
            "stencil.cpp",
            "helpers.cpp"
        },
        "include": {
            "stencil.hpp"
        },
    }
}

This can now be automatically be turned into a folder hierarchy that makes sense and can be incorporated into an external build system.

name: ClassVar[str]

Backend name

options: ClassVar[Dict[str, Any]]

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

storage_info: ClassVar[gt4py.storage.layout.LayoutInfo]

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.base.MakeModuleSourceCallable(*args, **kwargs)

Bases: Protocol

__call__(*, args_data: Optional[gt4py.cartesian.backend.module_generator.ModuleData] = None, **kwargs: Any) → str

Call self as a function.

class gt4py.cartesian.backend.base.PurePythonBackendCLIMixin(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.base.CLIBackendMixin

Mixin for CLI support for backends deriving from BaseBackend.

builder: StencilBuilder

generate_bindings(language_name: str) → Dict[str, Union[str, Dict]]

Pure python backends typically will not support bindings.

generate_computation() → Dict[str, Union[str, Dict]]

Generate the computation source code in a way agnostic of the way it is going to be used.

Returns

If a key’s value is a string it is interpreted as a file name and the value as the source code of that file If a key’s value is a Dict, it is interpreted as a directory name and it’s value as a nested file hierarchy to which the same rules are applied recursively. The root path is relative to the build directory.

Return type

Dict[str, str | Dict] of source file names / directories -> contents

Raises

NotImplementedError – If the backend does not support usage outside of JIT compilation / generation.

Example

def mystencil(...):
    ...

options = BuildOptions(name="mystencil", ...)
ir = frontend.generate(mystencil, {}, options)
stencil_src = backend.generate_computation(ir, options)

print(stencil_src)

# this might be output from a fictional backend:
{
    "mystencil_project": {
        "src": {
            "stencil.cpp",
            "helpers.cpp"
        },
        "include": {
            "stencil.hpp"
        },
    }
}

This can now be automatically be turned into a folder hierarchy that makes sense and can be incorporated into an external build system.

make_module_source: gt4py.cartesian.backend.base.MakeModuleSourceCallable

stencil python source generator method: In order to use this mixin, the backend class must implement a make_module_source() method or derive from BaseBackend().

gt4py.cartesian.backend.base.from_name(name: str) → Optional[Type[gt4py.cartesian.backend.base.Backend]]

gt4py.cartesian.backend.base.register(backend_cls: Type[gt4py.cartesian.backend.base.Backend]) → None

gt4py.cartesian.backend.cuda_backend module

class gt4py.cartesian.backend.cuda_backend.CudaBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.gtc_common.BaseGTBackend, gt4py.cartesian.backend.base.CLIBackendMixin

CUDA backend using gtc.

GT_BACKEND_T: str = 'gpu'

MODULE_GENERATOR_CLASS

alias of gt4py.cartesian.backend.gtc_common.CUDAPyExtModuleGenerator

PYEXT_GENERATOR_CLASS

alias of gt4py.cartesian.backend.cuda_backend.CudaExtGenerator

generate() → Type[StencilObject]

Generate the stencil class from GTScript’s internal representation.

Returns

• type – The generated stencil class after loading through python’s import API

• In case the stencil class for the given ID is found in cache, this method can avoid

• rebuilding it. Rebuilding can however be triggered through the options parameter

• if supported by the backend.

generate_extension(**kwargs: Any) → Tuple[str, str]

Generate and build a python extension for the stencil computation.

Returns the name and file path (as string) of the compiled extension “.so” module.

languages: ClassVar[Optional[Dict[str, Any]]] = {'bindings': ['python'], 'computation': 'cuda'}

Language support:

• “computation”: name of the language in which the computation is implemented.

• “bindings”: names of supported language bindings / wrappers. If a high-level language is compatible out-of-the-box, it should not be listed.

Languages should be spelled using the official spelling but lower case (“python”, “fortran”, “rust”).

name: ClassVar[str] = 'cuda'

Backend name

options: ClassVar[Dict[str, Any]] = {'add_profile_info': {'type': <class 'bool'>, 'versioning': True}, 'clean': {'type': <class 'bool'>, 'versioning': False}, 'debug_mode': {'type': <class 'bool'>, 'versioning': True}, 'device_sync': {'type': <class 'bool'>, 'versioning': True}, 'oir_pipeline': {'type': <class 'gt4py.cartesian.gtc.passes.oir_pipeline.OirPipeline'>, 'versioning': True}, 'verbose': {'type': <class 'bool'>, 'versioning': False}}

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

storage_info: ClassVar[gt_storage.layout.LayoutInfo] = {'alignment': 32, 'device': 'gpu', 'is_optimal_layout': <function layout_checker_factory.<locals>.layout_checker>, 'layout_map': <function make_cuda_layout_map>}

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.cuda_backend.CudaBindingsCodegen(backend)

Bases: gt4py.eve.codegen.TemplatedGenerator

Program = <gt4py.eve.codegen.MakoTemplate object>

classmethod apply_codegen(root, *, module_name='stencil', backend, **kwargs) → str

unique_index() → int

visit_DataType(dtype: gt4py.cartesian.gtc.common.DataType, **kwargs)

visit_FieldDecl(node: gt4py.cartesian.gtc.cuir.cuir.FieldDecl, **kwargs)

visit_Program(node: gt4py.cartesian.gtc.cuir.cuir.Program, **kwargs)

visit_ScalarDecl(node: gt4py.cartesian.gtc.cuir.cuir.ScalarDecl, **kwargs)

class gt4py.cartesian.backend.cuda_backend.CudaExtGenerator(class_name, module_name, backend)

Bases: gt4py.cartesian.backend.gtc_common.BackendCodegen

TEMPLATE_FILES: Dict[str, str]

__call__(stencil_ir: gt4py.cartesian.gtc.gtir.Stencil) → Dict[str, Dict[str, str]]

Return a dict with the keys ‘computation’ and ‘bindings’ to dicts of filenames to source.

gt4py.cartesian.backend.dace_backend module

class gt4py.cartesian.backend.dace_backend.BaseDaceBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.gtc_common.BaseGTBackend, gt4py.cartesian.backend.base.CLIBackendMixin

GT_BACKEND_T: str = 'dace'

PYEXT_GENERATOR_CLASS

alias of gt4py.cartesian.backend.dace_backend.DaCeExtGenerator

generate() → Type[StencilObject]

Generate the stencil class from GTScript’s internal representation.

Returns

• type – The generated stencil class after loading through python’s import API

• In case the stencil class for the given ID is found in cache, this method can avoid

• rebuilding it. Rebuilding can however be triggered through the options parameter

• if supported by the backend.

class gt4py.cartesian.backend.dace_backend.DaCeBindingsCodegen(backend)

Bases: object

classmethod apply(stencil_ir: gt4py.cartesian.gtc.gtir.Stencil, sdfg: dace.sdfg.sdfg.SDFG, module_name: str, *, backend) → str

generate_entry_params(stencil_ir: gt4py.cartesian.gtc.gtir.Stencil, sdfg: dace.sdfg.sdfg.SDFG) → List[str]

generate_sdfg_bindings(stencil_ir: gt4py.cartesian.gtc.gtir.Stencil, sdfg, module_name) → str

generate_sid_params(sdfg: dace.sdfg.sdfg.SDFG) → List[str]

mako_template = <gt4py.eve.codegen.MakoTemplate object>

unique_index() → int

class gt4py.cartesian.backend.dace_backend.DaCeCUDAPyExtModuleGenerator

Bases: gt4py.cartesian.backend.dace_backend.DaCePyExtModuleGenerator, gt4py.cartesian.backend.gtc_common.CUDAPyExtModuleGenerator

pyext_file_path: Optional[str]

pyext_module_name: Optional[str]

class gt4py.cartesian.backend.dace_backend.DaCeComputationCodegen

Bases: object

classmethod apply(stencil_ir: gt4py.cartesian.gtc.gtir.Stencil, builder: StencilBuilder, sdfg: dace.sdfg.sdfg.SDFG)

generate_dace_args(stencil_ir: gt4py.cartesian.gtc.gtir.Stencil, sdfg: dace.sdfg.sdfg.SDFG) → List[str]

generate_functor_args(sdfg: dace.sdfg.sdfg.SDFG)

generate_tmp_allocs(sdfg)

template = <gt4py.eve.codegen.MakoTemplate object>

class gt4py.cartesian.backend.dace_backend.DaCeExtGenerator(class_name, module_name, backend)

Bases: gt4py.cartesian.backend.gtc_common.BackendCodegen

TEMPLATE_FILES: Dict[str, str]

__call__(stencil_ir: gt4py.cartesian.gtc.gtir.Stencil) → Dict[str, Dict[str, str]]

Return a dict with the keys ‘computation’ and ‘bindings’ to dicts of filenames to source.

class gt4py.cartesian.backend.dace_backend.DaCePyExtModuleGenerator

Bases: gt4py.cartesian.backend.gtc_common.PyExtModuleGenerator

generate_base_class_name()

generate_class_members()

Generate additional stencil class members.

May contain any class level code including methods.

generate_imports()

Generate import statements and related code for the stencil class module.

pyext_file_path: Optional[str]

pyext_module_name: Optional[str]

class gt4py.cartesian.backend.dace_backend.DaceCPUBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.dace_backend.BaseDaceBackend

MODULE_GENERATOR_CLASS

alias of gt4py.cartesian.backend.dace_backend.DaCePyExtModuleGenerator

builder: StencilBuilder

generate_extension(**kwargs: Any) → Tuple[str, str]

Generate and build a python extension for the stencil computation.

Returns the name and file path (as string) of the compiled extension “.so” module.

languages: ClassVar[Optional[Dict[str, Any]]] = {'bindings': ['python'], 'computation': 'c++'}

Language support:

• “computation”: name of the language in which the computation is implemented.

• “bindings”: names of supported language bindings / wrappers. If a high-level language is compatible out-of-the-box, it should not be listed.

Languages should be spelled using the official spelling but lower case (“python”, “fortran”, “rust”).

name: ClassVar[str] = 'dace:cpu'

Backend name

options: ClassVar[Dict[str, Any]] = {'add_profile_info': {'type': <class 'bool'>, 'versioning': True}, 'clean': {'type': <class 'bool'>, 'versioning': False}, 'debug_mode': {'type': <class 'bool'>, 'versioning': True}, 'oir_pipeline': {'type': <class 'gt4py.cartesian.gtc.passes.oir_pipeline.OirPipeline'>, 'versioning': True}, 'verbose': {'type': <class 'bool'>, 'versioning': False}}

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

storage_info: ClassVar[gt_storage.layout.LayoutInfo] = {'alignment': 1, 'device': 'cpu', 'is_optimal_layout': <function layout_checker_factory.<locals>.layout_checker>, 'layout_map': <function layout_maker_factory.<locals>.layout_maker>}

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.dace_backend.DaceGPUBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.dace_backend.BaseDaceBackend

DaCe python backend using gt4py.cartesian.gtc.

MODULE_GENERATOR_CLASS

alias of gt4py.cartesian.backend.dace_backend.DaCeCUDAPyExtModuleGenerator

builder: StencilBuilder

generate_extension(**kwargs: Any) → Tuple[str, str]

Generate and build a python extension for the stencil computation.

Returns the name and file path (as string) of the compiled extension “.so” module.

languages: ClassVar[Optional[Dict[str, Any]]] = {'bindings': ['python'], 'computation': 'cuda'}

Language support:

• “computation”: name of the language in which the computation is implemented.

• “bindings”: names of supported language bindings / wrappers. If a high-level language is compatible out-of-the-box, it should not be listed.

Languages should be spelled using the official spelling but lower case (“python”, “fortran”, “rust”).

name: ClassVar[str] = 'dace:gpu'

Backend name

options: ClassVar[Dict[str, Any]] = {'add_profile_info': {'type': <class 'bool'>, 'versioning': True}, 'clean': {'type': <class 'bool'>, 'versioning': False}, 'debug_mode': {'type': <class 'bool'>, 'versioning': True}, 'device_sync': {'type': <class 'bool'>, 'versioning': True}, 'oir_pipeline': {'type': <class 'gt4py.cartesian.gtc.passes.oir_pipeline.OirPipeline'>, 'versioning': True}, 'verbose': {'type': <class 'bool'>, 'versioning': False}}

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

storage_info: ClassVar[gt_storage.layout.LayoutInfo] = {'alignment': 32, 'device': 'gpu', 'is_optimal_layout': <function layout_checker_factory.<locals>.layout_checker>, 'layout_map': <function layout_maker_factory.<locals>.layout_maker>}

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.dace_backend.SDFGManager(builder)

Bases: object

expanded_sdfg()

frozen_sdfg(*, origin: Dict[str, Tuple[int, ...]], domain: Tuple[int, ...])

unexpanded_sdfg()

gt4py.cartesian.backend.dace_backend._to_device(sdfg: dace.sdfg.sdfg.SDFG, device: str) → None

Update sdfg in place.

gt4py.cartesian.backend.dace_backend.freeze_origin_domain_sdfg(inner_sdfg, arg_names, field_info, *, origin, domain)

gt4py.cartesian.backend.dace_lazy_stencil module

class gt4py.cartesian.backend.dace_lazy_stencil.DaCeLazyStencil(builder: StencilBuilder)

Bases: gt4py.cartesian.lazy_stencil.LazyStencil, dace.frontend.python.common.SDFGConvertible

closure_resolver(constant_args: Dict[str, Any], given_args: Set[str], parent_closure: Optional[dace.frontend.python.common.SDFGClosure] = None) → dace.frontend.python.common.SDFGClosure

Returns an SDFGClosure object representing the closure of the object to be converted to an SDFG. :param constant_args: Arguments whose values are already resolved to

compile-time values.

Parameters

• given_args – Arguments that were given at call-time (used for determining which arguments with defaults were provided).

• parent_closure – The parent SDFGClosure object (used for, e.g., recursion detection).

Returns

New SDFG closure object representing the convertible object.

property field_info: Dict[str, Any]

Return same value as compiled stencil object’s field_info attribute.

Does not trigger a build.

gt4py.cartesian.backend.dace_stencil_object module

class gt4py.cartesian.backend.dace_stencil_object.DaCeFrozenStencil(stencil_object: 'DaCeStencilObject', origin: Dict[str, Tuple[int, ...]], domain: Tuple[int, ...], sdfg: dace.sdfg.sdfg.SDFG)

Bases: gt4py.cartesian.stencil_object.FrozenStencil, dace.frontend.python.common.SDFGConvertible

closure_resolver(constant_args, given_args, parent_closure=None)

Returns an SDFGClosure object representing the closure of the object to be converted to an SDFG. :param constant_args: Arguments whose values are already resolved to

compile-time values.

Parameters

• given_args – Arguments that were given at call-time (used for determining which arguments with defaults were provided).

• parent_closure – The parent SDFGClosure object (used for, e.g., recursion detection).

Returns

New SDFG closure object representing the convertible object.

domain: Tuple[int, ...]

origin: Dict[str, Tuple[int, ...]]

sdfg: dace.sdfg.sdfg.SDFG

stencil_object: gt4py.cartesian.backend.dace_stencil_object.DaCeStencilObject

class gt4py.cartesian.backend.dace_stencil_object.DaCeStencilObject(*args, **kwargs)

Bases: gt4py.cartesian.stencil_object.StencilObject, dace.frontend.python.common.SDFGConvertible

SDFG_PATH: str

closure_resolver(constant_args: Dict[str, Any], given_args: Set[str], parent_closure: Optional[dace.frontend.python.common.SDFGClosure] = None) → dace.frontend.python.common.SDFGClosure

Returns an SDFGClosure object representing the closure of the object to be converted to an SDFG. :param constant_args: Arguments whose values are already resolved to

compile-time values.

Parameters

• given_args – Arguments that were given at call-time (used for determining which arguments with defaults were provided).

• parent_closure – The parent SDFGClosure object (used for, e.g., recursion detection).

Returns

New SDFG closure object representing the convertible object.

freeze(*, origin: Dict[str, Tuple[int, ...]], domain: Tuple[int, ...]) → gt4py.cartesian.backend.dace_stencil_object.DaCeFrozenStencil

Return a StencilObject wrapper with a fixed domain and origin for each argument.

Parameters

• origin (dict) – The origin for each Field argument.

• domain (Sequence of int) – The compute domain shape for the frozen stencil.

Notes

Both origin and domain arguments should be compatible with the domain and origin specification defined in StencilObject.

Returns

The stencil wrapper. This should be called with the regular stencil arguments, but the field origins and domain cannot be changed. Note, no checking of origin or domain occurs at call time so it is the users responsibility to ensure correct usage.

Return type

FrozenStencil

static normalize_args(*args, backend: str, arg_names: Iterable[str], domain_info: gt4py.cartesian.definitions.DomainInfo, field_info: Dict[str, gt4py.cartesian.definitions.FieldInfo], domain: Optional[Tuple[int, ...]] = None, origin: Optional[Dict[str, Tuple[int, ...]]] = None, **kwargs)

classmethod sdfg() → dace.sdfg.sdfg.SDFG

gt4py.cartesian.backend.dace_stencil_object.add_optional_fields(sdfg: dace.sdfg.sdfg.SDFG, field_info: Dict[str, Any], parameter_info: Dict[str, Any], **kwargs: Any) → dace.sdfg.sdfg.SDFG

gt4py.cartesian.backend.gtc_common module

class gt4py.cartesian.backend.gtc_common.BackendCodegen(class_name: str, module_name: str, backend: Any)

Bases: object

TEMPLATE_FILES: Dict[str, str]

abstract __call__(ir: gt4py.cartesian.gtc.gtir.Stencil) → Dict[str, Dict[str, str]]

Return a dict with the keys ‘computation’ and ‘bindings’ to dicts of filenames to source.

class gt4py.cartesian.backend.gtc_common.BaseGTBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.base.BasePyExtBackend, gt4py.cartesian.backend.base.CLIBackendMixin

GT_BACKEND_OPTS: Dict[str, Dict[str, Any]] = {'add_profile_info': {'type': <class 'bool'>, 'versioning': True}, 'clean': {'type': <class 'bool'>, 'versioning': False}, 'debug_mode': {'type': <class 'bool'>, 'versioning': True}, 'oir_pipeline': {'type': <class 'gt4py.cartesian.gtc.passes.oir_pipeline.OirPipeline'>, 'versioning': True}, 'verbose': {'type': <class 'bool'>, 'versioning': False}}

GT_BACKEND_T: str

MODULE_GENERATOR_CLASS

alias of gt4py.cartesian.backend.gtc_common.PyExtModuleGenerator

PYEXT_GENERATOR_CLASS: Type[gt4py.cartesian.backend.gtc_common.BackendCodegen]

abstract generate() → Type[StencilObject]

Generate the stencil class from GTScript’s internal representation.

Returns

• type – The generated stencil class after loading through python’s import API

• In case the stencil class for the given ID is found in cache, this method can avoid

• rebuilding it. Rebuilding can however be triggered through the options parameter

• if supported by the backend.

generate_bindings(language_name: str, *, stencil_ir: Optional[gt4py.cartesian.gtc.gtir.Stencil] = None) → Dict[str, Union[str, Dict]]

Generate bindings source code from language_name to the target language of the backend.

Returns

• Analog to generate_computation() but containing bindings source code, The

• dictionary contains a tree of directories with leaves being a mapping from filename to

• source code pairs, relative to the build directory.

Raises

RuntimeError – If the backend does not support the bindings language

generate_computation() → Dict[str, Union[str, Dict]]

Generate the computation source code in a way agnostic of the way it is going to be used.

Returns

If a key’s value is a string it is interpreted as a file name and the value as the source code of that file If a key’s value is a Dict, it is interpreted as a directory name and it’s value as a nested file hierarchy to which the same rules are applied recursively. The root path is relative to the build directory.

Return type

Dict[str, str | Dict] of source file names / directories -> contents

Raises

NotImplementedError – If the backend does not support usage outside of JIT compilation / generation.

Example

def mystencil(...):
    ...

options = BuildOptions(name="mystencil", ...)
ir = frontend.generate(mystencil, {}, options)
stencil_src = backend.generate_computation(ir, options)

print(stencil_src)

# this might be output from a fictional backend:
{
    "mystencil_project": {
        "src": {
            "stencil.cpp",
            "helpers.cpp"
        },
        "include": {
            "stencil.hpp"
        },
    }
}

This can now be automatically be turned into a folder hierarchy that makes sense and can be incorporated into an external build system.

abstract generate_extension(**kwargs: Any) → Tuple[str, str]

Generate and build a python extension for the stencil computation.

Returns the name and file path (as string) of the compiled extension “.so” module.

make_extension(*, stencil_ir: Optional[gt4py.cartesian.gtc.gtir.Stencil] = None, uses_cuda: bool = False) → Tuple[str, str]

make_extension_sources(*, stencil_ir: gt4py.cartesian.gtc.gtir.Stencil) → Dict[str, Dict[str, str]]

Generate the source for the stencil independently from use case.

class gt4py.cartesian.backend.gtc_common.CUDAPyExtModuleGenerator

Bases: gt4py.cartesian.backend.gtc_common.PyExtModuleGenerator

generate_implementation() → str

Generate the work code inside the stencil object’s run function.

generate_imports() → str

Generate import statements and related code for the stencil class module.

pyext_file_path: Optional[str]

pyext_module_name: Optional[str]

class gt4py.cartesian.backend.gtc_common.PyExtModuleGenerator

Bases: gt4py.cartesian.backend.module_generator.BaseModuleGenerator

Module Generator for use with backends that generate c++ python extensions.

__call__(args_data: gt4py.cartesian.backend.module_generator.ModuleData, builder: Optional[StencilBuilder] = None, **kwargs: Any) → str

Generate source code for a Python module containing a StencilObject.

A possible reaosn for extending is processing additional kwargs, using a different template might require completely overriding.

generate_implementation() → str

Generate the work code inside the stencil object’s run function.

generate_imports() → str

Generate import statements and related code for the stencil class module.

pyext_file_path: Optional[str]

pyext_module_name: Optional[str]

gt4py.cartesian.backend.gtc_common.bindings_main_template()

gt4py.cartesian.backend.gtc_common.gtir_has_effect(pipeline: gt4py.cartesian.gtc.passes.gtir_pipeline.GtirPipeline) → bool

gt4py.cartesian.backend.gtc_common.gtir_is_not_empty(pipeline: gt4py.cartesian.gtc.passes.gtir_pipeline.GtirPipeline) → bool

gt4py.cartesian.backend.gtc_common.pybuffer_to_sid(*, name: str, ctype: str, domain_dim_flags: Tuple[bool, bool, bool], data_ndim: int, stride_kind_index: int, backend: gt4py.cartesian.backend.base.Backend)

gt4py.cartesian.backend.gtcpp_backend module

class gt4py.cartesian.backend.gtcpp_backend.GTBaseBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.gtc_common.BaseGTBackend, gt4py.cartesian.backend.base.CLIBackendMixin

GT_BACKEND_T: str

PYEXT_GENERATOR_CLASS

alias of gt4py.cartesian.backend.gtcpp_backend.GTExtGenerator

builder: StencilBuilder

generate() → Type[StencilObject]

Generate the stencil class from GTScript’s internal representation.

Returns

• type – The generated stencil class after loading through python’s import API

• In case the stencil class for the given ID is found in cache, this method can avoid

• rebuilding it. Rebuilding can however be triggered through the options parameter

• if supported by the backend.

name: ClassVar[str]

Backend name

options: ClassVar[Dict[str, Any]] = {'add_profile_info': {'type': <class 'bool'>, 'versioning': True}, 'clean': {'type': <class 'bool'>, 'versioning': False}, 'debug_mode': {'type': <class 'bool'>, 'versioning': True}, 'oir_pipeline': {'type': <class 'gt4py.cartesian.gtc.passes.oir_pipeline.OirPipeline'>, 'versioning': True}, 'verbose': {'type': <class 'bool'>, 'versioning': False}}

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

storage_info: ClassVar[gt_storage.layout.LayoutInfo]

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.gtcpp_backend.GTCppBindingsCodegen(backend)

Bases: gt4py.eve.codegen.TemplatedGenerator

Program = <gt4py.eve.codegen.MakoTemplate object>

classmethod apply(root, *, module_name='stencil', **kwargs) → str

Public method to build a class instance and visit an IR node.

Parameters

• root – An IR node.

• node_templates (optiona) – see NodeDumper.

• dump_function (optiona) – see NodeDumper.

• **kwargs (optional) – custom extra parameters forwarded to visit_NODE_TYPE_NAME().

Returns

String (or collection of strings) with the dumped version of the root IR node.

unique_index() → int

visit_DataType(dtype: gt4py.cartesian.gtc.common.DataType, **kwargs)

visit_FieldDecl(node: gt4py.cartesian.gtc.gtcpp.gtcpp.FieldDecl, **kwargs)

visit_GlobalParamDecl(node: gt4py.cartesian.gtc.gtcpp.gtcpp.GlobalParamDecl, **kwargs)

visit_Program(node: gt4py.cartesian.gtc.gtcpp.gtcpp.Program, **kwargs)

class gt4py.cartesian.backend.gtcpp_backend.GTCpuIfirstBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.gtcpp_backend.GTBaseBackend

GridTools python backend using gtc.

GT_BACKEND_T: str = 'cpu_ifirst'

builder: StencilBuilder

generate_extension(**kwargs: Any) → Tuple[str, str]

Generate and build a python extension for the stencil computation.

Returns the name and file path (as string) of the compiled extension “.so” module.

languages: ClassVar[Optional[Dict[str, Any]]] = {'bindings': ['python'], 'computation': 'c++'}

Language support:

• “computation”: name of the language in which the computation is implemented.

• “bindings”: names of supported language bindings / wrappers. If a high-level language is compatible out-of-the-box, it should not be listed.

Languages should be spelled using the official spelling but lower case (“python”, “fortran”, “rust”).

name: ClassVar[str] = 'gt:cpu_ifirst'

Backend name

storage_info: ClassVar[gt_storage.layout.LayoutInfo] = {'alignment': 8, 'device': 'cpu', 'is_optimal_layout': <function layout_checker_factory.<locals>.layout_checker>, 'layout_map': <function make_gtcpu_ifirst_layout_map>}

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.gtcpp_backend.GTCpuKfirstBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.gtcpp_backend.GTBaseBackend

GridTools python backend using gtc.

GT_BACKEND_T: str = 'cpu_kfirst'

builder: StencilBuilder

generate_extension(**kwargs: Any) → Tuple[str, str]

Generate and build a python extension for the stencil computation.

Returns the name and file path (as string) of the compiled extension “.so” module.

languages: ClassVar[Optional[Dict[str, Any]]] = {'bindings': ['python'], 'computation': 'c++'}

Language support:

• “computation”: name of the language in which the computation is implemented.

• “bindings”: names of supported language bindings / wrappers. If a high-level language is compatible out-of-the-box, it should not be listed.

Languages should be spelled using the official spelling but lower case (“python”, “fortran”, “rust”).

name: ClassVar[str] = 'gt:cpu_kfirst'

Backend name

storage_info: ClassVar[gt_storage.layout.LayoutInfo] = {'alignment': 1, 'device': 'cpu', 'is_optimal_layout': <function layout_checker_factory.<locals>.layout_checker>, 'layout_map': <function make_gtcpu_kfirst_layout_map>}

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

class gt4py.cartesian.backend.gtcpp_backend.GTExtGenerator(class_name, module_name, backend)

Bases: gt4py.cartesian.backend.gtc_common.BackendCodegen

TEMPLATE_FILES: Dict[str, str]

__call__(stencil_ir: gt4py.cartesian.gtc.gtir.Stencil) → Dict[str, Dict[str, str]]

Return a dict with the keys ‘computation’ and ‘bindings’ to dicts of filenames to source.

class gt4py.cartesian.backend.gtcpp_backend.GTGpuBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.gtcpp_backend.GTBaseBackend

GridTools python backend using gtc.

GT_BACKEND_T: str = 'gpu'

MODULE_GENERATOR_CLASS

alias of gt4py.cartesian.backend.gtc_common.CUDAPyExtModuleGenerator

builder: StencilBuilder

generate_extension(**kwargs: Any) → Tuple[str, str]

Generate and build a python extension for the stencil computation.

Returns the name and file path (as string) of the compiled extension “.so” module.

languages: ClassVar[Optional[Dict[str, Any]]] = {'bindings': ['python'], 'computation': 'cuda'}

Language support:

• “computation”: name of the language in which the computation is implemented.

• “bindings”: names of supported language bindings / wrappers. If a high-level language is compatible out-of-the-box, it should not be listed.

Languages should be spelled using the official spelling but lower case (“python”, “fortran”, “rust”).

name: ClassVar[str] = 'gt:gpu'

Backend name

options: ClassVar[Dict[str, Any]] = {'add_profile_info': {'type': <class 'bool'>, 'versioning': True}, 'clean': {'type': <class 'bool'>, 'versioning': False}, 'debug_mode': {'type': <class 'bool'>, 'versioning': True}, 'device_sync': {'type': <class 'bool'>, 'versioning': True}, 'oir_pipeline': {'type': <class 'gt4py.cartesian.gtc.passes.oir_pipeline.OirPipeline'>, 'versioning': True}, 'verbose': {'type': <class 'bool'>, 'versioning': False}}

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

storage_info: ClassVar[gt_storage.layout.LayoutInfo] = {'alignment': 32, 'device': 'gpu', 'is_optimal_layout': <function layout_checker_factory.<locals>.layout_checker>, 'layout_map': <function make_cuda_layout_map>}

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

gt4py.cartesian.backend.module_generator module

class gt4py.cartesian.backend.module_generator.BaseModuleGenerator(builder: Optional[StencilBuilder] = None)

Bases: abc.ABC

DOMAIN_ARG_NAME = '_domain_'

ORIGIN_ARG_NAME = '_origin_'

SOURCE_LINE_LENGTH = 120

SPLITTERS_NAME = '_splitters_'

TEMPLATE_INDENT_SIZE = 4

TEMPLATE_PATH = '/workspace/gt4py/src/gt4py/cartesian/backend/templates/stencil_module.py.in'

__call__(args_data: gt4py.cartesian.backend.module_generator.ModuleData, builder: Optional[StencilBuilder] = None, **kwargs: Any) → str

Generate source code for a Python module containing a StencilObject.

A possible reaosn for extending is processing additional kwargs, using a different template might require completely overriding.

args_data: gt4py.cartesian.backend.module_generator.ModuleData

property backend_name: str

property builder: StencilBuilder

Expose the builder reference.

Raises a runtime error if the builder reference is not initialized. This is necessary because other parts of the public API depend on it before it is guaranteed to be initialized.

generate_backend_name() → str

Return the name of the backend.

There should never be a need to override this.

generate_base_class_name() → str

generate_class_members() → str

Generate additional stencil class members.

May contain any class level code including methods.

generate_class_name() → str

Generate the name of the stencil class.

This should ususally be deferred to the chosen caching strategy via the builder object (see default implementation).

generate_constants() → Dict[str, str]

Return a mapping of named numeric constants passed as externals.

This is unlikely to require overriding.

generate_docstring() → str

Generate the docstring of the stencil object.

The default is to return the stencil definition’s docstring or an empty string. The output should be least based on the stencil definition’s docstring, if one exists.

abstract generate_implementation() → str

Generate the work code inside the stencil object’s run function.

generate_imports() → str

Generate import statements and related code for the stencil class module.

generate_module_members() → str

Generate additional module level code after all imports.

May contain any executable module level code including function and class defs.

generate_options() → Dict[str, Any]

Return dictionary of build options.

Must exclude options that should never be cached.

generate_post_run() → str

Additional code to be run just after the run method (implementation) is called.

generate_pre_run() → str

Additional code to be run just before the run method (implementation) is called.

generate_signature() → str

Generate the stencil definition specific part of the stencil object’s __call__ signature.

Unlikely to require overriding.

generate_sources() → Dict[str, str]

Return the source code of the stencil definition in string format.

This is unlikely to require overriding.

template: jinja2.environment.Template

class gt4py.cartesian.backend.module_generator.ModuleData(domain_info: Optional[gt4py.cartesian.definitions.DomainInfo] = None, field_info: Dict[str, gt4py.cartesian.definitions.FieldInfo] = <factory>, parameter_info: Dict[str, gt4py.cartesian.definitions.ParameterInfo] = <factory>, unreferenced: List[str] = <factory>)

Bases: object

domain_info: Optional[gt4py.cartesian.definitions.DomainInfo] = None

field_info: Dict[str, gt4py.cartesian.definitions.FieldInfo]

property field_names: Set[str]

Set of all field names.

parameter_info: Dict[str, gt4py.cartesian.definitions.ParameterInfo]

property parameter_names: Set[str]

Set of all parameter names.

unreferenced: List[str]

gt4py.cartesian.backend.module_generator.make_args_data_from_gtir(pipeline: gt4py.cartesian.gtc.passes.gtir_pipeline.GtirPipeline) → gt4py.cartesian.backend.module_generator.ModuleData

Compute module data containing information about stencil arguments from gtir.

This is no longer compatible with the legacy backends.

gt4py.cartesian.backend.numpy_backend module

class gt4py.cartesian.backend.numpy_backend.ModuleGenerator(builder: Optional[StencilBuilder] = None)

Bases: gt4py.cartesian.backend.module_generator.BaseModuleGenerator

args_data: gt4py.cartesian.backend.module_generator.ModuleData

property backend: gt4py.cartesian.backend.numpy_backend.NumpyBackend

generate_implementation() → str

Generate the work code inside the stencil object’s run function.

generate_imports() → str

Generate import statements and related code for the stencil class module.

template: jinja2.environment.Template

class gt4py.cartesian.backend.numpy_backend.NumpyBackend(builder: StencilBuilder)

Bases: gt4py.cartesian.backend.base.BaseBackend, gt4py.cartesian.backend.base.CLIBackendMixin

NumPy backend using gtc.

MODULE_GENERATOR_CLASS

alias of gt4py.cartesian.backend.numpy_backend.ModuleGenerator

generate() → Type[StencilObject]

Generate the stencil class from GTScript’s internal representation.

Returns

• type – The generated stencil class after loading through python’s import API

• In case the stencil class for the given ID is found in cache, this method can avoid

• rebuilding it. Rebuilding can however be triggered through the options parameter

• if supported by the backend.

generate_bindings(language_name: str) → Dict[str, Union[str, Dict]]

Generate bindings source code from language_name to the target language of the backend.

Returns

• Analog to generate_computation() but containing bindings source code, The

• dictionary contains a tree of directories with leaves being a mapping from filename to

• source code pairs, relative to the build directory.

Raises

RuntimeError – If the backend does not support the bindings language

generate_computation() → Dict[str, Union[str, Dict]]

Generate the computation source code in a way agnostic of the way it is going to be used.

Returns

If a key’s value is a string it is interpreted as a file name and the value as the source code of that file If a key’s value is a Dict, it is interpreted as a directory name and it’s value as a nested file hierarchy to which the same rules are applied recursively. The root path is relative to the build directory.

Return type

Dict[str, str | Dict] of source file names / directories -> contents

Raises

NotImplementedError – If the backend does not support usage outside of JIT compilation / generation.

Example

def mystencil(...):
    ...

options = BuildOptions(name="mystencil", ...)
ir = frontend.generate(mystencil, {}, options)
stencil_src = backend.generate_computation(ir, options)

print(stencil_src)

# this might be output from a fictional backend:
{
    "mystencil_project": {
        "src": {
            "stencil.cpp",
            "helpers.cpp"
        },
        "include": {
            "stencil.hpp"
        },
    }
}

This can now be automatically be turned into a folder hierarchy that makes sense and can be incorporated into an external build system.

languages: ClassVar[Optional[Dict[str, Any]]] = {'bindings': ['python'], 'computation': 'python'}

Language support:

• “computation”: name of the language in which the computation is implemented.

• “bindings”: names of supported language bindings / wrappers. If a high-level language is compatible out-of-the-box, it should not be listed.

Languages should be spelled using the official spelling but lower case (“python”, “fortran”, “rust”).

name: ClassVar[str] = 'numpy'

Backend name

property npir: gt4py.cartesian.gtc.numpy.npir.Computation

options: ClassVar[Dict[str, Any]] = {'ignore_np_errstate': {'type': <class 'bool'>, 'versioning': True}, 'oir_pipeline': {'type': <class 'gt4py.cartesian.gtc.passes.oir_pipeline.OirPipeline'>, 'versioning': True}}

Backend-specific options:

• info: - versioning: is versioning on? - description [optional] - type

storage_info: ClassVar[gt_storage.layout.LayoutInfo] = {'alignment': 1, 'device': 'cpu', 'is_optimal_layout': <function <lambda>>, 'layout_map': <function <lambda>>}

Backend-specific storage parametrization:

• “alignment”: in bytes

• “device”: “cpu” | “gpu”

• “layout_map”: callback converting a mask to a layout

• “is_optimal_layout”: callback checking if a storage has compatible layout

gt4py.cartesian.backend.numpy_backend.recursive_write(root_path: pathlib.Path, tree: Dict[str, Union[str, dict]])

gt4py.cartesian.backend.pyext_builder module

class gt4py.cartesian.backend.pyext_builder.CUDABuildExtension(dist, **kw)

Bases: setuptools.command.build_ext.build_ext, object

build_extensions() → None

gt4py.cartesian.backend.pyext_builder.build_pybind_cuda_ext(name: str, sources: list, build_path: str, target_path: str, **kwargs: str) → Tuple[str, str]

gt4py.cartesian.backend.pyext_builder.build_pybind_ext(name: str, sources: list, build_path: str, target_path: str, **kwargs: str) → Tuple[str, str]

gt4py.cartesian.backend.pyext_builder.get_cuda_compute_capability()

gt4py.cartesian.backend.pyext_builder.get_dace_module_path() → Optional[str]

gt4py.cartesian.backend.pyext_builder.get_gt_pyext_build_opts(*, debug_mode: bool = False, add_profile_info: bool = False, uses_openmp: bool = True, uses_cuda: bool = False, gt_version: int = 1) → Dict[str, Union[str, List[str], Dict[str, Any]]]

Module contents