Convert StableHLO/JIT To TT-IR: A Step-by-Step Guide

Aug 12, 2025 by Luna Greco 53 views

How to Convert StableHLO or JIT Functions to TT-IR Text Format

Hey guys! Ever wondered how to peek under the hood of your StableHLO or JAX jit-compiled functions and see what's really going on? Well, you're in the right place! This guide will walk you through the process of converting these functions into TT-IR (Tenstorrent Intermediate Representation) text format, making them human-readable and much easier to understand. Let's dive in!

Understanding the Need for TT-IR Conversion

Before we jump into the how-to, let’s quickly discuss why you might want to convert StableHLO or JIT functions into TT-IR. When you're working with high-performance computing, especially on specialized hardware like Tenstorrent's, understanding the intermediate representation of your code can be a game-changer. TT-IR provides a lower-level view, allowing you to:

Optimize Performance: By examining the TT-IR, you can identify potential bottlenecks and areas for optimization. This is crucial for squeezing every last bit of performance out of your hardware.
Debug More Effectively: When things go wrong, having a human-readable representation of your code's intermediate steps can make debugging much easier. You can trace the execution flow and pinpoint exactly where issues arise.
Understand Hardware Mapping: TT-IR can give you insights into how your code is being mapped onto the hardware. This understanding helps in making informed decisions about code structure and optimization strategies.
Collaboration and Sharing: Sharing TT-IR representations with other developers or experts can facilitate collaboration and knowledge sharing, as it provides a common language for discussing code transformations and optimizations.

So, whether you're a seasoned performance engineer or just starting to explore the depths of hardware-aware programming, converting to TT-IR is a valuable skill to have. Now, let’s get into the nitty-gritty of how to do it!

Prerequisites and Setup

Before we get started, there are a few things you'll need to have in place. Think of this as gathering your tools before starting a DIY project. Here’s what you should have:

Tenstorrent SDK: First and foremost, you’ll need the Tenstorrent Software Development Kit (SDK) installed. This kit provides the necessary tools and libraries to work with Tenstorrent hardware. Make sure you have the latest version installed to take advantage of the newest features and bug fixes. You can usually find installation instructions on the Tenstorrent developer website. The SDK is the backbone of your development environment, providing the compilers, debuggers, and other utilities you’ll need.
JAX (if applicable): If you’re working with JAX (which is a popular framework for numerical computation and machine learning), ensure it’s installed in your environment. JAX allows you to write NumPy-like code that can be automatically differentiated and compiled for various hardware backends, including Tenstorrent. You can install JAX using pip: pip install jax jaxlib. JAX’s ability to handle automatic differentiation and compilation makes it a powerful tool for machine learning research and development.
StableHLO: StableHLO is an intermediate representation (IR) for machine learning programs that aims to provide stability across different hardware backends. If you're starting from StableHLO, you'll need to have it available in your environment. Usually, StableHLO support comes with your JAX or other ML framework installation, but you might need to install additional packages or configure your environment to use it directly. StableHLO acts as a bridge between high-level ML frameworks and low-level hardware, ensuring portability and performance.
Python Environment: It’s always a good idea to work within a virtual environment to keep your dependencies isolated and avoid conflicts. You can create a virtual environment using tools like venv or conda. This ensures that the packages you install for this project don’t interfere with other projects on your system. A clean and isolated environment is crucial for reproducible results and easier debugging.

Once you have these prerequisites in place, you’re ready to start converting your StableHLO or JIT functions to TT-IR. Think of this setup phase as laying the foundation for your project – a solid foundation ensures a smooth and successful build!

Step-by-Step Guide to Converting to TT-IR

Alright, let's get to the main course! Here’s a step-by-step guide on how to convert your StableHLO or JIT functions to TT-IR text format. We'll break it down into manageable chunks, making it easy to follow along.

Step 1: Obtain Your StableHLO or JIT Function

First things first, you need to have the function you want to convert. This might be a StableHLO module or a JAX jit-compiled function. If you're working with JAX, you'll typically use the jax.jit decorator to compile your function. For StableHLO, you might have a pre-existing module or generate it from another framework.

JAX Example:
```
import jax
import jax.numpy as jnp

@jax.jit
def my_function(x, y):
    return jnp.sin(x) + y

x = jnp.ones((10, 10))
y = jnp.zeros((10, 10))
result = my_function(x, y)
```
In this example, my_function is jit-compiled, which means JAX will optimize it for execution on the target hardware. The @jax.jit decorator is your best friend here, as it tells JAX to perform just-in-time compilation, optimizing the function for the specific inputs and hardware.
StableHLO Example:

If you have a StableHLO module, it might look something like this (in text format):
```
module @my_module {
  func.func @main(%arg0: tensor<10x10xf32>, %arg1: tensor<10x10xf32>) -> tensor<10x10xf32> {
    %0 = stablehlo.sine %arg0 : tensor<10x10xf32>
    %1 = stablehlo.add %0, %arg1 : tensor<10x10xf32>
    return %1 : tensor<10x10xf32>
  }
}
```
This is a simple StableHLO module that adds the sine of one tensor to another. StableHLO’s text format is designed to be readable, but it’s still a low-level representation compared to Python code. The key here is to ensure you have your function or module ready to go before you start the conversion process. This initial step is crucial because it sets the stage for the rest of the conversion pipeline. Without a clear starting point, you’ll be wandering in the dark! So, make sure you have your function or module defined and accessible before moving on.

Step 2: Use Tenstorrent Tools to Convert

Now that you have your StableHLO or JIT function, the next step is to use Tenstorrent’s tools to convert it to TT-IR. The exact tooling might vary depending on your SDK version and setup, but here’s a general approach:

For JAX JIT Functions: You’ll typically use a combination of JAX and Tenstorrent-specific APIs to lower the JAX function to StableHLO and then to TT-IR. This involves tracing the JAX function and converting it through the appropriate intermediate representations.
```
import jax
import jax.numpy as jnp
from tt_jax.tt_metal import to_tt_tensor_rm

@jax.jit
def my_function(x, y):
    return jnp.sin(x) + y

x = jnp.ones((10, 10))
y = jnp.zeros((10, 10))

# Lower to StableHLO
stablehlo_module = jax.xla_computation(my_function)(x, y).as_stablehlo_module()

# Convert StableHLO module to TT-IR (this part will depend on TT SDK tools)
# Assuming a hypothetical function `stablehlo_to_ttir`
ttir_module = stablehlo_to_ttir(stablehlo_module)

# Print the TT-IR
print(ttir_module)
```
In this snippet, jax.xla_computation is used to get the StableHLO representation of the JAX function. The next step, stablehlo_to_ttir, is a placeholder for the actual Tenstorrent tool or function that performs the conversion from StableHLO to TT-IR. This is where you’ll need to consult the Tenstorrent SDK documentation for the specific command or API to use. The key here is the transition from JAX’s computational graph to a StableHLO module, which serves as a standardized format for further compilation.
For StableHLO Modules: If you already have a StableHLO module, you can directly use Tenstorrent’s tools to convert it to TT-IR. This might involve using a command-line tool or a Python API provided by the SDK.
```
# Assuming you have a StableHLO module in `stablehlo_module`
# and a hypothetical function `stablehlo_to_ttir` from TT SDK
ttir_module = stablehlo_to_ttir(stablehlo_module)

# Print the TT-IR
print(ttir_module)
```
Similar to the JAX example, stablehlo_to_ttir represents the Tenstorrent-specific tool or function. The process here is more direct since you’re starting from StableHLO. You’ll need to refer to the Tenstorrent SDK documentation to find the exact tool for this conversion. This step is crucial because it bridges the gap between a generic IR (StableHLO) and the specific hardware representation (TT-IR). Think of it as translating a document from one language to another – you’re preserving the meaning but adapting the form.

The exact commands or APIs will depend on the Tenstorrent SDK version you’re using, so make sure to check the documentation for the most accurate and up-to-date information. This conversion step is the heart of the process, transforming your high-level function into a format that’s understandable by the Tenstorrent hardware. It’s like taking a blueprint and turning it into a set of detailed instructions for construction workers.

Step 3: View the TT-IR Text Format

Once you’ve converted your function to TT-IR, the final step is to view it in a human-readable text format. This allows you to inspect the generated code, understand how your function is being translated to hardware instructions, and identify potential optimizations.

Printing the TT-IR: The simplest way to view the TT-IR is often to print it to the console. If the conversion tool provides the TT-IR as a string, you can simply use Python’s print function.
```
# Assuming `ttir_module` is a string containing the TT-IR
print(ttir_module)
```
This will display the TT-IR in your terminal, allowing you to scroll through and examine the code. It’s like reading the matrix – you’re seeing the underlying structure of your computation laid bare. The printed TT-IR will typically include information about operations, data layouts, memory access patterns, and other low-level details.
Saving to a File: For larger TT-IR modules, printing to the console might not be the most convenient way to view the code. In such cases, you can save the TT-IR to a file and open it in a text editor.
```
with open(
```