Visual Shader Scripting

For my specialization, I decided to expand the visual scripting tools I had developed for our engine to facilitate interactive and intuitive creation of shaders.

Premise

Having implemented a node scripting system into KittyEngine, creating a shader scripting tool was a compelling prospect as visual scripting tools are versatile for creation of content in games. A shader scripting tool offering artists an interface for making bespoke effects to elevate a game's quality would lend itself well to KittyEngine. One key area such a tool could elevate is the engine’s VFX system, allowing greater complexity in effects.

Implementation

This tool generates valid HLSL code from the node graph, as opposed to using existing framework solutions like DirectX11's Function Linking Graph. The aim is for the tool to be able to stand on its own as much as possible, avoiding cases where a graphics API change would entail a full rewrite. Additionally, generating code from the nodes allows for increased language adaptability.

How it works

The generation process is split into three steps: Language Definition, Script Parsing, and Shader Compilation.

Language Definition

Before any nodes can be parsed and any code generated, the system needs to know what it's working with. This is accomplished by a class creatively called LanguageDefiner, which receives a JSON file containing information for language comprehension, notably a dictionary of built-in data types, and a list of files containing annotated code to interpret.

> Generating Nodes

A node editor is little without its nodes. They are generated based on the files provided to the language definer. Using the aforementioned annotations, snippets from code files are read and processed into structs that provide convenient representations of the source. These structs are later provided to specialised nodes that are able to configure themselves based on their assigned data, setting up input and output pins.

A simple annotated function.

Struct representation of interpreted functions.

Script Parsing

To generate a code output, the system needs to be able to parse a node script, evaluating the order in which nodes should be turned into code, as well as what scopes within the code they belong to. When the data is prepared, it's expanded into code.

> Order Evaluation

The algorithm for evaluating parsing order is implemented as a recursive function, that for a given node repeats itself for each parent node — that is, a node from whom the given node receives an input — and then for each child node. Due to the recursive nature of the function, this results in a valid parsing order that ensures the resources needed for generating a node's code already exist.

Scope is trickier to calculate, as scripts may be structured so that nodes in different scopes share a parent, creating structural challenges.
The solution for this, as implemented, is using a ledger of node dependencies to find the parent scope shared among all dependents.

> Code Generation

Since the nodes are akin to wrappers for underlying language objects, they can simply be unwrapped at this stage. Unwrapping consists of generating a string of valid code that declares a new variable and assigns to it a call to the wrapped function. Creating the final output is done by iterating through all scopes in the code and filling them with their nodes' unwrapped strings.

Shader Compilation

Valid code has been generated – time for the exciting part!
As runtime recompilation is implemented in the engine, creating a working shader from the generated code is simple. The engine's shader factory receives the generated shader code, and uses DirectX11's D3DCompile function to turn it into a ready to use shader.

> Shader Preview Steps

To really make the shader editor feel polished and nice to use, it displays all the steps taken to achieve a final result.

The code generator keeps a list of nodes that output a colour. For each of these nodes, it creates a variant of the shader that ends at said node, treating its output as the final colour output of the shader.

These variants are then used to render a preview model into an atlas, which is selectively displayed as part of the node rendering process, which makes the shader editor feel more polished and nice to use.

Preview shaders being used to render steps in realtime.

Results & Ambitions

With all of the above implemented, the shader scripting tool is functionally finished!

There are some improvements that could improve the structure of the systems at play, such as decoupling them as much as possible from other parts of KittyEngine. These design changes would involve adjustments of engine-level things around it, and currently lie outside the project's scope. With that said, the tool is very capable of providing the desired intuitive workflow.

The language agnostic design of the code generation system opens the door to implementing support for other types of code in the future, such as Lua scripts for rapid iteration of in-game events and behaviours, accessible to non-programmers.