3D Rendering
============

The UI library provides the ability to draw hardware-accelerated 3D graphics. As for [2D
rendering](md:2D_Rendering), this happens on a separate thread ([stormname:ui.Render]) to avoid
blocking the [stormname:ui.Ui] thread.

Since a large part of the 3D library (particularly the language extensions) is implemented in Basic
Storm, it is located in its own sub-package: `ui.s3d` so that applications that do not require 3D
rendering do not need to load it. `s3d` is short for "Storm 3D".


The following people have greatly helped developing the 3D rendering in the UI library:
Viggo Ingmarsson, Mattis Olevall, Rasmus Ring, Max Ryttersten, Alfred Lindström, Emil Jönsson, Elsa
Annell


Painters and Graphics
---------------------

3D rendering interacts with the windowing system very similarly to the 2D rendering system. This is
perhaps not surprising since they share much of the infrastructure for hardware-accelerated
rendering. As such, to render 3D graphics in a window, it is necessary to subclass the
[stormname:ui.Painter3D] class, override the `render` member, and attach the painter to a
window by calling the [stormname:ui.Window.painter(core.Maybe(ui.BasicPainter))] member in
[stormname:ui.Window]. This will cause the UI library to call the `render` member whenever a new
frame should be rendered. Either at a steady framerate, or whenever `repaint` is called.

The [stormname:ui.Painter3D] class has the following members:

```stormdoc
@ui.Painter3D
- .render(*)
- .repaint()
- .captureImage(*)
- ui.BasicPainter.captureImage(*)
```

As can be seen from above, the [stormname:ui.Painter3D] class provides a
[stormname:ui.Graphics3D] instance that can be used for 3D rendering. In contrast to
[stormname:ui.Graphics] used for 2D rendering, the [stormname:ui.Graphics3D] class provides
a relatively sparse public interface:

```stormdoc
@ui.Graphics3D
- .fill(*)
- .clearDepth(*)
```


Pipelines
---------

To render 3D geometry through a [stormname:ui.Graphics3D] object it is necessary to define a
*pipeline*. This is done as follows in the language extension that the `ui.s3d` library provides to
the Basic Storm language:

```bs
use ui:s3d;

pipeline ExamplePipeline {
    // Describe the data flow through the pipeline:
    data (Input input, uniform Float time) -> Intermediate -> Output;

    // Other members may be defined here. They become uniforms.
    Transform camera;

    // It is also possible to define arbitrary Basic Storm members. They
    // are only accessible to Storm, not to shaders.
}
```

As can be seen above, defining a pipeline uses the keyword `pipeline` followed by a name. This
intentionally resembles a class definition, since a pipeline behaves much like a class. The
exception is that it contains a data definition, the data members become accessible as uniforms in
the shaders. Any member functions are only accessible from Storm, not from shaders.

The first line inside a pipeline is special. It defines the data flow through the pipeline. The
`data` keyword is followed by a set of parentheses that specify which parameters should be passed as
inputs to the parameter. The syntax resembles the definition of function parameters. This is also
intentional. As we shall see, the types passed here will become parameters to the `render` function
of the pipeline.

There are two types of parameters. Vector inputs and uniforms. The first parameter (`Input input`)
is a vector input. This means that we will give the `render` function an array of vertices of the
type `Input` stored in a `VertexBuffer`. The rendering pipeline will then apply the shaders on each
element individually (you can think of it as the system adding a `for` loop for you).

Uniform do not work this way. They are set once, and have the same value across all invocations of
the shaders. As such, the system does not expect an array of `Float`s for the `time` parameter for
example. Rather, it expects a single `Float` that will be used for all vertices. It is also possible
to define uniforms as data members inside the pipeline. As we shall see, they work a little
differently to uniforms defined in the parameter list.

After the input definition, there is an arrow `->` followed by another vertex type (`Intermediate`).
This specifies the output type of the vertex shader and subsequently the input type of the pixel
shader. Finally, there is another arrow `->` and another vertex type (`Output`). This type specifies
the output vertex type for the pixel shader.


A pipeline is defined as a class that (indirectly) inherits from [stormname:ui.Pipeline], and
inherits some members from those classes. However, what is relevant is typically the following
members generated by the pipeline definitions:

- For each member that is defined in the pipeline, the library generates a getter and a setter
  function. This makes it possible to access the members as if they are normal member variables
  (although the getters and setters track which members have changed, to know when data needs to be
  uploaded to the GPU).

- The static function `create(VertexShader, PixelShader)`. This function is used to create an
  instance of the pipeline. The `VertexShader` and `PixelShader` arguments refer to a vertex and a
  pixel shader that should be used in the pipeline. These are explained below.

- The `render(Graphics3D, DrawPrimitive, ...)` function. This function is used to render geometry using
  the pipeline. The first parameter (apart from the `this` parameter) is a reference to a
  [stormname:ui.Graphics3D] object that encapsulates the render target. The second parameter is of
  the type [stormname:ui.DrawPrimitive], and determines what type of geometry to render (e.g.
  [stormname:ui.DrawPrimitive.triangles]).

  The remaining parameters depend on the parameters after the `data` declaration of the pipeline.
  For each parameter that is *not* marked as `uniform`, the `render` function will accept a
  `VertexBuffer` of the corresponding vertex type. For each parameter marked as `uniform`, the
  `render` function will accept a parameter of the corresponding type. For example, the `render`
  function for `ExamplePipeline` above has the following signature:
  `render(Graphics3D, DrawPrimitive, VertexBuffer<Input>, Float)`

  It is worth noting that uniforms defined as parameters to the `render` function are expected to
  change with each call to `render`. As such, in contrast to variables defined on the pipeline
  itself, these parameters are uploaded to the GPU every time, even if the same value is passed
  multiple times.

- There is also a version of `render` that accept an [stormname:ui.s3d.IndexBuffer] for rendering
  indexed geometry. This version of the `render` function has the same signature as the one
  mentioned above, except that the [stormname:ui.DrawPrimitive] parameter is replaced with a
  [stormname:ui.s3d.IndexBuffer]. The index buffer specifies which primitive to render.

- If the input vertex type is [stormname:ui.s3d.Vertex] (i.e. the vertex type used in
  [stormname:ui.s3d.Model]), there is also a version of `render` that accepts a `Model` instead of
  the the [stormname:ui.DrawPrimitive] and the vertex buffer. For example, if we replace `Input` with
  `ui:s3d:Vertex` in `ExamplePipeline`, the following `render` overload would also be available:
  `render(Graphics3D, Model, Float)`.


Vertices
--------

The pipeline requires that all inputs, outputs, and uniforms are *vertices*. In essence, a vertex
type is a type that can be manipulated on the GPU. The name stems from the fact that they are
typically used to represent vertex data in a 3D mesh (e.g. position, normal, etc.).

The following built-in types are considered primitive vertex types:

- [stormname:core.Bool]: booleans.
- [stormname:core.Int]: signed integers.
- [stormname:core.Nat]: unsigned integers.
- [stormname:core.Float]: floating point numbers.
- [stormname:core.geometry.Angle]: 1D float, known to represent an angle.
- [stormname:core.geometry.Point]: 2D vectors of floats.
- [stormname:core.geometry.Size]: 2D vectors of floats.
- [stormname:core.geometry.Vector]: 3D vectors of floats.
- [stormname:core.geometry.Vector4]: 3D vectors of floats, homogenous coordinates to allow representing
  translations and projections as matrices. The 3D rendering pipeline requires homogenous coordinates in
  some places. Otherwise it is more or less interchangeable with non-homogenous coordinates.
- [stormname:core.geometry.Rotation]: 4D quaternions used to represent arbitrary rotations in 3D.
- [stormname:core.geometry.Transform]: 4x4 matrices to represent transformations in 3D.
- [stormname:graphics.Color]: 4D vector representing a color (R, G, B, and alpha).

It is also possible to define custom composite vertex types using the `vertex` keyword. As can be
seen below, vertex types are similar to class and value definitions. However, there are some
limitations, most importantly that all members need to be vertex types themselves.

For example, the input type from the pipeline above may be defined as follows:

```bs
use ui:s3d;

vertex Input {
    Vector pos;
    Color color;
}
```

At the time of writing, it is not possible to define member functions in vertex types. This may be
added in the future. However, as the shader language follows the same lookup rules as Basic Storm,
it is possible to call free functions as if they are member functions.

Vertex types automatically generate a set of constructors, both for the shader language and for
Basic Storm. These constructors simply accept one parameter for each member of the vertex type and
initialize members one by one. There are also constructors with fewer parameters than members. These
simply initialize the remaining members to zero.

For example, the expression `Input(Vector(1.0, 1.0, 1.0), Color(0.0, 1.0, 0.0))` initializes both
`pos` and `color`. It is also possible to write `Input(Vector(1.0, 1.0, 1.0))` to initialize `pos`
as specified and `color` to zero (i.e. transparency).


Vertex types also allow marking members based on how they should be used by the rendering pipeline
(DirectX call these "semantics"). This is used to (1) use the output from the vertex shader to
determine which pixels need to be rendered, (2) determine how to interpolate vertices from the
vertex shader as inputs to the pixel shader, and (3) determine which output from the pixel shader
represents the color on the screen.

To fullfill (1) and (2) above, the vertex type that is used as an output from the vertex shader
has the following requirements:

- One [stormname:core.geometry.Vector4] needs to be marked as `position`. It should contain the
  screen-space coordinates of the vertex. The rendering pipeline will then use this value to
  determine which pixels need to be rendered.
- All members of the vertex type will be interpolated according to an interpolation mode.
  If nothing else is specified, `perspective` is used by default. This can be changed by
  specifying any other supported interpolation mode.

The following interpolation modes are supported:

- `perspective`: Apply perspective correction to the interpolation.
- `linear`: Apply linear interpolation without perspective correction.
- `none`: No interpolation.

For example, the intermediate vertex can be defined as follows:

```bs
use ui:s3d;

vertex Intermediate {
    Vector4 pos : position, perspective;
    Color color : perspective;
}
```

As can be seen above, `pos` is marked as both `position` and `perspective` for clarity. The `color`
member is only marked as `perspective` for clarity. Again, this is optional as `perspective` is the
default interpolation.

To fullfill (3) above, the output from the pixel shader needs to have at least one
[stormname:graphics.Color] member marked `color` or `output` to let the rendering pipeline know
which member to paint onto the screen. It is also possible to modify the depth value of the pixel by
annotating a member with the `depth` flag (a [stormname:core.Float]).

As such, the `Output` vertex type from above can be defined as below:

```bs
use ui:s3d;

vertex Output {
    Color out : color;
}
```


Shaders
-------

Shaders are defined using a syntax that is similar to ordinary functions in Basic Storm. The main
difference is that shaders need to know which pipeline they belong to, so that the system knows
which uniforms are available (i.e. shaders are similar to member functions, except for the fact that
they are typically defined outside of the pieline itself). Shaders also need to know if they are
used as a vertex shader or a pixel shader to check that inputs and outputs match the specification
in the pipeline.

All of this is done using a set of keywords before a normal function definition. Vertex shaders
start with `vertex shader for <pipeline>:` and pixel shaders start with `pixel shader for
<pipeline>:`. In both cases, `<pipeline>` is the name of a pipeline.

The remainder of the shader definition follows the semantics of a Basic Storm function definition.
That is: `<output> <name>(<parameters>)`, where `<output>` is the name of the output type, `<name>`
is the name of the shader, and `<parameters>` is a comma-separated list of parameters to the shader.

Note that the output and parameter list need to match the `data` declaration in the associated
pipeline. For vertex shaders, this means that the parameter list needs to match the types (but not
names) in the parentheses after `data`. However, any uniform parameters should not be listed to the
shader. They are available anyway by virtue of being uniforms. The output type must match the type
after the first arrow (`->`).

For the pixel shader, the input must match the type after the first arrow (`->`). There may be
additional inputs that correspond to quantities that are generated by the system (e.g. current Z
depth, but this is not yet implemented). The output type must match the type after the last arrow
(`->`) in the pipeline.

The code in the body of the shader is written in a reduced version of Basic Storm. In general,
operations that can be used with the vertex types above are supported, basic control flow (`if`,
`for`, `while`), and `return`, but not much else. The shader code is then compiled to GLSL or HLSL
depending on which rendering system is used on the target platform.

Below are two example shader functions that can be used with the `ExamplePipeline` defined earlier.
Note that the uniforms (e.g. `camera`) can be accessed as if they are global variables.

```bs
use ui:s3d;

vertex shader for ExamplePipeline: Intermediate vertexShader(Input input) {
    Vector4 pos = Vector4(input.pos) * camera;
    Intermediate(pos, input.color);
}

pixel shader for ExamplePipeline: Output pixelShader(Intermediate point) {
    Output(point.color);
}
```

Shader functions then become available in the name tree. They are exposed as functions that simply
return a corresponding shader object. As such, they can be used to create a pipeline as shown below.

```bsstmt
SimplePipeline pipeline = SimplePipeline:create(vertexShader, pixelShader);
```


It is also possible to define functions that are usable by shaders (shader functions). These differ
from vertex and pixel shaders in that they are not associated with any particular pipeline. They are
therefore usable from any shader from any pipeline. However, this also means that it is not possible
to access uniforms from within shader functions, unless they are passed as parameters to the shader
function.

Shader functions are declared by prepending the keyword `shader` before a Basic Storm function
definition. For example, a function that mixes two colors together may be defined as follows:

```bs
use ui:s3d;

shader Color lerp(Color a, Color b, Float t) {
    return a * t + b * (1 - t);
}
```

Note: it is currently *not* possible to call neither shaders nor shader functions directly from
Basic Storm. This may be possible in the future to simplify testing.


Textures
--------

Textures in the 3D rendering process are represented by the [stormname:ui.Texture] class. Instances
of this class represent a texture that is accessible to the GPU, and the associated sampler state
(e.g. how to handle wrapping, mip-mapping, etc.).

Even though textures are not vertex types, it is possible to define a texture variable in a
pipeline. The pipeline is aware of the [stormname:ui.Texture] class and handles it appropriately.

The [stormname:ui.Texture] class has the following members:

```stormdoc
@ui.Texture
- ._init(*)
- .image()
- .image(graphics.Image)
- .sample(*)
```

In particular, note that `Texture` objects are uploaded to the GPU on demand. Replacing the image
inside a texture always causes the new image to be uploaded to the GPU. As such, if you are using
one pipeline to render multiple textures in a single frame, it is better to create multiple
`Texture` objects and simply assign different ones to the shader rather than replacing the image in
the texture.

The [stormname:ui.Texture.sample(core.geometry.Point)] function can be called both on the CPU and
the GPU to sample textures. As such, applying a texture to geometry can be done with a pixel shader
that looks like below (assumes that `Intermediate` has a `tex` member that stores texture
coordinates, and that the pipeline has a texture named `texture`).

```bs
use ui:s3d;

pixel shader for TexturePipeline: Output pixelShader(Intermediate point) {
    Color texColor = texture.sample(point.tex);
    Output(texColor);
}
```


Models
------

The 3D library contains the class [stormname:ui.s3d.Model], which represents some 3D model that can
be rendered. In essence, it wraps an index buffer and a vertex buffer together to provide a
standardized representation for exchanging geometry and some rudimentary operations.

The [stormname:ui.s3d.Model] class has the following members:

```stormdoc
@ui.s3d.Model
- .vertices(*)
- .indices(*)
- .append(*)
- .transform(*)
- .color(*)
- .withColor(*)
```

It also provides the following static member functions to create some standard 3D shapes:

```stormdoc
@ui.s3d.Model
- .cube(*)
- .unitCube(*)
- .sphere(*)
- .unitSphere(*)
```

It is also possible to load 3D models from `.obj` files using the `loadObj` function:

```stormdoc
- ui.s3d.loadOBJ(*)
```

The model uses the vertex type [stormname:ui.s3d.Vertex], which has the following members:

```stormdoc
@ui.s3d.Vertex
- .pos
- .normal
- .color
- .tex
```


Example usage
-------------

The [tutorials section](md:/Getting_Started/Tutorials/3D_Rendering) of the documentation contains a
few examples of how to use the 3D part of the UI library to render some simple geometry and apply
some simple effects.
