libretro/common-shaders

================================ -- Common shaders --

This is a package of pixel shaders intended for old school emulators. Copyrights are held by the respective authors. The shaders are coded in the Cg language, suitable for both OpenGL and D3D. Most of these shaders are converted from other languages (GLSL, FX, HLSL, etc).

The shaders all follow a convention which must be followed by the implementer of an OGL/D3D backend.

Known implementations of this spec are currently:

• RetroArch
• SNES9x (Win32 port)
• OpenEmu

Entry points: Vertex: main_vertex Fragment: main_fragment

Texture unit: All shaders work on texture unit 0 (the default). 2D textures must be used. Power-of-two sized textures are recommended for optimal visual quality. The shaders must deal with the actual picture data not filling out the entire texture. Incoming texture coordinates and uniforms provide this information.

The texture coordinate origin is defined to be top-left oriented, i.e. a texture coordinate of (0, 0) will always refer to the top-left pixel of the visible frame. This is opposite of what most graphical APIs expect. The implementation must always ensure that this ordering is held for any texture that the shader has access to.

Every texture bound for a shader must have black border mode set. I.e. sampling a texel outside the given texture coordinates must always return a pixel with RGBA values (0, 0, 0, 0).

Uniforms: Some parameters will need to be passed to all shaders, both vertex and fragment program. A generic entry point for fragment shader will look like:

float4 main_fragment (float2 tex : TEXCOORD0, uniform input IN, uniform sampler2D s_p : TEXUNIT0) : COLOR {}

The input is a struct looking like: struct input { float2 video_size; float2 texture_size; float2 output_size; float frame_count; float frame_direction; };

TEXCOORD0: Texture coordinates for the current input frame will be passed in TEXCOORD0. (TEXCOORD is a valid alias for TEXCOORD0).

COLOR0: Although legal, no data of interest is passed here. You cannot assume anything about data in this stream.

IN.video_size: The size of the actual video data in the texture, e.g for a SNES this will be generally (256, 224) for normal resolution frames.

IN.texture_size: This is the size of the texture itself. Optimally power-of-two sized.

IN.output_size: The size of the video output. This is the size of the viewport shown on screen.

IN.frame_count: A counter of the frame number. This increases with 1 every frame. This value is really an integer, but needs to be float for CGs lack of integer uniforms.

IN.frame_direction: A number telling which direction the frames are flowing. For regular playing, this value should be 1.0. While the game is rewinding, this value should be -1.0.

modelViewProj: This uniform needs to be set in vertex shader. It is a uniform for the current MVP transform.

Pre-filtering: Most of these shaders are intended to be used with a non-filtered input. Nearest-neighbor filtering on the textures themselves are preferred. Some shaders, like scanline will most likely prefer bilinear texture filtering.

Genres: There are several different types of shaders available, some of the relevant types are sorted into folders.

2x-classic: These are the typical, classic filters usually run on CPU, such as HQ2x, 2xSaI, SuperEagle, etc, converted into shaders.

Blur: Shaders focusing on bluring the output image.

Enhance: Shaders focusing on enhancing the image quality through other means than bluring only.

TV: Shaders focusing on replicating the visual image of a game running on a CRT screen.

Misc: Shaders that do not directly fit into any of the above categories.

Meta: Contains meta-shaders *.cgp.

============================= -- Cg meta-shader format --

Rationale: The .cg files themselves contain no metadata necessary to perform advanced filtering. They also cannot process an effect in multiple passes, which is necessary for some effects. The CgFX format does exist, but it would need current shaders to be rewritten to a HLSL-esque format. It also suffers a problem mentioned below.

Rather than putting everything into one file (XML shader format), this format is config file based. This greatly helps testing shader combinations as there is no need to rearrange code in one big file. Another plus with this approach is that a large library of .cg files can be used to combine many shaders without needing to redundantly copy code over. It also helps testing as it is possible to unit-test every pass separately completely seamless.

Format:

The meta-shader format is based around the idea of a config file with the format: key = value. Values with spaces need to be wrapped in quotes: key = "value stuff". No .ini sections or similar are allowed. Meta-shaders may include comments, prefixed by the "#" character, both on their own in an otherwise empty line or at the end of a key = value pair.

The meta-format has four purposes:

• Combine several standalone .cg shaders into a multipass shader.
• Define scaling parameters for each pass. I.e., a HQ2x shader might want to output with a scale of exactly 2x.
• Control filtering of textures. Many shaders will want nearest-neighbor filtering, and some will want linear.
• Define external lookup textures. Shaders can access external textures found in .tga files.

Parameters:

• shaders (int): This param defines how many .cg shaders will be loaded. This value must be at least one. The path to these shaders will be found as a string in parameters shader0, shader1, ... shaderN, and so on. The path is relative to the directory the meta-shader was loaded from.

• filter_linearN (boolean): This parameter defines how the texture of the input to pass N will be filtered. N = 0 affects how the pass 0 (initial pass) samples the raw input frame. N = 1 affects how pass 1 samples the raw input frame; in other words, it controls the sampling of pass 0 output (A boolean value here might be true/false/1/0). Should this value not be defined, the filtering option is implementation defined.

• float_framebufferN (boolean): This parameters defines if shader N should render to a 32-bit floating point buffer. This only takes effect if shaderN is actually rendered to an FBO. This is useful for shaders which have to store FBO values outside [0, 1] range.

• frame_count_modN (int): This positive parameter defines which modulo to apply to IN.frame_count. IN.frame_count will take the value frame_count % frame_count_modN.

• scale_typeN (string): This can be set to one of these values: "source": Output size of shader pass N is relative to the input size as found in IN.video_size. Value is float. "viewport": Output size of shader pass N is relative to the size of the window viewport. Value is float. This value can change over time if the user resizes his/her window! "absolute": Output size is statically defined to a certain size. Useful for hi-res blenders or similiar.

  If no scale type is assumed, it is assumed that it is set to "source" with scaleN set to 1.0.

  It is possible to set scale_type_xN and scale_type_yN to specialize the scaling type in either direction. scale_typeN however overrides both of these.

  Exceptions: If no scale_type is set for the very last shader, it is assumed to output at the full resolution rather than assuming a scale of 1.0x, and bypasses any frame-buffer object rendering. If there is only one shader, it is also considered to be the very last shader. If any scale option is defined, it has to go through a frame-buffer object, and subsequently rendered to screen. The filtering option used when stretching is implementation defined. It is encouraged to not have any scaling parameters in last pass if you care about the filtering option here.

  In first pass, should no scaling factor be defined, the implementation is free to choose a fitting scale. This means, that for a single pass shader, it is allowed for the implementation to set a scale, render to FBO, and stretch. (Rule above).

• scaleN, scale_xN, scale_yN (float/int): These values control the scaling params from scale_typeN. The values may be either floating or int depending on the type. scaleN controls both scaling type in horizontal and vertical directions.

  If scaleN is defined, scale_xN and scale_yN have no effect. scale_xN and scale_yN controls scaling properties for the directions separately. Should only one of these be defined, the other direction will assume a "source" scale with value 1.0, i.e. no change in resolution.

  Should scale_type_xN and scale_type_yN be set to different values, the use of scaleN is undefined (i.e. if X-type is absolute (takes int), and Y-type is source (takes float).)

• textures (multiple strings): The textures param defines one or more lookup textures IDs. Several IDs are delimited with ';'. I.e. textures = "foo;bar" These IDs serves as the names for a Cg sampler uniform. I.e. uniform sampler2D foo; uniform sampler2D bar;

  The path of the textures can be found in the IDs, i.e. foo = image0.tga bar = image1.tga The paths of these textures are relative to the directory the meta-shader was loaded from.

  It is also possible to control the filtering options of the lookup texture as a boolean option in ID_linear = true/false. I.e. foo_linear = false, will force nearest neighbor filtering for texture "foo". If this param is not set, it is assumed to be linearily filtered.

  The textures will be loaded "as-is", and coordinates (0, 0), (0, 1), (1, 0), (1, 1) will correspond to the corners of the texture. Since the texture coordinates of the texture in TEXUNIT0 might not be as convenient, the texture coordinates for all lookup textures will be found in TEXCOORD1. (Note: You cannot assume which texture unit the lookup textures will be bound to however!)

  The implementation only guarantees to be able to load plain top-left non-RLE .tga files. It may provide possibilities to load i.e. .png and other popular formats.

Multi-pass uniforms:

During multi-pass rendering, some additional uniforms are available.

With multi-pass rendering, it is possible to utilize the resulting output for every pass that came before it, including the unfiltered input. This allows for an additive approach to shading rather than serial style.

The unfiltered input can be found in the ORIG struct:

uniform sampler2D ORIG.texture: Texture handle. Must not be set to a predefined texture unit. uniform float2 ORIG.video_size: The video size of original frame. uniform float2 ORIG.texture_size: The texture size of original frame. in float2 ORIG.tex_coord: An attribute input holding the texture coordinates of original frame.

PASS%u: This struct holds the same data as the ORIG struct, although the result of passes {1, 2, 3 ...}, i.e. PASS1.texture holds the result of the first shader pass. If rendering pass N, passes {1, ..., N-2} are available. (N-1 being input in the regular IN structure).

PREV: This struct holds the same data as the ORIG struct, and corresponds to the raw input image from the previous frame. Useful for motion blur.

PREV1..6: Similar struct as PREV, but holds the data for passes further back in time. PREV1 is the frame before PREV, PREV2 the frame before that again, and so on. This allows up to 8-tap motion blur.

For backend implementers:

-- Rendering the shader chain: --

With all these options, the rendering pipeline can become somewhat complex. The meta-shader format greatly utilizes the possibility of offscreen rendering to achieve its effects. In OpenGL usually this is referred to as frame-buffer objects, and in HLSL as render targets (?). This feature will be referred to as FBO from here. FBO texture is assumed to be a texture bound to the FBO.

As long as the visual result is approximately identical, the implementation does not have to employ FBO.

With multiple passes our chain looks like this conceptually:

|Source image| ---> |Shader 0| ---> |FBO 0| ---> |Shader 1| ---> |FBO 1| ---> |Shader 2| ---> (Back buffer)

In the case that Shader 2 has set some scaling params, we need to first render to an FBO before stretching it to the back buffer.

|Source image| ---> ... |Shader 2| ---> |FBO 2| ---> (Back buffer)

Scaling parameters determine the sizes of the FBOs. For visual fidelity it is recommended that power-of-two sized textures are bound to them. This is due to floating point inaccuracies that become far more apparent when not using power-of-two textures. If the absolute maximum size of the source image is known, then it is possible to preallocate the FBOs.

Do note that the size of FBOn is determined by dimensions of FBOn-1 when "source" scale is used, not the source image size! Of course, FBO0 would use source image size, as there is no FBO-1 ;)

I.e., with SNES there is a maximum width of 512 and height of 478. If a source relative scale of 3.0x is desired for first pass, it is thus safe to allocate a FBO with size of 2048x2048. However, most frames will just use a tiny fraction of this texture.

With "viewport" scale it might be necessary to reallocate the FBO in run-time if the user resizes the window.