Lost Knowledge & Secrets

When knowledge becomes common knowledge, the curse of time starts affecting it and eventually it may be lost. This is a page dedicated to such knowledge I possess or rediscovered through encountering the same problems. I can't guarantee that someone does or doesn't hold a patent on the knowledge recorded here.

Terminology

  • Bits per Pixel (bpp, BpP)
    The number of bits that are used for a single pixel in total.
  • Bits per Channel (bpc, BpC)
    The number of bits that represent a single channel, such as Red, Green, Blue or Alpha. Usually describes a buffer that will be aligned with 2^n sizes.
  • Bits per Pixel (bpp, BpP)
    The number of bits that are used for a single pixel in total.
  • Bits per Channel (bpc, BpC)
    The number of bits that represent a single channel, such as Red, Green, Blue or Alpha. Usually describes a buffer that will be aligned with 2^n sizes.
  • Dithering
    A technique used to create a percieved or actual additional color by taking advantage of human or physical faults. Very common in Games and Software written during the CRT days, where color reproduction was based on physical properties.
  • Frames per Second (fps, FpS)
    The number of frames displayed in a single second at a constant pace.
  • SDR
    Often used interchangeably with a precision of 8 bits per channel or less. Usually paired with a color space unable to provide most of the human vision color spectrum.
  • HDR
    Often used interchangeably with a precision of 10 bits per channel or more. Usually paired with a color space able to cover 95% or more of the human vision color spectrum.
  • FakeHDR
    An ancient technique that abused the last channel in a 8/8/8/8 texture as a multiplier.
  • A-B-C-D, A/B/C/D
    A combined texture format where each of the A, B, C, and D elements specify the bits for a specific channel, usually in the order RGBA or BGRA.

Computer Graphics

Particle Systems

Calculate maximum Particle count ahead of time

Many modern Particle simulation engines rely on user input for their maximum particle count, despite the maximum particle count being something that can be calculated ahead of time. All you need is the highest possible spawn rate (as particles per second), the minimum decay rate (as decay value per second), and the maximum life time (in seconds). Once you have these variables the following formulae will result in the maximum particle count:

particles = min(maxSpawnRate / minDecayRate, maxLifeTime * maxSpawnRate)

Color Handling

FakeHDR: Simulating HDR by abusing the Alpha channel of an 8/8/8/8 Texture

This is a technique dating back to the days of DirectX 7 and was implemented differently from engine to engine - it was a wild west of ideas after all. The technique works by using the Alpha channel as a Multiplier, often a logarithmic or exponential multiplier. This allowed representing values above 255 somewhat safely, but using it reduced the precision significantly.

Thanks to advanced in computing performance we no longer have to rely on this technique as a 10/10/10/2 RGBA texture can safely represent values up to 1023 with no issue. It is still possible to use the Alpha channel as a multiplier, but with only 4 values for the multiplier it is significantly more limited. Both techniques, old and new, are still in use today.

Is YUV/RGB Conversion lossless?

No, but it's basically visually lossless. Usually it does not matter for final encoding, but if you do this for some effect, consider staying in a higher precision format for the converted values. Alternatively you can also use temporal dithering to hide the imperfections, which is what some streaming and recording software does to improve their output quality.

Dithering: Avoiding color banding by tricking human vision

Dithering - a technique that is somewhat lost to time, thanks to advances in computing power. Dithering used to be everywhere in older games, as the hardware as incapable of rendering what people wanted to do. It works by abusing a significant flaw of Human vision, namely that we aren't that good at identifying colors surrounded by other colors, and will often take a shortcut instead. Thanks to advances in computing power, we can now also do temporal dithering - a technique that monitors already use to make HDR more affordable (see "Frame Rate Control").

You can also make use of this in your own games, either as a stylistic choice, or to enhance the visual fidelity of the output for any player. All you need is a higher precision input than output and a dithering function of your choice. My personal choice for spatial dithering is Bayer, seeded with some per-frame per-pixel random number to make it temporal. Take a look at this ShaderToy example, this image comparison, or this image comparison to see what Dithering can do for you.

Hosting

TrueNAS & FreeNAS

Ideal Record Size

The Record Size of a storage dataset in TrueNAS and FreeNAS matters a lot. It can be the difference between being able to transfer 2 GiB in a second, and taking multiple minutes to do so. While documentation is a bit sparse and fractured, through testing I was able to find out that the best record size is the default it has already selected. It offers the best mixture of performance for large and small writes, especially on faster storage.

My tests went through many record sizes, halving and doubling from 128K outwards. Every halving resulted in large read/writes getting significantly slower, but small read/writes getting slightly faster. Every doubling resulting in large read/write getting slightly faster, but small read/writes getting significantly slower. The best value ended up being 128K, which managed to keep most of the NVMe performance as is (~80MiB/s in RND4K, ~2.5GiB/s in RND1M).

Comments for: Lost Knowledge & Secrets