2D accuracy: it's a rabbit hole too
Yeah. I've been wanting to fix mosaic support in melonDS, and, well, got sidetracked (I know I still have to release the JIT and DSi betas).

Anyway, mosaic is a typical feature of old consoles with 2D engines, including the DS. It basically applies a pixelation effect to graphics, as shown here:



The basic idea is that the screen is split in a grid, whose dimensions are variable (configured by register 0x0400x04C on the DS). For each 'cell' in the grid, all the pixels are colored the same as the first (top-left) pixel. In reality, it's a bit more complex, as on the DS the effect can be applied per-layer and per-sprite, and you can even specify different grid sizes for BG layers and sprites, but fundamentally it's more or less the same thing, it pixelates shit.

Sounds simple enough, right?

It's a bit tricky to implement when you're trying to write a performant renderer, though. Which is more or less why blargSNES never supported it.

As far as melonDS is concerned, mosaic was implemented in version 0.5, but (among other silly bugs) it never worked quite right as far as sprites were concerned, especially when those use rotation/scaling. But, at the time, I didn't do much past the original implementation, mostly because I don't know of a lot of games that use the mosaic effect. The lack of test cases meant it stayed supbar.

Until, well, now.

First thing to do is to write some test cases for sprite mosaic. BG mosaic seems quite simple, even though I would still have to probe it extensively for edge cases, but sprite mosaic is a bit more oddball, as seen here:






Those snapshots were taken from Grey, my capture-card DS. This isn't some weird buggy emulator, sprite mosaic really is that weird.

The sprite being mosaic'd is a simple ball sprite. The cyan sprite behind doesn't have mosaic applied to it, but both share the same size and rotscale parameters. The sprites on the left have double-size mode enabled, which merely doubles the bounding box of rotscaled sprites.

I have yet to work out all the oddities, but, from what I have observed, it seems that sprite mosaic doesn't work as we thought. If vertical mosaic is simply done by adjusting the source Y coordinate when rendering BG layers or sprites, horizontal mosaic is a different topic. As far as sprites are concerned:

* horizontal mosaic is restrained to the sprite's bounding box (vertical mosaic is too, but only in one direction)
* seeing how it is affected by the presence of neighbor sprites, horizontal mosaic seems to be applied after all the sprites are rendered
* it seems to keep track of which pixels belong to each sprite, even when said pixels are transparent


As of now, none of the existing DS emulators get sprite mosaic right. Probably, none of the GBA emulators get it right either, since the GBA uses a similar renderer, and it doesn't look like anybody has ever figured out the mystery of sprite mosaic.


I wanted to be able to check my mosaic implementation against hardware for correctness in many situations, to make sure I'd gotten the logic right. But constantly dumping frames from Grey and hand-comparing them against melonDS would be tedious. I wanted to have a better tool.

So, I set to work. I opened ds_capture.exe in IDA, opened up the Linux DS-capture code example and WinUSB documentation, and set to work. And, in one evening, MelonCap was born.



This is pretty simple. Left screen is the output from melonDS, middle is what is captured from Grey, right is a color-coded visualization of the differences.

The precision isn't exactly optimal though: melonDS outputs RGB666 color, like the DS does, but the capture card degrades it to RGB565 due to technical constraints. Well, this is atleast better than RGB555.

This is pretty much made for graphical testing in specific use cases, as there's no synchronization mechanism, one has to ensure that both melonDS and the DS are rendering the same things.

Anyway, the example in the screenshot above shows something interesting. There's been all that effort put towards pixel-perfect 3D, but our 2D engine isn't pixel-perfect (neither is the 3D one, but... yeah). We figured pixel perfection with the 2D engine would be a given, and, yet... here we are.

The bottom screen has BLDCNT configured so that all layers/sprites will be dimmed, with BLDY=1. This is hardly noticeable, and is likely an oversight. However, the colors generated by melonDS in this situation don't quite match hardware output.

The function for the brightness-down effect is described by GBAtek as:

out = in - ((in * EVY) / 16);
However, a quick hardware test tends to show that the actual function is:

out = in - (((in * EVY) + 7) / 16);
There's probably more of this shit everywhere, so the 2D engine will have to be heavily tested for accuracy. Fun shit ahead.
ari says:
Sep 15th 2019
Nice!
So will pixel perfect 2d soften the edges of sprites? I mentioned this earlier but 2d sprites are really blocky compared to those sweet, sweet upscaled 3d models.
(I've been trying to run Chrono Trigger, not great)
Junko says:
Sep 15th 2019
"So will pixel perfect 2d soften the edges of sprites?" No? This will just render things as intended, just like the real hardware (aka "pixelated" as the real thing). It's not "blocky", it's simply a 2D image being scaled to a bigger screen, obviously you'll notice the lower resolution and there is no magic or code that could "fix" this (some can argue that filters help, but 99% of the filters and post processing shaders will just hide the issue or make it even worse)
Generic alias RSDuck says:
Sep 15th 2019
> So will pixel perfect 2d soften the edges of sprites? I mentioned this earlier but 2d sprites are really blocky compared to those sweet, sweet upscaled 3d models.

you might confuse it with hd mode 7, which is a feature recently introduced in some snes and gba emulators. It basically does sprite and background scaling and rotation on a higher resolution, so they look much cleaner. The chances for this to be implemented in melonds very low, since it increases the complexity of the 2d code by a lot. At the same time sprite/bg transform isn't even used much on the ds, since it has a 3d capable gpu, which is more flexible anyway. And for it we already have the ogl renderer, rendering at higher resolution.
Arisotura says:
Sep 15th 2019
this.

higher-res 3D is possible because we substitute the whole renderer with a different one that works at a higher resolution (which is doable since the 3D renderer is entirely separate from the 2D one), and the 2D renderer can be adjusted for that relatively easily.

on the other hand, upscaling individual 2D elements (layers, sprites) would require either

a) rendering 2D at a higher resolution, which complexifies the code and reduces performance dramatically the higher you go

b) using a different, hardware-accelerated 2D renderer, that could render at higher resolutions with much less performance penalty. however the 2D renderer is vastly different from what we can do with OpenGL. it would be like the hardware renderer in blargSNES -- the main challenge is handling midframe configuration changes, which aren't possible with the 3D renderer but are possible (and exploited) with the 2D renderer. it's also much more difficult to emulate various quirks when you emulate things at that level.
bajol says:
Sep 15th 2019
"Probably, none of the GBA emulators get it right either"
Have you taken a look at mGBA? I think endrift already fixed it (or got close to it)
Arisotura says:
Sep 15th 2019
from what I could gather, mGBA is close, but not perfect
Zinx says:
Sep 15th 2019
I think the only DS game that uses "Mode-7" in any large extant is Chrono Trigger DS and even than it's used in like 3 places lol.
Arisotura says:
Sep 16th 2019
this. also mode7 is done by changing the rotscale parameters every scanline, which, bdfgfhg

rotscale was shitty to emulate in the blargSNES hardware renderer already. I don't really recall how DiscostewSM tackled it. but anyway, it's possible to scale the rotscale layer to being infinitely small and have it repeat, which is a really shitty case if you try to render it tile-by-tile (which is how the hardware renderer rendered things).
MelonMan says:
Sep 16th 2019
Don't overwork yourself trying to do mosaic, JIT and DSi at the same time though.
ReductiveChaos says:
Sep 16th 2019
Multiple projects doesn't always mean overworking. Sometimes I found when I was working on coding projects, albeit nothing close to this scale or complexity, that having a few choices on what to work on kept me fresher when something was getting boring or frustrating.

Also, I have to say I love posts like this. As with mGBA's similar posts, I find these technical details fascinating.
MelonMan says:
Sep 16th 2019
ReductiveChaos: I suppose viewing tasks with a fresh mind might help as well.
MelonMan says:
Sep 16th 2019
ReductiveChaos: I suppose viewing tasks with a fresh mind might help as well.
MelonMan says:
Sep 16th 2019
Dodgy mouse strikes again
They says:
Sep 16th 2019
I have an idea to suggest, a fancy launcher which instead of open files shows list of games with icons like Citra do.
poudink says:
Sep 16th 2019
I think that was already planned
Post a comment
Name:
DO NOT TOUCH