|Home | Downloads | Screenshots | Forums | Source code | RSS|
How far we have gone, and where we are going next
Nov 9th 2017, by StapleButter
melonDS has been going for one year now.
Well, the repo was opened in May 2016, but the real work started in November 2016.
I didn't really know what I was in for, but hey, I wanted to have fun making something new. There wasn't much else in terms of goals, other than the famous goal of successfully emulating local wifi.
Of course, getting an emulator project going is initally a gruelling task, as it takes quite a bit of work before you can get interesting results. In the case of the DS, as I went with the goal of emulating the BIOS boot procedure, there was quite a lot to be done before the firmware would try to display things. In the case of melonDS, it took about 2.5 months to get there, but I don't consider it a good idea to look at durations. I was unemployed at the time (or rather, waiting to start that job), I'm a lazy fuck, and a couple vicious CPU bugs got in the way.
The first release of melonDS was a mere curiosity. It was inferior to the existing emulators. But that didn't matter, it was out there for the curious, and I was having fun with that project. The next releases would deliver quality.
The 3D renderer was my first attempt at a polygon rasterizer. I initially fumbled getting things like perspective correction working, but gradually gained more understanding of the process. Conducted tests to try understanding how the actual GPU did things and which shortcuts it took. And it has been an interesting ride.
I guess it turned out okay. This renderer has become the most accurate among all DS emulators. I thought, since the 2D renderer is going to be pixel-perfect, why not make 3D pixel-perfect too? I'm not there yet though, but the renderer is more than good enough for most games. Not good enough for the aging cart tests (which demand perfection to the pixel), but I put that on hold for a while.
On the other hand, the renderer code evolved as I discovered things, and is probably not as clean as it could be. In long term, it could benefit from a rewrite.
And this shit has been going on for one year now. Time flies, it's crazy.
melonDS became something serious. Not only does it have a pretty solid emulator core, it even does things no other emulator does. Be it very niche things like properly emulating the mainmem display FIFO, or emulating enough of the wifi system for local multiplayer to be possible.
Of course, it's not perfect.
Local wifi suffers from data loss every now and then, and some games just refuse to connect at all.
CPU timings are still grossly inaccurate. It's a shitty zone, especially on the ARM9, where the caches can affect performance radically. There are basically two unappealing choices: emulating the whole MPU/cache system (you can figure the impact on emulator performance), or guessing the timings in a grossly inaccurate fashion.
Sound has its lot of issues too, and I have yet to figure those out. Surround modes don't work well, when they work at all (generally they don't work). There are a couple issues with sound quality, one coming from bad emulation and one coming from the fact that the frontend doesn't do any resampling -- it picks the output frequency the closest to the core's sampling rate, and hopes things will go right. And, in the DS, sound mixing and sampling is driven by the system clock, which results in a non-integer output frequency.
And, most notably, the UI is still pretty lacking in functionality. I intend to keep melonDS lightweight and straightforward, but regardless, some features can't hurt. This is being worked on since 0.5, but at a slow-ish pace.
Real life has caught up to me. I had a job, but it ended two months ago, and I need to find something else. There are also other projects I've been rather busy into, lowering the amount of time I spend on the internet.
And... things I recently discovered about myself. Actions I want to take in that regard, but nothing is going to be easy.
So, even if the pace has slowed down, melonDS isn't dying.
Here's a glimpse of what's being worked on:
Rotation, for that occasional game that asks you to hold your DS sideways.
But there are also fancy screen dispositions:
• Natural: screens are arranged as they are on a DS.
• Vertical: screens are stacked on top of eachother (regardless of rotation).
• Horizontal: screens are placed next to eachother, top screen first.
Different sizing modes are provided too:
• Even: the screens are given the same size.
• Emphasize top: the bottom screen is kept at native resolution, the top screen is given all the remaining space.
• Emphasize bottom: vice-versa.
• Auto: emphasizes the screen which receives 3D graphics (typically the most important one), works well in NSMB for example. If 3D is displayed on both screens, behaves the same as Even.
There are still details to iron out, but so far those display settings are working rather well, and I like how they turned out.
Later on (probably after 0.6 at this rate), I want to work on non-local wifi, and some other things. For example, emulating the DS browser in melonDS and getting it to browse this site would be a cool feat. Can't tell how well it would do these days though, that browser was already subpar when it was released :P
There is also that Pokémon cart, which interests me. There are not a whole lot of players waiting to play that game, I guess, but it's interesting from a technical standpoint. The cart contains a Broadcom controller that seems to be quite capable, atleast enough to replace typical backup memory entirely. I just received a DS cart slot, so I'll be able to mess with it.
I am also very interested in that GBA-slot camera addon, but it's elusive as fuck. There is one game that comes with it (supposedly another game is compatible, but when I ordered that game, I received it without a camera), but when I search for it, all I can find is the DSi version. I need the DS version, the DSi version uses the DSi's builtin camera and thus doesn't come with the camera addon.
So, if you happen to have that camera addon, or know where I can get one... I am very interested.
Upscaling would be another thing to work on, according to popular demand.
This will need a new renderer, likely using OpenGL or a separate well-optimized software renderer. This is a bit of an issue.
On one hand, the current 3D renderer is meant to be accurate, and by that, I mean replicating the hardware's low-level behavior. The GPU is pretty much tailored for the DS, so I'm not positive that such a renderer can be made to render at higher resolutions and give good results. Not to mention the speed impact from that: rendering at 2x IR requires filling 4 times more pixels, and so on. The growth of performance requirement is quadratic.
Besides, people who want upscaling will generally want improved graphics. Texture filtering and maybe texture upscaling, for example. Or actual antialiasing (as opposed to the faked edge antialiasing the DS does). Which the current renderer will not support, because the DS doesn't do that.
It would make sense to implement these improvements in a hardware renderer. And also be less costy.
However, don't forget that the DS GPU is quirky. Some things would be difficult, or even impossible, to emulate correctly with OpenGL. One of them is how the DS keeps track of polygon IDs per pixel, but does so separately for opaque and translucent pixels. Those affect rendering of translucent and shadow polygons, and edge marking. Replicating the whole behavior in OpenGL would require a 16-bit stencil buffer, which is way atypical. Or maybe some cool trickery.
Which reminds me of blargSNES and its hardware renderer. While all the trickery we had pulled to render SNES graphics on the 3DS GPU was definitely cool, it took its toll on performance.
Sure, the average computer GPU is ways more powerful than the 3DS, but... yeah.
There is also the bit where the DS happily mixes 2D and 3D.
Old consoles featured 2D tile engines. Newer consoles feature 3D GPUs. The DS is inbetween, and... features both. A unique situation.
Situation which gives us the following problem: if we upscale 3D graphics, what should we do to 2D graphics?
And the answer pretty much depends on what the game draws. Something like NSMB could use a xBR filter, for example. But these filters don't work well on more complex graphics.
There is also the whole issue that the 2D renderer in melonDS is scanline-based. I have a good idea to implement upscaling in a non-intrusive way, but it will be an issue as far as filters are concerned. The 2D layers are added to the final framebuffer as they're rendered (rendering them separately would be slower). So, while the 3D layer is rendered separately and can have shit done to it, this doesn't apply to the 2D layers.
So most likely the 2D layers would just not be filtered at all.
Which reminds me of display capture, which allows to write rendered graphics to VRAM. Captured images have to be at the internal resolution. Those can also be modified by software before being displayed.
Which is less than ideal. For example, display capture can be used to display 3D graphics on both screens. While one screen will be rendering a 3D frame, the other will be rendering a captured version of its previous frame. The captured 3D frame is already degraded from 18-bit color to 15-bit due to how the hardware works. Throw upscaling in the mix and the captured 3D frame ends up being a downscaled version of the upscaled 3D frame, which would be re-upscaled with no filtering as it's displayed by the 2D engine. The quality loss would likely be visible and result in some bad flickering.
There would be ways to get around it, but it's neither trivial nor error-proof (see above-- the captured frame can very well be modified by software. There are also several ways a captured frame can be displayed, some games use large sprites).
The best solution in this kind of situation would be to disable upscaling entirely.
Oh well. Time will tell where this goes.
|13 comments have been posted.|