When your innocuous little code goes kaboom
So until we get something amazing to release, we're going to post some technical shito again.


As said in the previous post, I've been coding savestates. The base idea seems simple enough, just throw all the emulator's state into a file (or read it from that file if you were loading a state), make extra sure that you're not missing anything important, regenerating things that need it when loading, etc...


So I first designed the bases of the implementation:

• A savestate file is divided in several sections. Those can be ordered arbitrarily. The file format itself is very simple.

• To reduce complexity, we only handle loading/saving between frames.

• Some questions are still not clear, and I still need a way to avoid loading a savestate over the wrong ROM. (or I could allow it as a fun Easter egg, even if it generally results in a crash)


Then, the gruelling work, storing all the emulator state.


Including, you guess, the event scheduler, which is an array of event structures with an entry per possible event source. Just write that to the savestate file, right?

Think again.

typedef struct
{
   void (*Func)(u32 param);
   s32 WaitCycles;
   u32 Param;

} SchedEvent;


This is the event structure. We have a handler for when the event fires, how many cycles remain before it fires, and a parameter that is passed to the handler. The idea is that we can use different handlers for one event source, acting sort of like a state machine, which is what is done in display emulation for example (HBlank, next scanline, frame end).

The handler is a function pointer. Which means that storing it as-is in a file is a bad idea. For various reasons, there is absolutely no guarantee that the functions those pointers refer to will be at the same addresses between two runs. In fact, it even seems completely impossible, because modern-day OSes will randomize the address at which the melonDS process is loaded. Then you have things like using savestates across different versions or even different builds...


Well, good thing I thought about that before trying to run that code.


(now I also realized that there are several other instances of dumping structure arrays straight to the file, namely in the 3D pipeline, which might cause compatibility issues between builds as compilers might decide to order the structures differently; that will have to be dealt with too)


So, what do we do now?


As I originally wanted to keep the evnt system fairly flexible, an idea I had was to store relative function pointers, ie basing them off of main()'s address or something like that. While this would work for making savestates reusable between runs, it wouldn't fix any of the other issues. For example, run a different melonDS build and the event functions may be at different places, meaning the relative pointers wouldn't point to them anymore.


So instead, the way I went with was to store the handlers as indexes.

The idea is to keep a table of event handlers. When saving, we lookup the table to convert each event entry's handler to an index. When loading, we do the conversion backwards. Et voila, no problem. There are a couple safety checks around it, for example, to prevent haxing via a bogus savestate file containing an event handler ID that is out of range.

The main disadvantage of this method is that any function that is to be used as an event handler needs to be added to the event handler table.

But atleast our savestates are portable. I guess we win.


That being said, if any of you folks has a better idea for this problem, I'd like to hear about it.
gettin' there
Apologies for not keeping this up to date lately.

Anyway, savestates are being coded, and we need your help to test them out. More about this in this reddit post.

Other than that, I tried some hax on my new green DSi. Was able to install unlaunch. I'm not yet able to run DSi homebrew though-- ndstool builds oldschool DS ROM headers, which causes unlaunch to load these ROMs in DS mode. I have yet to figure out how to make ndstool build DSi ROM headers.

On the real life side... I'm getting there. Depression is a bitch tho. And capitalism too. I would feel a lot better if that didn't exist.
the plaaaaaans: getting staaaaaarted
this is spaaaaaarta





Meet Green.

It's a big DSi. Pretty cool, eh?

Quick inspection reveals that everything is alright. It was factory-reset, tho, so it might not have Flipnote on it. But whatever, we'll find a way. actually it does still have Flipnote. All good!


Still no news of the other order. If you happen to remember, it was returned to sender, and the sender notified me about it and asked me what they should do. I gave them another address to ship to, and even offered to pay for the reship, but since then, silence.

I guess I'll have to give the seller a reminder. Or ask for a refund.


Regardless, I think we can get started with the interesting things... once I'm a bit more settled.
The 'timing' branch
In the meantime, I figured I'd give this a try.

Basic idea is that emulating timings with reasonable accuracy will require emulating the ARM9 caches.

I want to see if I can emulate those without completely killing performance. If I pull it off, there'll be several benefits:

* much more accurate timings, fixing games that do weird things
* homebrew developers using melonDS wouldn't fall into the typical 'forgot to flush/invalidate cache' pitfall that reveals itself when testing on hardware
* if we take it even further, we can emulate the MPU (memory protection unit), which means emulating crashes

I guess that if I manage to get it going, there will be a sorta beta release so I can get feedback about this, whether it's killing performance, etc...
Updates on the plaaaaaaans
The package was, surprise, returned to sender, because some idiot can't find a mailbox that is street-facing and not hidden or anything fancy. Hell, it's right next to the building's front gate.

Whatever.

I'm trying to see if I can have it shipped at another address which I know should work.

I ordered another one, too (that was before the seller for the first one contacted me about the package), but it won't be shipped until a week.


I guess it can't be a bad idea to get two consoles either way. Like, if one turns out to be less/not haxable, or if I need to do hardmods...
oh well
The plan was to receive the package and post a picture of the contents as a reveal, but obviously this is not going as planned. So I guess a shitty reveal is better than teasing people and letting them hang in the void forever.

So here it is.


The next direction for this is DSi emulation.


In the meantime, depending on how things go, I might do some work on GPU accuracy again, but for now I'm taking a few days of vacation.

Upscaling is also something I might try someday. I have some ideas for it, but we'll see.


See ya!
blarg.
Did the package arrive since then?

Of couuuuurse not.

I guess I'm in for another round of fighting the incompetent postal services. Yay.

The sender also didn't provide a tracking number so I have no idea what's going on. This is so fucking amazing.
Thank you all! :)
I don't always take the time to respond to your comments on this site, I'm sorry about it.

But I read your comments. I always do.

And I want to say, thank you. Those comments are real nice.

It's nice to see that you are still following this blog, despite the lack of modern subcription/following features.


(it's also nice to note that the viagra spammer stopped posting spam comments on the blog's second entry. not that their attempts at making links ever worked)


I have been ordering something that should arrive next week, hopefully Monday :)

This is related to the surprise. You'll know soon :)
plaaaaans
I finally intend to keep going with melonDS, in one of the possible directions.


You will know more about this soon, but for now, it's a surprise ;)
melonDS status, and plans for the future
I figured I would write this, to give you an idea of melonDS's current status, what has been done and what remains to do.

To reply to the comments on the previous post: melonDS is already open-source, and it has always been. You will find the source code on Github. You can do whatever you want with it, as long as you are respecting the GPL.

So, here's a quick list of how things are and what needs to be done, from the top of my head. Searching the codebase for TODO/FIXME/HACK will find all the items.


CPU

All the instructions supported by the ARM7 and ARM9 are implemented.

ARM9-only instructions are enforced, altough this may be incomplete. It's unclear how some of them behave on the ARM7: some throw undefined instruction exceptions, but some seem to do nothing or behave like another instruction.

Cycle counting is rather crude and likely to be wrong for the more complex ARM9. Speaking of, the MPU and cache still have to be implemented (hopefully without killing performance). The first would allow emulating some game/homebrew crashes that happen on hardware, the second would be a requirement for reasonably accurate timings.


System

All DMA types are implemented, with reasonably accurate timings and operation. Running DMA and ARM9 simultaneously (possible under specific conditions) isn't implemented.

Timers might need checking/rework?

Memory map is accurate, including things like overlapping VRAM banks.

ROM/SPI transfers are emulated with proper delays, and should function like their hardware counterparts, minus some uncovered details (see all-caps printf() calls).

ARM9-side div/sqrt are emulated with delays.


2D

All features are implemented. Even the rarely used 'main memory display FIFO'.

OBJ mosaic may be wrong, especially with rotated/scaled sprites.

DISPCNT latching is missing. Other registers may be latched, that hasn't been researched much yet.

VRAM access timings aren't implemented.


3D

Most features are implemented.

The geometry engine is emulated mostly accurately. Polygon/vertex limits are enforced. Culling and clipping haven't been researched closely and may produce results different from hardware.

GXFIFO overflow isn't implemented properly (should incur penalty for ARM9 and presumably ARM7).

Misc things like effects of invalid operations (incomplete polygons etc) may not be implemented.

The rendering engine is emulated with much more accuracy than needed for your average game. Much research went into figuring out how the rasterizer works. melonDS is not pixel-perfect but close. Pointless besides for passing aging cart tests.

Antialiasing over shadow pixels is glitched.

The scanline cache is not emulated (requires figuring out rasterizer timings, including VRAM access and all).


Sound

Capture addition and source selection are missing (have yet to find something that uses them).

Other than that, the mixer is complete and emulated with sample accuracy.

Memory access timing isn't emulated, however buffering mechanisms (FIFO) are implemented.


Wifi

Much of wifi still has to be researched.

The current implementation allows some form of local multiplayer (and connecting to the internet with proper support), but communication errors aren't properly reported, which results in bad things. So far my attempts at fixing that only made things worse.

Wifi is subpar compared to the rest of the emulator :P



With that in mind, where do we go next?


I'm not sure about attempting to continue the JIT. Considering I went for accuracy, preserving it at a reasonable level with a JIT is going to prove tricky, even moreso when you're emulating two simultaneous CPUs and a bunch of extra hardware. Make it too accurate and the synchronization overhead negates any speed benefits from the JIT, but going too far in the other direction will inevitably break games.


DSi emulation is another of the possible avenues. There's not a lot of DSiware to play, and even less cartridge games, but regardless, there are things to be done there.


Or I might go and emulate something entirely new for me? N64? Dreamcast? Xbox?


I wonder.