melonDS aims at providing fast and accurate Nintendo DS emulation. While it is still a work in progress, it has a pretty solid set of features:

• Nearly complete core (CPU, video, audio, ...)
• JIT recompiler for fast emulation
• OpenGL renderer, 3D upscaling
• RTC, microphone, lid close/open
• Joystick support
• Savestates
• Various display position/sizing/rotation modes
• (WIP) Wifi: local multiplayer, online connectivity
• (WIP) DSi emulation
• DLDI
• (WIP) GBA slot add-ons
• and more are planned!







Download melonDS

If you're running into trouble: Howto/FAQ
Pages:1234567···17
The local multiplayer saga, ep 10
Sep 8th 2022, by Arisotura
This tenth episode is going to close the first season of this saga. At this point, local multiplayer has been tested and found to be smooth in most cases, so we are mostly done with the actual wifi work.

We have also been porting our semaphore code so that our shared-memory based communication layer could work under Linux and macOS. This has shown to be a bit challenging, and I've been making several attempts to try and determine what was the best method for fast and reliable IPC.

What is left is mostly UI-related work. Handling several situations that can be problematic during multiplayer: supporting multiple controllers, maybe also things like multiple firmware settings, making sure the multiple running games don't write to the same save file, firmware file, and so on. I have a few ideas for that.

That will help turn local multiplayer support into a more finished product and less of a clunky experimental thing.

But for now, I'm going to take a break. I've been pouring a lot of my free time into this, and I think I need to relax for a lil' while.

I'm also going to debate the direction to take now with the rest of the melonDS team. There will be a second season to this saga, and it will be about implementing netplay. The results of all this wifi work are well beyond my original expectations, and now we have big ideas and hopes for this. But first, I want to take a collective decision about whether to make a release before season 2, whether to go straight for melonDS 1.0, these things. Stable local multiplayer was one of the must-haves for melonDS 1.0, but that's not all there is.

I'm also collecting ideas for netplay, so if you know more about this than I do (which is, relatively little), feel free to reach out to me!
24 comments (last by A) | Post a comment
The local multiplayer saga, ep 9
Aug 28th 2022, by Arisotura
In the previous post I said the slowdown problems in multiplayer games were fixed, but actually, they weren't. Oops.

I looked deeper into the issue. I couldn't let that slip. But I also didn't really have an idea what could be causing it or where to begin looking.

I thought about bit0 in USCOMPARE, again. Basically, when they're connected to a host and receiving beacons from it, games do that weird little dance where they take the beacon's timestamp value, add some offset to it, and write that to USCOMPARE, with bit0 set.

A bit of background. In the DS wifi module, there are two ways of triggering IRQ14: BEACON_COUNT and USCOMPARE. The former triggers IRQ14 every time the BEACON_COUNT timer reaches zero, the latter triggers it when USCOUNT matches USCOMPARE. All fine and dandy.

Back then, I had done hardware tests to figure out what bit0 in USCOMPARE does, given it's a special write-only bit. I had observed that it blocks BEACON_COUNT from triggering IRQ14 until USCOUNT matches USCOMPARE, effectively ensuring that the next IRQ14 will be triggered by USCOMPARE.

But that still didn't quite make sense. Here, the value written to USCOMPARE was based off the beacon's timestamp, which was basically the host's USCOUNT register, which was obviously different than the client's. So I couldn't really see what that was supposed to achieve.

Yesterday, as I went to bed, I had an idea. What if, when receiving a beacon frame with the right BSSID, the DS automatically sets its USCOUNT register to the beacon's timestamp?

But you'd think that sounds far-fetched? I thought so too.

I tested it anyway this morning. Guess what I found out?

... read more
19 comments (last by poudink) | Post a comment
The local multiplayer saga, ep 8
Aug 26th 2022, by Arisotura
Remember the issue I mentioned in the last post? I fixed that since then. Turns out Nintendo's wifi code does quite a few weird things during a multiplayer connection, and we need to deal with these to keep a stable connection.

In this case, it was related to the power-down registers. The game code may regularly decide to stop and restart the wifi hardware to reset some of its state, and I found out that sometimes that would happen while a frame was being received, which caused a bunch of problems in melonDS.

The code uses register W_POWERFORCE to turn off the wifi transceiver. But, that's the thing, we thought that operation was instant. However, the game had a loop waiting on some status registers after writing to W_POWERFORCE, which implied that the shutdown operation might not always be instant. Hmm...

A couple hardware tests later, this theory is confirmed: if the transceiver is turned off while transmitting or receiving a frame, it will first finish that operation before actually turning off. There we go.

Implementing that into melonDS didn't fix the issue with NSMB minigames, but it helped make local-multiplayer connections more stable overall, for example the slowdowns I had observed in MvsL seem to be gone now. Then I also fixed the issue with the minigames, which happened to be caused by some leftover code that was bad and no longer needed.

As it seems that I have gotten local multiplayer pretty stable and resilient now, folks from our community have been stress-testing it. For example:



All in all, certainly not bad. Consider that there is no way this would ever work reliably on the current melonDS release.

However, we're not done yet. While we can claim victory on the horseman of wifi, there's still a bunch of things to do for a better user experience.

... read more
12 comments (last by HeroMayCry) | Post a comment
The local multiplayer saga, ep 7
Aug 25th 2022, by Arisotura
At this point, we should have this published on Netflix or something :P

In the previous post, we saw that we had lag in some games because the client was sometimes not replying, so I had to deal with that issue.

I introduced the new local-multiplayer data exchange/sync mechanism in ep 2, and since then it has seen several iterations as I made it simpler and more efficient.

For one, I used special-purpose microsecond counters for this instead of relying on USCOUNT. The first issue there was with that was that USCOUNT can be disabled or modified at any time by software, which isn't ideal if we're going to rely on it for sync purposes. Second benefit is that now, when a client connects, I can just sync up its counter to the host's, and not have to deal with any offset between the two. So, all in all, it's more reliable this way, and a lot easier to deal with.

Next change was that I got rid of the specific 'sync point' messages and simply attached timestamps to packets. This reduces the amount of data being exchanged, and let me simplify the sync mechanism too. Now, when receiving a packet from the host, clients know when to actually start emulating reception of that packet, and how long they can keep running on their own before having to wait for another packet from the host. This way, clients can't run too far ahead of the host, and we also ensure the host doesn't run too far ahead by dropping any replies that are too old. All in all, this system is more efficient.

Lately, I also reworked how replies are dealt with, to make things more efficient and reliable, and to deal with the issue at hand in this post. Replies go through a separate FIFO, and instead of being broadcast to everybody, they're only received by the host, which mirrors the way local-multiplayer communication works in real life.

To deal with the problem of clients sometimes not being ready to reply, I simply made clients send a blank if they receive a CMD frame they can't reply to. The blank is a frame with no data, that the host ignores -- it's just there so that there is something to receive, so that we don't hit the timeout.

A problem with this system is that it won't function well if there are two or more concurrent local-multiplayer games running on the same computer, but I don't think that's a use case we need to worry about.

Anyway, this isn't perfect yet, but the results are encouraging. Tetris DS and Mario Kart DS connect with no hitch now, so that issue is dealt with. However, I'm noticing some regressions. For example, NSMB minigames won't connect: I immediately get a comm error screen on the client. I looked into it quickly, and found out that the client is doing weird things (like occasionally setting RXBUF_RDCSR to 0x05FE, which doesn't look right), but I need to look deeper into it. It's possible that the issue was always here but was less apparent due to the different way of doing things. I have also seen some occasional slowdown in NSMB and Mario Kart, so that may be related.

Oh well, we'll get there. It's getting close at this point.
6 comments (last by Rayyan) | Post a comment
The local multiplayer saga, ep 6
Aug 22nd 2022, by Arisotura
For melonDS, there are three horsemen of apocalypse. Over the years, we have been able to deal with nearly everything the DS has thrown at us, but there are three main categories of long-standing issues: timing issues, precision errors in the OpenGL renderer, and local multiplayer.

And I certainly did deal a blow to that horseman back in 2017 when a working local multiplayer connection was emulated for the first time, but of course it wasn't going to be all. While the connection worked, it was always finicky, and getting it to work was a bit like voodoo magic. After melonDS 0.4 was released, melonDS changed from being a mere curiosity to being 'the wifi emulator', and many people came in for that feature. And we just kept telling them to disable their framerate limiter, pray, sacrifice a goat or two to the wifi deities, and hope things will work.

It was certainly substandard for melonDS. One of the main reasons why it stayed that way for long was, besides having to work on other fronts, that it was a precarious position of equilibrium. Any attempts at improving wifi emulation resulted in a more unstable connection. I thought it would take an overhaul of how the connection is emulated to make it work reliably, but I always had that fear of putting much time and effort into it only to get terrible results, so the status quo was maintained. But you know how it is: when making your emulation more accurate gives worse results, it means you're missing something else. After 5 years of business, we're no stranger to this.

So what does it take to beat the horseman of wifi? Two things: a stable, reliable medium of communication, and reverse-engineering of the DS wifi hardware, which is full of fun little details that sometimes matter a lot.

But, of course, it's not going to go down without resistance. It's a horseman of apocalypse. It will put up a fight at every stage.

Lately, we got to a point where local multiplayer connections are pretty stable, and it's possible to reach decent speeds on good hardware. For reference, my 2014 laptop Crepe can run New Super Mario Bros. MvsL multiplayer at near-fullspeed with the JIT on. I'm confident that any decent computer will be up to the task when we're done.

But, of course, that's with well-behaving games, where everybody sends their packets as they should and everything is good. While we were testing other games, it appeared to us that there are... cursed games. Tetris DS and Mario Kart DS, for example, have exhibited intense lag while trying to get a multiplayer game going.

I looked at what was going on in Tetris DS. The lag is caused by the host hitting the timeout when waiting for incoming client replies, because sometimes the client isn't sending any reply at all. It was egregious because I set the timeout to a ridiculous value (500 milliseconds) to make sure everything would get through reliably. A lower timeout reduces the lag in this situation, but also increases the chance of missing incoming packets when we need them, so this is essentially a compromise (we might end up making the timeout user-configurable).

But, why are we not receiving replies from the client to begin with? In a previous post, I have mentioned that the DS, when it acts as a multiplayer client, always replies to CMD frames, even when no reply frame is configured, and melonDS emulates this behavior too.

... read more
11 comments (last by DuelingPotato) | Post a comment
The local multiplayer saga, ep 5
Aug 20th 2022, by Arisotura
It keeps going!

Lately, I have been working on faster IPC, so I could get a more accurate idea of how much it would take to emulate local multiplayer at acceptable speeds, and whether my ideas were feasible at all. First order was getting rid of the old inefficient BSD sockets.

Instead, I built a data exchange system based on shared memory. The idea is simple: have a large block of memory shared between all melonDS instances, where you have a FIFO buffer for exchanging wifi packets, another FIFO buffer for exchanging sync points, and a small header containing some useful status bits. Then each instance has a set of semaphores that other instances can use to signal when new packets or sync points are sent. All fine and dandy.

I figured this would be easily taken care of, with QSharedMemory and QSystemSemaphore. And I ran into my first issue: QSystemSemaphore sucks. Basically, it isn't possible to wait on a QSystemSemaphore with a timeout, which means that waiting on it will block until it is signaled. What if, for whatever reason, the other side isn't signaling the semaphore? That's a deadlock. In our situation (and probably many others), this is unacceptable. The problem has been reported to the Qt team in 2008 and they don't seem very interested in fixing it. So this basically means I had to code my own thing, directly using the OS's named semaphores. Currently, I have only coded it for Windows, so it will have to be ported to Linux and macOS.

On the other hand, QSharedMemory works like a charm, so there's that.

I also reworked the sync system some. It's currently not perfect, but atleast now I know I'm not being held back by faulty emulation: as long as no frames are being dropped or delayed, the communication should work fine.

Overall, this gives much better performance than the previous iteration: NSMB multiplayer reaches near-fullspeed on my laptop Crepe, and that's with the interpreter. There are still issues to iron out, but this gives me hope that we can pull off fun multiplayer features in melonDS.

Also worth noting that there is another candidate for optimization, which I haven't talked much about yet: the wifi module itself.

The wifi module, you say? What's worth optimizing in there?

... read more
9 comments (last by Sivecano) | Post a comment
The local multiplayer saga, ep 4
Aug 17th 2022, by Arisotura
If you happen to remember the previous episode, you'd know I'd ran into an issue caused by short CMD frames, as melonDS was erroneously replying to these with long replies. In that situation, the hardware skips the reply and sends a default empty reply, and I had yet to implement that feature properly.

So I did.

Except I still had data loss problems and random hangs in Pictochat.

I ran several hardware tests to make sure I wasn't missing any detail of this feature, and all seemed correct.

So I turned to debugging Pictochat. I noticed that when the client was getting stuck, it just stopped sending reply frames entirely. I backtracked and backtracked to find out why it was doing that, and finally found the cause. It was quite stupid.

Basically, when the DS sends a reply frame, it marks the previously sent frame by changing the first byte of its header to 0x01. All fine and dandy. melonDS of course does the same thing.

Except that when I added support for substituting a frame with a default-empty frame, I copypasted some code without paying enough attention to what it did. And that code was setting the internal reply frame address to zero, which caused the above frame header manipulation to fail unless the previous frame happened to be at address zero (basically, the beginning of the wifi RAM).

Oops.

After fixing this oversight, I was rewarded with a perfectly stable and smooth connection. No hiccups, no data loss, no hanging. I didn't even see the host try to retransmit any frames.

... read more
11 comments (last by fintogive) | Post a comment
The local multiplayer saga, ep 3
Aug 16th 2022, by Arisotura
From the previous post:

However, there is still data loss in the client->host direction, which I need to investigate further.

So I've been doing exactly that. I'm not done yet, because the DS wifi is a rabbit hole, so many little details everywhere and especially in the local-multiplayer specific features.

If you have messed with wifi in melonDS before, you might have seen this message appear in the console:

wifi: !! unusual TXSLOT_CMD bits set C000

What is this about? The value for TXSLOT_CMD has two known parts. Bits 0-11 are the address of the CMD frame, and bit 15 tells whether the CMD slot is enabled. The rest is undocumented as of now.

In this situation, the game is setting bit 14, which prevents the hardware from automatically filling in the CMD frame's sequence number. This feature comes in handy if the host needs to retransmit a CMD frame for whatever reasons (for example, if some clients failed to respond). All in all, nothing too difficult to deal with.

Except implementing it in melonDS worsened the problem of client->host data loss in Pictochat. So I just ignored that feature, which made the transfers work nearly perfectly (albeit with some apparent lag). But you know our standards: that's not correct emulation, and also more likely to lead to other problems.

So I investigated the data flow. There are a few things of note in the Pictochat data flow:

* The host sends a CMD frame every 4 milliseconds. Each frame can be addressed to a maximum of 3 clients. Thus, the host needs 5 frames to poll every possible client. This is probably done to avoid monopolizing the wifi channel too much.

* The client, when receiving a CMD frame not addressed to it, will interact with the unknown hardware registers 0x244 and 0x228. Register 0x244, in this case, seems to be used to inhibit the transfer of a reply frame, which I'm not sure why they do it when no reply would be sent anyway. I don't know what register 0x228 does, I haven't observed any effects in my tests. Over all, this part doesn't seem very important.

... read more
4 comments (last by mrjfilippo) | Post a comment
The local multiplayer saga, ep 2
Aug 5th 2022, by Arisotura
Slowly but steadily, I'm working on it. For now, I'm keeping my work private, so I'm not bothered by people wanting to try an unfinished branch, reporting issues with it, etc.

We assume here that you're familiar with local multiplayer terms. If you're not: read this.

I introduced a crude synchronization mechanism where the host of a local multiplayer game can instruct clients to stay in sync. When a client connects, the host sends its USCOUNTER (the wifi system's microsecond counter) value to that client, which can then compare that to its own USCOUNTER value: the point a connection is established is considered the reference point, and the client will then wait for the host to allow it to run, so it doesn't get too far ahead or behind.

Thus, the host can enforce sync (and also wait for clients to have caught up) before sending a CMD frame to its clients, thus ensuring that everybody is at the same point at that time. It can then send its CMD frame and allow clients to run ahead for a given amount of time that covers the entire MP exchange process.

The results aren't perfect, but they are somewhat promising.

For example, Pictochat doesn't seem to get stuck trying to send messages. I have only tested with two instances for now, but they seem to stay in sync fine.

However, there is still data loss in the client->host direction, which I need to investigate further.

I suspect there might be issues with ack frames being received too late. I intend to just fake them, to simplify things and reduce the amount of data being transferred. I made a quick attempt at that yesterday, but so far it's not working as intended.

It may also be caused by lacking emulation of the error handling processes in hardware (like automatically resending the CMD frame if some clients failed to respond, for example). I will have to do some more reverse-engineering to determine how that works in detail.

... read more
11 comments (last by MPS64) | Post a comment
Plans for local multiplayer
Jul 19th 2022, by Arisotura
I know I haven't been active a lot lately, the usual.

Well, I tried looking into the DSi sound app, and found that it crashes due to a NULL access. I tried debugging that, but it seems to be some weird timing problem and I don't know why that crash doesn't happen on hardware. This will need further investigation.

Anyway, local multiplayer.

It's one of the big things to take care of, now. We have been mostly putting it aside over time, favoring various popular requests, quality of life improvements, and fixes to DSi emulation. But we can't keep pushing it away like that. We have decided that improved local multiplayer is a must-have for melonDS 1.0, and we aren't going to keep making 0.9.247 type releases forever.

(besides, 0.9.x releases aren't very pleasant, for the same reason why I don't like $29.99 type prices: 9 is brown)

Anyway, I brainstormed some ideas for reliable local multiplayer. I have already explained before why local multiplayer is so finicky to implement, but the basic story is that we can't just run two melonDS instances and have them hurl packets at eachother and pray that things will work out. You might already know that we keep telling people to disable the framerate limiter if trying local multiplayer, and that's linked to this too.

We need a smarter mechanism for synchronizing melonDS instances taking part in local multiplayer. I have some ideas in mind for this, but I have to see how well they will translate to reality. I'm concerned about the performance implications this could have.

If this is successful, there are other possible quality of life improvements that relate to local multiplayer. For example, how to deal with user input when multiple melonDS instances are running on the same machine? You might want them to have different input mappings, but how do we deal with that without making things too complicated?

The craziest thing to achieve would be netplay, akin to Citra. Well, of course, there is no chance local multiplayer could work over the internet, but there are probably ways around this; we can look into how popular emulators handle netplay.

Oh well, we'll see how this goes, I guess.
40 comments (last by keisui) | Post a comment
Pages:1234567···17