https://melonds.kuribo64.net The latest news on melonDS. https://melonds.kuribo64.net/comments.php?id=212 Fri, 30 Aug 2024 08:44:21 +0000
Anyway, thank you all for your kind comments, it means a lot!

I haven't been working much on melonDS lately, mostly been relaxing by working on another personal project.

As far as melonDS is concerned, my plan is to release version 1.0 by November, ideally around the project's birthday. It is going to be 8 years old, by the way. I am amazed that it has been going this far.


So here's a quick list of the things that need to be done until then:

Implementing the multi-window feature. It shouldn't be very difficult, I had already coded the required support a while ago. It's mostly matter of integrating it into the UI and ensuring everything works correctly.

Fixing up the JIT to work with multiple instances. Generic is the JIT expert over here, so that'll be for him to deal with.

Maybe fixing up LAN, if I ever manage (and if I feel more motivated to work on it).

Fixing all the issues that have cropped up during the big refactor. There's also still a bunch of code cleanup to do. We will also need a bunch of testing and QA to make sure the 1.0 release works as intended.

Trying to remember all the changes we have made since 0.9.5, because there's a lot.


After the release, I will take a break from melonDS. I don't want to abandon the project, but I also have a lot of other things I want to do. Things indirectly related to melonDS (like updating this site, maybe things like a better, not outdated screenshot gallery). Other personal projects. Real life changes and stuff.]]> https://melonds.kuribo64.net/comments.php?id=212 https://melonds.kuribo64.net/comments.php?id=211 Sat, 17 Aug 2024 08:45:49 +0000
The scope of this feature means it requires more development and testing time than I would be able to do before November.

You can tell I was in a really active period. I'm trying ADHD meds, again. They work, but I overestimated things, and I burnt out.

The recent wifi stuff was the straw that broke the camel's back, I guess. I mean, why did I spend so long implementing the powersaving stuff, channels, etc, in the first place? That's right, I was trying to fix games that had trouble connecting. I thought I had figured it out, I had a fix that worked well for me, but then I kept getting reports that it wasn't working. I did more of the same a few days ago, once again thinking I had a decent fix, and nope. Only makes things slightly better, or doesn't make any change. This feels utterly defeating. Like nothing I do will ever fix it.

It's kinda like when I make posts about my difficulties and all I get in the comments is "can you add feature X" "can you fix Y". I'm not a programming robot, you know.

I'm burnt out, I guess. I want to put out melonDS 1.0, but I need a big break from melonDS.]]> https://melonds.kuribo64.net/comments.php?id=211 https://melonds.kuribo64.net/comments.php?id=210 Wed, 14 Aug 2024 22:24:27 +0000
I didn't quite know where to look. The LAN interface itself functions the same way as its season2 counterpart. The only thing I could think of was the improvements that we'd done to wifi emulation since season2.

I fired up my other computer and gave LAN a test. I had to do a bit of setup to get both computers connected over ethernet, since wifi has too much latency. But then I observed the same thing that had been reported to me: I could sometimes get a connection going, but it had a high failure rate.

To further confirm it, I even happened to get a connection failure over local multiplayer. It was just a lot less likely there.

When logging the wifi traffic, I observed something peculiar: the client would send a 802.11 authentication frame and receive another authentication frame from the host, then it would send an association frame, but at that point the host had entered power saving mode. It would only leave power saving mode a lot later, and that was too late.

The LAN interface does something a bit particular: due to how it functions, it may receive frames at any time, and these are added to a receive queue to be consumed where needed. To avoid clogging up that queue, frames that are more than 16 milliseconds old are considered stale and are deleted. This should normally be more than enough: when the wifi system is active, it checks for incoming frames fairly often, that is atleast every 0.5 milliseconds.

I could see how this was a problem in the aforementioned case, but raising the 'stale frame' threshold as far as 500ms did not fix the problem. So I had to approach it differently, namely, by considering how things work in wifi land.

The host sends beacon frames every ~200 milliseconds, to advertise its presence to potential clients. A client that wants to connect will then initiate the process by sending the host an authentication frame after receiving a beacon, and so on. The authentication+association exchange is aligned to beacon frames, and that is important.

The host uses power saving mode between beacons. The basic process is as follows: send beacon frame, wait for a while to see if any clients want to connect, then enter power saving mode until next beacon. There is a timer (W_POST_BEACON) that determines how long to wait before entering power saving mode. It is typically set to 10 milliseconds or so, which is more than enough for an authentication+association exchange.

But what happens when we try to emulate wifi communication over different protocols with different constraints? We have a synchronization mechanism that ensures everything is received on time, but it only kicks in after a client has connected. Before that, things are much less reliable, and the authentication/association frames may lag behind and arrive too late (like when the host has already entered power saving mode). This is exactly what is happening with LAN here. Sometimes the frames may be delivered the next time the host leaves power saving mode, sometimes it works but sometimes not.

And why was this not a problem before? Because power saving mode wasn't emulated correctly. It's all part of this post: I implemented channels, but things broke precisely because of the power saving bugs, and you know how this goes. Either way, we were kinda just receiving things at all times, which made things work.

So, how do we fix this bug, anyway? I could think of two possible ways.

First way, extending the synchronization mechanism to kick in earlier, during the auth+assoc process or even earlier. It could be worth exploring, but it is also likely to cause more problems than it would fix. The sync mechanism was designed around the multiplayer protocol especially, so I don't know how well it would perform outside of that use case, or if it could cause problems when connecting more than two players.

Second way, extending the post-beacon interval when receiving authentication or association frames. It is simply done by increasing W_POST_BEACON by 10 or so, making sure there will be enough time for the auth+assoc process to complete. However, it's a hack. It deviates from hardware behavior, and if anything is coded to expect a specific post-beacon interval (instead of just waiting for the post-beacon IRQ), it will cause problems.

I decided to experiment with the second way. It is less invasive, considering how heavy-handed the sync mechanism is. I also doubt that it will ever cause problems, since the post-beacon interval stuff is mostly used for power saving purposes. But if it ever does cause problems, we can always change it back (and we get to write another juicy tech blog post about it, heh).

It's a bit anticlimactic given our standards, but remember, emulation is a bunch of compromises. Especially when we have to emulate low-level wifi communication with its peculiar timing requirements.

In the meantime, this hack seems to have fixed the problems I had observed with getting connections going over LAN. It seems to work pretty smoothly now.]]> https://melonds.kuribo64.net/comments.php?id=210 https://melonds.kuribo64.net/comments.php?id=209 Sat, 10 Aug 2024 21:34:09 +0000
You may now grab it over at Github, in the Actions tab (you will need a Github account).

The way this works is the same, so I will simply copy-paste from that old post:


How to get a LAN game going:

1. On the host machine: open melonDS, then System -> Multiplayer -> Host LAN game. You enter a player name and you're good to go.

2. On client machines: open melonDS, then System -> Multiplayer -> Join LAN game. Enter your player name there, then it should list any existing LAN games. If not, you can always try using the direct connect button.

3. Once all sides are connected to the LAN game, you can open a ROM on each machine and try getting a game going. Do note that you will need a good local network for this to work. Ethernet should work, but anything else may fail if the latency can get too high. In my experience, wifi has too much latency.


This marks the end of season 3, and the beginning of season 4: netplay. Ironically, netplay was the original goal of season 2 -- LAN happened more or less as an experimental side project, and it happened to work out rather well, so it stuck around.

So, stay tuned for the netplay adventures!


Also, we are working out how to properly label melonDS builds from the CI, releases and nightlies alike. When we figure it out, I will probably add the download section for nightlies.]]> https://melonds.kuribo64.net/comments.php?id=209 https://melonds.kuribo64.net/comments.php?id=208 Fri, 09 Aug 2024 19:13:14 +0000
Namely, the downloads page didn't do our project justice. It was just a very plain listing of all the melonDS releases since 0.1. Technically, it was a static page that I manually updated after each release.

But this is no more! Go check out the new downloads page.

Now it shows just the last melonDS release. However, if you go down, there is an extra page that will list all the older releases, just like the old download page. Except it's not a static page, so new releases can be added without having to modify everything by hand. I am even thinking of adding a Github hook to post releases automatically when they are posted there, but one thing at a time.

There is also another section named 'Misc. tools'. For now, there's nothing in it, but I plan to add all sorts of tools that are relevant to melonDS or DS emulation in general. For example the DSi dumper comes to mind -- there would be a stable place to get it rather than having to dig into the blog. I will probably add that in real quick.

(edit- I did; after 3 years, about time)

And there's a last section, that isn't finished yet: nightly builds.

The idea was to provide access to builds from the Github CI without requiring users to have a Github account. I coded a Github hook and a backend for it, which downloads the builds from Github onto the server, and keeps the last 100 commits. However, this raises a few questions.

We need to make sure nightly builds don't spread and cause confusion. Right now, any melonDS build just uses the same version number as the last release, which isn't ideal. We would need a way to properly identify builds to avoid confusion.

I also hope that releasing nightly builds doesn't end up killing the incentive to do proper releases.

So we need to plan this out a bit, but hopefully we will take care of it all soon.


Have fun!]]> https://melonds.kuribo64.net/comments.php?id=208 https://melonds.kuribo64.net/comments.php?id=207 Wed, 07 Aug 2024 22:25:07 +0000 season3 branch.

The plan was to backport the LAN and netplay features to a recent branch, since the old season2 branch was way outdated. It is over one year old, and a lot of refactoring has happened since then. I figured that backporting them would be easier than trying to get season2 up to date, so that was why season3 was started.

I didn't even realize season2 was that old before I checked. Shows how good my perception of time is, really.


Anyway, the backporting is going smoothly so far. The LAN feature is fully ported, and it seems to be functioning. Also, JesseTG helpfully moved the network interfaces to their own module, so different frontends can reuse them. I based my work on that, and added an interface to switch between multiplayer modes (local, LAN, netplay).

All that is missing is some polish on the UI side. I will take care of the biggest problems, make sure it all works smoothly, and we will merge season3 very quickly.

This will enable you to test LAN multiplayer by simply grabbing our nightly builds (from Github's Actions tab; you need a Github account).


After that, I will tackle netplay in what will be season4. For now, I brought in the netplay related code, but I just commented out a lot of it so melonDS would build. It is nowhere near functional for now, and there is more work with it.

But on the other hand, several of the issues I had encountered during the original implementation are now solved.

For example, my original idea to get multiple instances started with the exact same state was to use the savestate system. Take a snapshot of one instance's state, send it over the network to other instances, which can just apply it and be all set (or so I hope). Sounds easy enough. Except the savestate system was coded to deal with files directly. I had worked around it in a very ugly way, but the issue has since been fixed, the savestate system operates on memory chunks now.

In a similar vein, I had hardcoded the RTC in season2 for testing purposes. Games use the RTC to seed their RNGs, so for netplay to work it is crucial that every side starts with the same RTC time or they will desync. The old RTC would just directly read the system time, so there was no control over it. But the new RTC would let us start with any time just fine. The time counting is also synchronous to the emulator core, so it will not desync during netplay.

And of course, there is the object of the big refactor: multi-instance support. The old netplay system had to deal with multiple separate processes and keep them in sync, which is far from ideal. Say one of the processes crashes, or hangs, how do we deal with such problems? But now, all the required emulator instances can be contained within one process, which makes things a lot easier to deal with.

Now it is mostly matter of reworking the old netplay code to take all this into account. But there are also other things I want to fix. For example, reworking the network protocol to work all on one single port, rather than using several different ports. If port forwarding is required, it would be a net improvement.

There is also some reworking required on the input front, especially touchscreen input. Basically, as of now, melonDS just receives mouse events and feeds them straight to the core. This has the problem that it's not compatible with netplay, because we need to be able to reproduce exactly the same inputs on all sides. A solution would be to set the touchscreen coordinates once per frame, like we do for button input. But this is also not ideal: DS games may sample the touchscreen multiple times per frame for higher precision, and it seems that some games actually need it.

We haven't yet figured out a perfect solution for this. We could have the core query touchscreen inputs whenever the game samples the touchscreen. The problem with this is a timing one: between each frame, melonDS may run way faster than realtime, so this would result in some amount of distortion to touchscreen coordinates over time. An alternative would be to sample touchscreen input during one frame and replay it with the correct timing during the next frame. It would mostly eliminate the distortion problem, but it would add a frame of input lag.

I don't know. It's worth noting that the current touchscreen handling also suffers from the aforementioned time distortion problem. I don't know how much of a problem it actually is, in practice. Touchscreen input just seems to work well enough as far as I know.


Either way, we will keep you informed about all this. Expect a merger of season3 very soon.]]> https://melonds.kuribo64.net/comments.php?id=207 https://melonds.kuribo64.net/comments.php?id=206 Wed, 24 Jul 2024 11:41:46 +0000 https://melonds.kuribo64.net/comments.php?id=206 https://melonds.kuribo64.net/comments.php?id=205 Thu, 18 Jul 2024 12:17:47 +0000
LAN should not be too difficult to backport, as it was largely functional.

Netplay will be another story. It was built around the old multi-process IPC system, and is generally obsolete, so a lot of reworking will be needed to get it functional again.

I'm hoping to be able to get all that to a functional state by November. We'll see how this goes.

There's also work needed to properly integrate this to the UI. I'm thinking of enforcing one melonDS instance only when using LAN or netplay, to keep things simpler. For netplay, it would pretty much be required anyway, as we will need to have instances set up in a specific way.

You can tell things aren't super fast on my side, I've pretty much taken a one-month break after getting doublemelon done. I need a big break from melonDS, which I will hopefully take after 1.0 is released.]]> https://melonds.kuribo64.net/comments.php?id=205 https://melonds.kuribo64.net/comments.php?id=203 Sat, 15 Jun 2024 14:57:12 +0000
This was the second big step of the refactor (after refactoring the core). It is now done, so we have a good base upon which to build the features we want for melonDS 1.0.

There is still a lot of work to do. Many areas of the codebase are less than ideal, there are some old bugs that linger and that we need to address, but we will get there in due time. melonDS 1.0 will be big, and we can't afford to let any stupid bugs slide.

But for now, a quick list of the changes the doublemelon merger is bringing in:

Some base work is laid for multi-window support. There will be more work to get it working correctly, but we do have a good base going now. This was the original intention behind the doublemelon branch, it just became a step in the refactor because it was pretty much required.

The configuration system has been remade to be much more flexible. There is code in place to translate legacy config files to the new format, so you won't lose your settings.

The frontend components have been properly encapsulated, which allows for proper multi-instance support. As of now, we still need to deal with a workaround in the JIT, so multi-instance will only work with the interpreter.


Thank you all for your support, and stay tuned for more!]]> https://melonds.kuribo64.net/comments.php?id=203 https://melonds.kuribo64.net/comments.php?id=202 Mon, 10 Jun 2024 22:23:03 +0000
These need to be adapted to the refactor, too.

Adapting the local multiplayer interface wasn't that hard. What I did was simply change the shared memory buffer to a regular memory buffer guarded by a mutex, and change the interprocess semaphores to regular semaphores. This makes the code simpler and easier to deal with.

From a performance perspective, I haven't observed a huge improvement from this change, atleast under Linux. However, the gains will be apparent when we will be reimplementing netplay: managing multiple emulator instances is going to be a whole lot easier and more reliable when they are all contained within the same process.

Then I moved on to the outer network interfaces. I had a bit of a debate as to whether to make them instance-specific. Because it could make sense to have them be shared across instances: these instances could be networked together rather than being isolated. It's a bit of a hypothetical scenario, but I like the possibility.

It requires, among other things, sharing around data frames. So this is what I'm working on: designing a packet queue system that can broadcast packets across all emulator instances in a reliable and efficient way.

Once I've tackled the network part of melonDS, the refactor will be mostly done. There will be a few bugs and misc things to iron out, and we will be good to finally merge it.

There will still be a lot of stuff to fix and rework, but we will atleast be in a somewhat workable state.]]> https://melonds.kuribo64.net/comments.php?id=202