melonDS RSS https://melonds.kuribo64.net The latest news on melonDS. News of the refactor: doublemelon is merged -- by Arisotura https://melonds.kuribo64.net/comments.php?id=203 Sat, 15 Jun 2024 14:57:12 +0000
This was the second big step of the refactor (after refactoring the core). It is now done, so we have a good base upon which to build the features we want for melonDS 1.0.

There is still a lot of work to do. Many areas of the codebase are less than ideal, there are some old bugs that linger and that we need to address, but we will get there in due time. melonDS 1.0 will be big, and we can't afford to let any stupid bugs slide.

But for now, a quick list of the changes the doublemelon merger is bringing in:

Some base work is laid for multi-window support. There will be more work to get it working correctly, but we do have a good base going now. This was the original intention behind the doublemelon branch, it just became a step in the refactor because it was pretty much required.

The configuration system has been remade to be much more flexible. There is code in place to translate legacy config files to the new format, so you won't lose your settings.

The frontend components have been properly encapsulated, which allows for proper multi-instance support. As of now, we still need to deal with a workaround in the JIT, so multi-instance will only work with the interpreter.


Thank you all for your support, and stay tuned for more!]]>
https://melonds.kuribo64.net/comments.php?id=203
News of the refactor: almost there! -- by Arisotura https://melonds.kuribo64.net/comments.php?id=202 Mon, 10 Jun 2024 22:23:03 +0000
These need to be adapted to the refactor, too.

Adapting the local multiplayer interface wasn't that hard. What I did was simply change the shared memory buffer to a regular memory buffer guarded by a mutex, and change the interprocess semaphores to regular semaphores. This makes the code simpler and easier to deal with.

From a performance perspective, I haven't observed a huge improvement from this change, atleast under Linux. However, the gains will be apparent when we will be reimplementing netplay: managing multiple emulator instances is going to be a whole lot easier and more reliable when they are all contained within the same process.

Then I moved on to the outer network interfaces. I had a bit of a debate as to whether to make them instance-specific. Because it could make sense to have them be shared across instances: these instances could be networked together rather than being isolated. It's a bit of a hypothetical scenario, but I like the possibility.

It requires, among other things, sharing around data frames. So this is what I'm working on: designing a packet queue system that can broadcast packets across all emulator instances in a reliable and efficient way.

Once I've tackled the network part of melonDS, the refactor will be mostly done. There will be a few bugs and misc things to iron out, and we will be good to finally merge it.

There will still be a lot of stuff to fix and rework, but we will atleast be in a somewhat workable state.]]>
https://melonds.kuribo64.net/comments.php?id=202
Refactor still underway... -- by Arisotura https://melonds.kuribo64.net/comments.php?id=201 Mon, 27 May 2024 21:40:39 +0000
I finished porting everything to use the new configuration system, and thus the old system (really just a pile of global variables) is no more.

Most of the frontend is also properly encapsulated. What remains at this point is the core-frontend glue (Platform.cpp) and the networking code.

There will still be a bunch of cleanup and fixes to do, but we're getting there! The first goal is to get everything encapsulated so it becomes possible to run multiple DS instances within the same melonDS process. The code may not be the best or the prettiest, but we can always go back and fix up things later, when the bulk of the work is done.

Quite the audacious refactor, given the sheer scale of it, but it's definitely worth it. Even if it can feel paralyzing at times.

Stay tuned!]]>
https://melonds.kuribo64.net/comments.php?id=201
Refactor underway -- by Arisotura https://melonds.kuribo64.net/comments.php?id=200 Wed, 15 May 2024 17:06:52 +0000
This means that while the refactor is in progress, CI builds will be more or less rather broken and unstable, so don't rely on them.

We also plan on putting up 'beta' builds with certain features of interest, namely the compute shader renderer that has been merged recently, or season2 (the experimental LAN/netplay branch).


Also, side note: the comment section is not for posting your brainfarts. Pointless comments will be deleted. Repeated pointless comments will get you banned.

Thank you.]]>
https://melonds.kuribo64.net/comments.php?id=200
A couple notes about the Delta release -- by Arisotura https://melonds.kuribo64.net/comments.php?id=199 Wed, 24 Apr 2024 21:06:08 +0000

First, we the melonDS team are not responsible for third-party ports (with the exception of Generic supporting the Switch port, since he made it). Our scope is limited to the melonDS core and the desktop frontend (ie. the Windows, Linux and macOS versions).

If you are running into issues with Delta, or with any port of melonDS: you should contact the authors of that port. Members of our community may also be able to help you, but contacting the port authors directly may be quicker.

It is also worth noting that port authors are responsible for keeping their port up-to-date.

At the time of writing this, Delta is using melonDS 0.9.1 as its core, which is quite outdated by now. If you're running into emulation issues with Delta, chances are that the bugs have long been fixed, but you will need to bug the Delta authors to update their port. We have tried to contact them about this, but haven't received a response yet.


tl;dr: if you are running into any issue with Delta, or any melonDS port, please ask the authors of that port first.


Second point, emulation frontends like Delta, RetroArch or BizHawk (to name a few) are definitely impressive pieces of work, but it is worth keeping in mind that they wouldn't be possible without the work from the emulator authors who provide the cores these frontends use. So if you're considering donating towards these frontend projects, try to keep the underlying emulator core projects in mind too.

That being said, we have already seen an influx of new patrons, so thank you folks!


Third point, not related to Delta per se, but this is prompted by the situation: I'm planning a bunch of updates to the site.

Among the first updates, I added some extra measures against spam on the blog. Nothing very fancy.

Next update to come will be attachment support for the board. It has come to my attention that some of the old threads link images and assets from Discord, and Discord has been cracking down against hotlinking lately. So we will provide some space for people to attach images and stuff to their posts, that way it will stay on our server and not disappear on the whims of some external entity.

We're also open to your feedback regarding the site, the content, melonDS itself, ...

I do kinda want to post more content on my Patreon page, but I don't really know what to post there that wouldn't just mirror this blog. Regardless, in a move of transparency, I've made the earnings public.]]>
https://melonds.kuribo64.net/comments.php?id=199
News of the great refactoring -- by Arisotura https://melonds.kuribo64.net/comments.php?id=198 Fri, 19 Apr 2024 09:09:07 +0000 this post I showed an attempt at running two melonDS windows at the same time. Just a silly test, but groundwork for more.

What became apparent during this is the fact that our current configuration system won't do.

The current config system was born out of simple needs: at the time we just needed to store a few configuration values. The keyboard and joystick input mappings, and misc settings like direct boot or 3D renderer threading. For this, a simple pseudo-INI format was deemed enough. The config file is named melonDS.ini, but it is just a simple key-value pair format with a custom parser, not an actual INI format.

The configuration data in melonDS is just a bunch of global variables in the Config namespace exposed to the entire program. The parser simply goes through all the lines in melonDS.ini, checks the key against a list of possible keys, and when a match is found, the corresponding variable is filled with the value that was read. The same list also serves to initialize the config data with default values (for example, if melonDS.ini wasn't created yet). All fine and dandy.

This system, however, doesn't shine in its flexibility. This started becoming apparent in 0.9.5 when the new local multiplayer system was added. I realized we needed a way to allow each melonDS instance to have different settings for things like input mappings. Prior, the only way to do that was to run the different instances from different folders.

The solution I had come up with, was to create other melonDS.ini files: melonDS.2.ini for instance 2, melonDS.3.ini for instance 3, and so on. melonDS.ini would contain the bulk of the settings: the settings that are common across all instances, and the settings for instance 0. Then, when opening a new instance, it will get the global settings from the main melonDS.ini, and its instance-specific settings from its specific config file. If said specific file doesn't exist, it is initialized from the main melonDS.ini.

It's nice and colorful, but a bit unwieldy...

When I started working on that proof-of-concept multiple window thing, I had a realization in the same vein. Basically, the config file also serves to store settings related to the window: size, layout settings, etc. For a multi-window feature to be useful, we want each window to have its own settings (say, having one window display the top screen, and another the bottom screen). Clearly, the current config system won't do here.

So what can we do? Find other creative ways to get around the lack of flexibility of the pseudo-INI system?

Nah.

Instead, we are going for a radical approach: replacing the pseudo-INI system entirely. We decided to go with TOML for a flexible config system. With a TOML file, we can store arbitrary amounts of entries, sub-tables, arrays, and so on.

For example:
RecentROM_0=/home/arisotura/ndsrom/sm64ds.nds
RecentROM_1=/home/arisotura/ndsrom/mkds.nds
RecentROM_2=/home/arisotura/ndsrom/sonic.nds
RecentROM_3=/home/arisotura/ndsrom/nsmb.nds
RecentROM_4=/home/arisotura/ndsrom/2d_latch.nds
RecentROM_5=/home/arisotura/ndsrom/a_blarg.nds
RecentROM_6=/home/arisotura/ndsrom/libfatdir.nds
RecentROM_7=/home/arisotura/ndsrom/pokeplat.nds
RecentROM_8=/home/arisotura/ndsrom/party.nds
RecentROM_9=/home/arisotura/ndsrom/httpget.nds
The recent ROM list in the current pseudo-INI format is basically 10 separate variables. If you want more than 10 entries in the recent files menu? You would have to add more variables.

In TOML, it can be stored like this:
RecentROM = [
"/home/arisotura/ndsrom/sm64ds.nds",
"/home/arisotura/ndsrom/mkds.nds",
"/home/arisotura/ndsrom/sonic.nds",
"/home/arisotura/ndsrom/nsmb.nds",
"/home/arisotura/ndsrom/2d_latch.nds",
"/home/arisotura/ndsrom/a_blarg.nds",
"/home/arisotura/ndsrom/libfatdir.nds",
"/home/arisotura/ndsrom/pokeplat.nds",
"/home/arisotura/ndsrom/party.nds",
"/home/arisotura/ndsrom/httpget.nds",
]
Then the RecentROM item is accessed as an array. The only limit on its length is decided by the UI code.

In a similar vein, multiple instances and windows can simply be handled by adding more data tables for them in the config file. It is possible to store them as an array of tables, but I chose to explicitly name them 'Instance0', 'Instance1', 'Instance2', and so on, to make the file more readable.

Obviously, this requires reworking the way melonDS accesses the settings. Just exposing a bunch of global variables won't do here.

So far, I added a simple wrapper class for TOML tables. Besides keeping the TOML library code in one place, it also allows me to handle default values and allowable ranges for integer values transparently. Then any given emulator instance can keep track of its own config table, as well as the global table. The current format isn't set in stone, but it gives me an idea how things should be laid out.

Of course, there is also code in place to convert pseudo-INI files to the new format if needed, so you won't lose your settings.

I am now in the process of adapting the frontend code to use the new config system. Easier said than done, because this is intertwined with the rest of the refactoring: basically encapsulating everything so that multiple melonDS instances can be run within the same process, without them sharing global state.

So, yeah. Slowly but steadily, we're getting there.]]>
https://melonds.kuribo64.net/comments.php?id=198
Wifi fixes: channels and power -- by Arisotura https://melonds.kuribo64.net/comments.php?id=197 Wed, 10 Apr 2024 19:56:03 +0000
Anyway, I recently felt like getting back into melonDS dev. I figured that trying to fix some wifi bugs could help me build up momentum and motivation I'll need to finish the big refactor.

The point of the big refactor is to lay groundwork for turning netplay from a clunky proof of concept into an actual finished product. But netplay or LAN won't be very useful if the underlying wifi implementation is malfunctioning...


The first bug I looked at was that weird bug in Meteos: when starting a download play game, if you're using a DSPhat firmware, you will instantly get a connection error, but everything will work fine if you're using a DSLite firmware.

So I researched the bug, but the outcome was a bit disappointing. Long story short, it's a timing issue. The ARM9 starts wifi initialization and allows it about 44 milliseconds before deciding it has failed. The wifi initialization doesn't actually take this long, but on a DSPhat (or any console equipped with a type 2 RF module) the RF initialization includes an extra delay of 40 milliseconds. Changing the ARM9 cache timings fixes the bug.

Cache emulation is something we're looking at for the next release, as it may fix quite a bunch of these timing issues -- there have been preliminary tests and it's looking promising. But in the meantime, that particular bug goes beyond my scope.


No biggie, we have a plethora of other problematic games. In particular, I had gen4 Pokémon games in mind. In these games, players can meet into the Union room where they can decide to chat, battle, etc... but the Union room was said to be malfunctioning in melonDS if more than two players were in it.

I first ran into some 'easy' bugs. For example, melonDS would not generate different MAC addresses for extra instances when using an external firmware. I also had to obtain different savefiles -- the game will get confused if multiple players using the same save file try to connect.

Past that, I figured that the issue might be caused by the lack of wifi channel support in melonDS. You probably know about wifi channels: the 802.11 standard specifies a set of 14 channels, which correspond to specific frequencies.

The tricky part is that on the DS, you don't have an explicit, clear register to tell it to "tune in to channel 6". Tuning to a channel is done by changing a bunch of RF and BB registers with values that are stored in the console firmware. Some of these values seem consistent across units, some of them are different. The way they are structured also depends on the RF type used in the console.

So the way melonDS dealt with this was to... just ignore channels entirely. We're just receiving everything at all times. Which I figured might cause problems...

I decided to base myself on RF registers to detect the current channel, as it seemed to be the most reliable way. I was thinking of how to implement it all. I eventually thought, we already read the firmware to determine which RF type to emulate, might as well pull the required register indices and values for each channel from there. Then, whenever the corresponding RF registers are modified, we check them against the values from the firmware to determine which channel we are tuning in to.

Next it's a simple matter of adding the current channel number to the frames that we send out. When receiving frames, we check their channel number against our current channel, and discard them if they're on the wrong channel. Simple.


In the end, I don't think the lack of channel support was the problem, but implementing them gave me useful insight.

To make sure I wasn't breaking anything basic, I tested the WFC config utility in Mario Kart, and started an AP scan. It did not find melonAP. Oops.

The AP scan basically cycles through all 14 channels, sending a bunch of probe requests, listening for responses, then moving to the next channel, and so on. When I looked at my logs, I saw that it was only actually sending frames on channel 1, and melonAP is set to channel 6, so you guess how that goes.

Upon closer inspection, it was failing to send frames on channels 2 and up because it was stuck in power-down mode. When changing the channel, there is a sequence to power down the transceiver, change the channel, then power it back up.

In this case, melonDS wasn't recognizing the instruction to turn the power back on. The power system of the wifi hardware is kind of complex and there is a bunch of different registers affecting it, and the support in melonDS was incomplete.

The issues other games had when trying to connect more than 2 players came from the same problem. Transceiver staying powered down for too long, causing starvation -- when trying to listen to beacons, the game would just see nothing for extended periods of time, then be receiving a ton of frames at once, which doesn't bode well. In some cases this would happen while other instances were already engaging in a MP data exchange, and the game would be completely overwhelmed from receiving an absolute ton of MP frames at once.

You guess how that goes.


The wifi power system is a bit complex, I've done a bunch of research but there's a lot to be said and there are still some unknowns. But I have a better view of how it all works and how the various registers interact.

I am still reworking the implementation in melonDS. It's unfinished, but the results so far are promising.



Mario & Sonic at the Olympic Games, one of the problematic games. Now connecting 4 players, smooth as butter.


I may write another post about the details of the wifi power system when I'm done with the implementation. Stay tuned!]]>
https://melonds.kuribo64.net/comments.php?id=197
Quick news -- by Arisotura https://melonds.kuribo64.net/comments.php?id=196 Mon, 19 Feb 2024 23:39:17 +0000

This is just a quick attempt at supporting multiple windows. There's still a whole pile of issues to fix, but it's looking promising so far.

This is not only part of the ongoing refactor, but would also pave the way for separate windows, a feature that has long been requested.

I'm having a bit of a question there: if we got multiple windows, which one should have the OSD? The main window? The currently active one? All of them?


Other than that, sorry for the lack of updates lately. January has been rough for me, but things are getting better. I've also been caught in a fun side project that involves reverse-engineering (and emulating, heh) ancient FPGAs. Maybe I could have a sort of personal blog for non-melonDS projects of that kind, or just personal stuff in general.]]>
https://melonds.kuribo64.net/comments.php?id=196
Merry belated Christmas -- by Arisotura https://melonds.kuribo64.net/comments.php?id=195 Thu, 28 Dec 2023 14:51:23 +0000

As I have received and set up my new laptop, I've been able to work on melonDS some. The goal is still the same: to adapt melonDS for supporting multiple instances within one process.

The frontend is proving to be tricky, because it was largely built without many regards for code quality. Originally, the frontend was just quickly built around the program entry point in main.cpp, as it just needed to provide a means to use the emulator. We had a simple window, a dedicated thread to run the actual emulation, and that was it.

Obviously, over time we added various features to the emulator, rebuilt the frontend around different UI toolkits, piled on more features, and so on. Basically, the frontend code became a mess, and everything was more or less just dumped in the global namespace -- there are some modules which live in their own namespaces, but it's all not very modular, and doesn't lend itself to running multiple instances of melonDS.

I started by splitting main.cpp into separate files. We are far from done with the cleanup, so there's a lot of cruft around, but the basic idea is the following:

* main.cpp just holds the entry point (main()) and global application-related stuff.

* Window.cpp holds the MainWindow code, for managing the emulator window. We may also add support for multiple windows.

* Screen.cpp holds the ScreenPanel code, that is, the panel widget that goes inside the emulator window and handles display and touchscreen input. There are two kinds of ScreenPanel: one that relies on Qt's graphics API, and one that uses OpenGL.

* EmuThread.cpp holds the emulation thread.

From there, each EmuThread would hold its own instance of the emulator core (which we mostly finished porting), and optionally point to one or two windows to display to (or none, in the case of a headless instance).

After laying out this basic separation, I figured it would make sense to integrate the OSD feature into ScreenPanel. The OSD module was a bit of a mess, with having to maintain different code paths for Qt or OpenGL drawing. Now things are a lot cleaner.

There's still quite a bunch of global state and stuff that we need to adapt before the dream of emulating multiple DS's within the same process can come true. But we're getting there, slowly but steadily. This is the perfect occasion to finally clean up and improve our frontend code.]]>
https://melonds.kuribo64.net/comments.php?id=195
Long-due status update -- by Arisotura https://melonds.kuribo64.net/comments.php?id=194 Sun, 17 Dec 2023 13:33:22 +0000
What's new since the last post? I had started working on refactoring the melonDS codebase for the ambitious changes to come. JesseTG mostly finished the work on the core, so that's one big thing out of the way.

The frontend code is now going to need adequate refactoring too. That's, well, more work. We'll get there over the coming days.

On my side, I (finally) ordered a new laptop, so I'm waiting to receive it and set it up before I do more big work on melonDS.

I will need a while to adjust to the new changes that have been done to the melonDS core. It's also likely not perfect, we will likely have to do more work to clean up and harmonize the codebase. But atleast, the biggest part of the work is done.

On the frontend side, we need to add support for multiple different emulator windows. Historically, the frontend was structured around one emu thread and one emu window. The latter provides a means to display the emulator's video output and lets the user interact with the emulator, the former feeds the inputs into the melonDS core, runs it and manages it.

It wouldn't be difficult to spawn multiple emu threads, one per emulated DS. Window management is another deal. We thought we might want to support attaching multiple windows to one emu thread, for example to support displaying each DS screen on a separate window. For netplay purposes, we might also want to support headless instances, that is, running an emu thread with no window attached, no sound output, and inputs received from the network.

This also poses the question of OpenGL context management, all things we will need to look into.

We'll get there. It just... takes time.]]>
https://melonds.kuribo64.net/comments.php?id=194