< The netplay saga, ep 2The netplay saga, ep 3.5 >
The netplay saga, ep 3
Apr 7th 2023, by Arisotura
In the previous episode, I had basic input forwarding working, but we had problems due to the initial state being different on each side. So in this episode, I've been working on tackling this.


The first problem is the ROMs themselves. The first iteration of netplay required that each side have the same ROM, but this has the problem that there can be multiple revisions of the same game, and some games (hi Pokémon) even support multiplayer interaction with different games. Requiring every player to have the exact same ROM feels really restrictive, especially compared to a real-life DS multiplayer session, where each player has their own game cart (or doesn't, and uses download play).

Yet, we do need to ensure that every mirror client is using the exact same ROM as their mirror host. Two solutions: either having mirror hosts send their ROM to their mirror clients, or requiring all ROMs to already be present on all sides.

From an end user perspective, I don't like the second solution. It may require users to deal with complex multi-ROM setups, making sure they load everything in the right place; there's quite the potential for things to go wrong, or just for users to be confused.

So I went and experimented with the first solution. While it keeps things simple, it has the downside that transferring DS ROMs takes a while, due to their average size of 64MB. But there are ways to alleviate this: compressing the transferred data, but also skipping the transfer entirely if all sides already have the exact same ROM (which we can verify with a simple CRC).

Keep in mind that none of this is set in stone, and I'm largely experimenting here. We are still pretty far from a finished product.

Next step is ensuring that the emulator state on boot is the same on each side. For this, I had the idea of using the savestate system: basically, have the mirror host take a savestate after the ROM is loaded, send that state over to mirror clients, have them apply it, and it's guaranteed that all sides start with the exact same state.

I ran into a few issues with this. First, the savestate system doesn't save the BIOS and firmware, because it wasn't deemed necessary at the time I designed it. But right now, it's a requirement if we want our mirror clients to have the same user settings, MAC address, etc... as their mirror host. I also ran into a bug in the savestate system itself, which isn't a problem in most cases but turned out to be problematic in this current situation. After addressing all this, I was finally able to have all sides start from the exact same state. And it does fix the issues I had observed: games stay in perfect sync, items in Mario Kart will always pull the same item on each side, the AI players will stay in sync, etc...

This does have a bit of the same problem as sending ROMs around, though: melonDS savestates tend to be ~18MB in size. So, definitely, compression will come in handy here. I also want to look into other ways to optimize this: enet (the network library we use for this) isn't well suited for transferring large amounts of data like that, so it's slow.

But, overall, at this point we have something close to a viable netplay implementation. As I said, there's still a lot of work to turn this into a finished product, but so far it's looking pretty promising.

For one, the current savestate system uses files, which isn't ideal in this situation. We've had the idea of changing it to use memory buffers for a while, because the way it works (lots of small fread/fwrite calls) isn't ideal on some platforms (like the Switch). JesseTG is working on it, and this change will come in handy here, so I'm waiting for it. For the sake of testing, I circumvented this limitation in a pretty gross way, but... yeah.

Then there's a lot of UI work to be done. Integrating all this into the user interface properly, making things configurable instead of being hardcoded, making everything clear and intuitive, handling problematic situations gracefully... And, of course, I need to add support for headless melonDS instances, so other players can stay hidden from your view, replicating the true DS multiplayer experience.

There's also a bunch of performance testing and tuning to be done. During my testing, I observed some hiccups, but it's also worth noting that my computers are like a decade old, so they're not the best hardware around. I also haven't had the chance to test this over the internet, so I want to see how it performs in these situations.
22 comments have been posted.
< The netplay saga, ep 2The netplay saga, ep 3.5 >
ZeDestroyer says:
Apr 7th 2023
If I understand the savestate idea correctly, it would mean that by loading someone elses save, you would also have their game data. That would in Turn limit some of the Multiplayer Games out there like Dragon Quest 9, since everyone would preferably have their own character and class. But due to how RNG works in some Games, you‘d have to share the same game data, right?
Arisotura says:
Apr 7th 2023
Nah, it would work like multiplayer in a real world setting. On your computer, the melonDS instace you control will have your data, and the other instances will merely mirror the other players.
ZeDestroyer says:
Apr 7th 2023
I see, I was under the impression that the clients would load the full savestate data, including save data like progress and such. But it seems you only pick certain data that is necessary for synchronizaton. Very neet, Thank you for you work :)
Sarkes says:
Apr 7th 2023
When do you think you will finish this multiplayer? I'm not very good in these things but I appreciate your effort for us
poudink says:
Apr 7th 2023
I feel like there might be some bad legal implications to sending entire ROMs over the internet. Is this something you've considered?
Arisotura says:
Apr 7th 2023
I intend to look around at other emulators that support netplay and see how they deal with this. But, in practice, this isn't going to enable piracy, so I doubt there would be any serious legal issues.
Generic aka RSDuck says:
Apr 7th 2023
I'm in disagreement with Arisotura on this one from the start. Assuming you got your roms the legal way (they're private backups of your real DS cartridges), by my layman understanding, it would not be legal to send them over the network to anyone you're playing with.
Shadowwolf1337 says:
Apr 8th 2023
what makes this whole thing even more fun is that games like mario party ds do not save any progress, meaning if the game disconnected for even a moment, boom, all progress gone
Sarkes says:
Apr 8th 2023
Will the online be completed in this year for your thoughts Arisotura? Thx for reading
Sarkes says:
Apr 8th 2023
Will the online be completed in this year for your thoughts Arisotura? Thx for reading
Vortex says:
Apr 8th 2023
Sarkes, the online be ready when it's ready.

Arisotura, I noticed your mention of decade old hardware. Later on in progression if you need various specification testers, perhaps the community can provide a helping hand. We could list our system specs on the forums or where ever as needed, and you can gauge a variety of specifications accordingly based upon our feedback.
ProfessionalGopnik says:
Apr 8th 2023
I have also tested some stuff with a friend who's in america while i'm in europe. Are the hardcoded 4 frames of delay enough for this distance? Because we both got netplay running fine locally, but not together. Never managed to sync up properly
Anonymous says:
Apr 9th 2023
Everything is sounding extremely exciting! It does seem like the whole sending full-on ROMs and save-states may be a bit of a hiccup; I don't think it'll be possible to easily get around the whole save-state situation, but I would assume anyone dealing with ROMs would be easily able to share their exact copy via other methods (file-sharing, e-mail, Discord, etc.) with their gaming partner and thus be able to cut down on potential legality issues vs the emulator itself facilitating this.
^.^ says:
Apr 9th 2023
Ok so we are going for a "send everything" approach, good, I like it, as long as things are separated between different instances for each user. (And no personal data is disclosed)
I understand how sending Roms may be crossing the line from gray area to black one, so maybe it could be an opt-in function, with a warning like:

CRC MISMATCH, the Rom files may be incompatible, do you want to send your files to the other clients? This may take a while.
Warning:MelonDs is not a rom sharing app and does not endorse piracy, only play games you own a copy of, etc...
Also make sure you trust the other clients to not send you malware.
[Continue anyway] [send rom to all clients] [cancel]

Aside from this, I'm glad to know this is already taking a shape, resembling something usable :D
You know, NoCash also had pretty old computers, and that made his emulator the most efficient ;)

If we are going for user friendliness, it would be nice to have some lobby system like Citra3ds local multiplayer system does.

Thanks for the update and happy Easter!
^.^ says:
Apr 9th 2023
Ignore the bad grammar at the end...
I think this the first emulator that sends all the game executables via the internet, since download speeds are much higher now.
One thing you could do to avoid legal issues is on the fly encryption/decryption of the roms for the mirror instances, in a way that makes this sending system unusable for piracy unless you willingly decrypt them, but that might generate some overhead...
Pages:12
Post a comment
Name:
DO NOT TOUCH