melonDS RSS The latest news on melonDS. By popular request -- by Arisotura Sat, 15 Feb 2020 12:13:14 +0000

You guess.

I'm not forgetting about the OpenGL work that was started, either, but I really needed to work on something new to get some motivation back and not let melonDS die.

Also, our comrades RSDuck/Generic and Chagall have been working on some pretty rad shit too. We're coordinating to unite all that nicely for an epic release eventually.]]>
New melonDS feature: GBA connectivity -- by Chagall Tue, 28 Jan 2020 23:12:11 +0000
This feature is mostly used in DS games to grant bonuses to players who own specific GBA titles. For example, inserting a GBA Ace Attorney game and launching the DS version will unlock all cases in the remake. Some games go further and read/write save data from/to the GBA cartridge (as in Pokémon games), or in one case polls its I/O port (Lunar Knights/Boktai DS).

All these features are now integrated in the master branch of the melonDS git repository. As far as I'm aware, this makes melonDS the first emulator to support solar sensor emulation in Lunar Knights. The video below demonstrates using the Solar Sensor item in Boktai DS with Bokura no Taiyou in the GBA slot.

Of course, you can also send your pocket monsters from GBA to DS, as demonstrated in this video:

As on real hardware, the GBA save file is modified during migration as the Pokémon are moved to the DS game:

Thanks to endrift of mGBA for the solar sensor emulation code that I used as a primary reference, and to DeSmuME authors for the GBA cartridge Flash save read/write logic that helped a lot with supporting the Pokémon games. Martin Korth's GBATEK document was also a great general reference.

For those curious to try this new feature as soon as possible, another bit of news is the addition of continuous integration to the melonDS git repository, which generates binaries for Windows and Linux from the master branch. If you have a GitHub account, head over to the Workflows page, select the newest build for your platform, and download the artifact shown on the page. We don't have a way to provide the latest build to users who are not logged into GitHub yet, but the latest build for Windows at this time is available here.

To load a GBA title, either drag and drop a .gba file over melonDS, use the "Open ROM..." menu and select the "GBA ROM" file type, or, if you happen to use the command-line interface, add the path to the GBA ROM file as an extra argument following that of the DS one. Note that if the DS is on while a GBA cartridge is inserted, it will usually not be recognized in-game (as on real hardware). Either load the GBA cartridge before booting the system, or reset afterwards.

Solar sensor level is bound by default to the left (4) and right (6) keypad arrow keys, but can be rebound in the settings. Note that it is only active while a valid Boktai cartridge is inserted.

I hope everyone enjoys this new feature. melonDS is moving forwards at a good pace, so it's a great time to get involved if you are interested in improving DS emulation.]]>
Sorry for the silence lately -- by Arisotura Fri, 17 Jan 2020 14:53:18 +0000
Well, if you know me, you know that:

* I tend to only be able to focus on one big thing at once
* I am a trans girl

About the second point, I'm wanting to 'finish' my transition. There isn't a specific point where a gender transition can be considered finished; when your transition is finished depends on your criteria.

For me, that means I'm currently putting together the paperwork to get my name changed. Not the most difficult part, but anything involving paperwork tends to drain my energy, so you guess how that goes.

And it's not over, next comes getting my gender marker changed, and that one is more difficult (surprise).

Finishing my transition also means a few other things, like finally working on my voice or taking some time to figure out my style and build a better wardrobe.

There's also the whole bit where my job ends in about one month, and for now I don't quite know what I'll do next.

Sorry for the lack of activity wrt melonDS, but hopefully that will pick up again soon.

I began adapting the 4xBRZ filter shader to the OpenGL renderer a while ago, which is giving decent results, but I need to finish it (namely adding the 3D layer to it). I'm experimenting with ways to filter the 2D elements when the 3D layer is being upscaled.]]>
melonDS wishes you a merry Christmas! -- by Arisotura Sun, 22 Dec 2019 15:42:59 +0000
We're also past melonDS's third birthday. Time flies, it's crazy.

So what's new on that front? Well, not a lot, alas. I want to improve the OpenGL renderer, but for now, I started working on things, hit a roadblock as to how to do certain things, and, surprise, lost motivation. So until I get that back, there won't be a whole lot happening.

Also, Christmas might be more or less of a shitshow, as I'm trying to make it to my hometown to see my family whom I haven't seen in ages. I'm eagerly waiting for that, but it's not going to be that simple. Remember the general strike from Dec 5? Lo and behold, it's still going, and the train I had booked was cancelled due to it. I booked a bus instead, which is going to be a lot of fun (11-hour trip during the night, in conditions where I can hardly sleep). Besides, the crazy Macron government is indirectly pushing these bus drivers to drive as long as possible without rest (mostly to make up for the strike), by passing a bill to remove the requirement for breaks every 2h, thus completely disregarding security for the sake of profit. So, if I happen to die in a bus crash, you know who to blame.

Anyway, back to OpenGL. So, what I wanted to do:

1. Taking care of dual-screen 3D, and more generally any case where display capture is used. Currently, 3D frames that go through display capture end up downscaled back to 256x192, obviously because they need to fit in VRAM, and anything that uses them expects them to be that resolution.

There are ways to address that for the common use cases, though. As long as a captured frame isn't accessed by the CPU, it is possible to bypass VRAM entirely and instead use something like an OpenGL texture, without causing any visible problems. Sure, it's not too accurate to how the DS works, but neither is the general idea of upscaling. I am fine with this as long as we keep the accurate pathway (basically, the software renderer).

Anyway, the issue there is that I have to envision all the possible use cases. And for example, one possibility is to capture to a VRAM bank, with capture blending set to blend between the frame being rendered and a previous frame sitting in a VRAM bank (which may be the same bank we are capturing to).

I could always start with a simple use case (like dual-screen 3D). The issue is that I always feel the need to envision the whole scope of the project, all the use cases etc, because I fear that if I start building something for one use case, it may later prove unsuitable for other uses cases, and cause me to have to start again from scratch.

So, while simple display capture can be implemented as a simple colorbuffer copy, any case that relies on blending will need more work. I will need to think of a workaround for the case where we read and write to the same VRAM bank, because doing so is a no-no in OpenGL.

Oh well.

2. Add filtering to 2D elements. This is less simple than it sounds, due to how layer compositing is handled in OpenGL mode. I thought about adding something simple first, basically, linear filtering. But a clear issue is how to handle borders between 2D and 3D. Keeping them pixelated is the easiest way but also the least nice one. The more obvious choice would be to smoothly fade from one to the other, but due to them being distinct layers with specific blending rules, it's easier said than done. I'll have to experiment there, but filtering on 2D elements is something I definitely want to pull off. I plan on adding other filters, like xBRZ. Those would do wonders on a game like NSMB, for example.

Oh well, we'll find ways.

Other improvements would be adding texture filtering and fixing some rendering bugs, nothing big.

Also, we need to push the JIT some more. The JIT beta seems to have gotten much less traction than I anticipated.]]>
JIT beta builds and introducing myself -- by Generic aka RSDuck Sat, 07 Dec 2019 00:20:22 +0000
As Arisotura has stated in her last post, there's now another head working on melonDS. I'm Generic on IRC and RSDuck on Github.

For some reason I started messing with DeSmuME a bit more than a year ago, trying to write a JIT recompiler targetting ARM64, to improve performance on the Switch. After many struggles something working came to be, but it missed it's original goal of being fast for several reasons (see the link if you're interested).

After recovering from the disappointment I put my hopes into melonDS, but since it lacked any JIT compiler (besides the abandoned attempt by Arisotura herself) I wanted to start with a x64 recompiler. Skip forward a bit and the JIT started take on form. After taking everything apart and putting things more cleanly back together around the beginning of last summer things only got better, with most instructions being recompiled and the opening of the pull request on Github, which later got merged.

At this time the ARM64 backend was started this time avoiding my previous mistakes. With autumn it reached a similar level of completion as the x64 backend while both backends received some optimisations. Until this point we were catching up to DeSmuME but at this point started to surpass it. Here is where I want to thank all developers of open source emlulators whose dynarec I was eable to look into and especially the developers of the JIT of Dolphin. We're not only using their code emitter but they also gave me some great advice on IRC!

With some further optimisations of the 2D GPU emulation using ARM NEON instructions, most 2D games already run at fullspeed on the Switch, while most 3D games run at fullspeed with overclocking. The ARM64 JIT and the GPU optimisations are currently located in my own fork and are distributed by me in binary form on Gbatemp in an admittedly chaotic way.

I am very grateful to Arisotura for creating the emulator in the first place, as well as trusting in me and my work. For the future the already mentioned GPU optimisations are my current focus. I'm currently rewriting the old ARM NEON optimisations to be more clean and adding some additional ones which hopefully increase the performance a bit more. The logical step following would be a NEON 3D GPU backend. I have some more plans lying around, e.g. there is still one major JIT optimisation we're missing out on (fastmem).

So but now enough storytelling, here are the builds:


These builds are my JIT branch (generic_jit) merged with master into a separate branch.

The JIT recompiler has to be enabled in Settings -> Emulation. The default maximum block size is 10, this is a very conservative number, as I don't know of any game which breaks with high maximum block size. If you experience any bugs which can't be reproduced with JIT disabled, please try lowering the maximum block size and disabling one or both of the options Literal Optimisations and Branch Optimisations and let me know. Though there already has been some testing through people building from source themselves and the semiofficial Switch builds, both uncovered some bugs.

melonDS 0.8.3 JIT beta for x64 Windows
melonDS 0.8.3 JIT beta for x64 Linux
Status update II -- by Arisotura Thu, 05 Dec 2019 22:34:34 +0000

First of all, things have been a bit shitshow-y, and involved a few bursts of depression, but, finally, it's there, I have my apartment now. I'm not quite finished settling here, but that's a big relief for now. Plus, the place is fairly nice! Except for the wall sockets being upside-down, but, eh.

Next, I'm being sucky at delivering a JIT beta. Thing is, unlike the DSi branch which was hacked together by me, the JIT branch is Generic's work, so I will need to take a while and get familiar with it.

In the meantime, I figured I would let Generic handle things himself. I gave him access to this blog and all, so he will be able to post a beta build, which he said he would do tomorrow.

I figure the best way to work with this long-term is to work as a duo. I'm less skilled on things like fast CPU emulation, and more skilled on other parts of emulation, so other people like Generic can complement me nicely there and together we can deliver an excellent product.

And, holy fucking shit, reconciling the JIT and DSi branches is something I'm totally looking forward to. At this point, the DSi branch already has several conflicts with the master branch (it's based off 0.8.1, so yeah), and, no idea about the JIT branch.

Immediately tho, I'm going to improve the OpenGL renderer. This means reorganizing some of the code so it's easier to work with, fixing some longstanding issues like the case of dual-screen 3D, and adding some filtering.

You can follow the progress in the appropriately named blackmagic_II branch.

Hell, even writing this post is taking me forever.

Also, today's Dec 5, and it's the general strike day. Seeing how successful it was, if they manage to keep going at this rate (and they totally intend to keep going), president Macron is going to have a hard time. But, this also means that certain things may be delayed, or whatever, so we'll see.]]>
Status update -- by Arisotura Sun, 17 Nov 2019 10:28:33 +0000

Couple things though.

On melonDS, things are going to be mostly on hold until I manage to get a proper apartment and am all settled in and all. But, two things. First, Generic said that the JIT branch is ready for a beta release, so I'll be doing that. Second, as a next direction to take the project, I'm thinking about adding support for AR cheat codes.

I'll also be trying to fix the remaining issues with the OpenGL renderer, but some are going to be tricky. Also, things related to this renderer: I need to come up with a not-too-hacky solution for dual-screen 3D and upscaling, and to add support for filtering in OpenGL mode (including xBRZ or something of that ilk).

But, blegh. The apartment. I visited one a while ago, now I'm going to have to put together the paperwork and hope it isn't too late. There are other apartments I can try, but, hey, I liked that one. So, here's hoping.

Oh also, this is Nov 17, birthday of the yellow vest movement. French protest movement that started one year ago, was originally against a gas tax raise but turned radical and against precarity in general, which seems to be a common theme these days (thinking about Hong Kong, Chile, ...). Despite the fact that the movement as it is will not bring revolution, it seems to want to never end... (what kind of protest movement lasts one whole year?)

But we can see that it's having an effect on protests in general, they turn more radical now. Like recently, a student from Lyon set himself on fire out of despair after being thrown into precarity. Had that happened two years ago for example, nobody would have cared, it'd just have been "another depressive sicko attempts suicide, film at eleven". But instead, this sparked several student-led protests and blockades, including one that broke into the Ministère de l'Enseignement Supérieur.

Dec 5 is going to be the general strike, too. I first thought that would be lame, as most of these attempts were a while back, but, now, within the different context, this looks like it's going to be quite something. May also cause some trouble on my end, but, we'll see.

Anyway, happy birthday to the yellow vests. I've been following the movement closely, and even taking part in it, as it gave me hope when I was at the worst of depression, so this means a lot to me.]]>
When things turn out unexpectedly simple -- by Arisotura Sun, 20 Oct 2019 10:02:42 +0000
Shows that taking a break does wonders.

Just yesterday, I idly looked at that bug again. Reproduced the freeze, then pressed F11 (debug key printing some status to the console) a few times to see if I could see a pattern. The ARM9 seemed to exhibit typical 'idle thread' behavior; the ARM7 however, seemed to get stuck in a loop. A quick glance at my IDB of the firmware code confirmed this:

AES wait routine

If you aren't fluent in ARM assembly, this function waits for the AES engine to finish whatever crypto operation it is doing, by checking the busy flag in AES_CNT.

But, in our case, the ARM7 got stuck inside this because, somehow, said busy flag remained set forever.

Some logging of AES operations shown that, while loading the game, the firmware would start a decrypt operation with a block count of zero. melonDSi didn't check for that case, and instead waited for data input that would never come (heh), causing the busy flag to remain set forever. You guess how this goes.

After quickly fixing this bug, I tried loading a cart game again. It still froze on a white screen, but the logs showed it was getting further. In particular:

UNIMPLEMENTED ARM9 NDMA3 START MODE 04, 04100010->021153A0 LEN=1 BLK=1 CNT=A4044000

That meant the game was actually booting, and attempting to load its data, using NDMA ('new DMA' implemented in the DSi) for whatever reason. NDMA start mode 04 is for cart slot DMA, which is triggered every time a data word is transferred from the cart.

This was quickly taken care of, too. A bunch of the NDMA start modes mirror those of 'old' DS DMA, so I just hooked those into the functions responsible for taking care of DS DMA. The only one that needs more extensive checking is GXFIFO mode, since it has a maximum block length of 112 words, I'm not sure how that interacts with the block subdivision system of NDMA.

Anyway, with this, I was finally able to boot DSi-enabled games.

Nothing terribly exciting to show tho. After all, most of the DSi-enhanced games out there don't have much extra features other than making use of the DSi cameras, so we have to do something to emulate them.

Loading DSi ROMs is also a bit finicky as they have to be one specific way for it to work (right now, with the DSi-specific data encrypted). And, some of the ROM dumps out there lack the DSi-specific data, so it would be good to add some verification and warn the user if their ROM is a bad dump.

Speaking of which, I am thinking of the ways I can make this evolve, now that it's atleast working to some extent. Namely:

* lowering the entry barrier, that is, upgrading dsidumper to dump all the required material in one go

* getting rid of some of the required files. boot2_7/9.bin can be sourced from the NAND directly, provided I can find a way to derive the required decryption keys. initmem7/9.bin could be made optional, the required data can also be sourced from certain particular BIOS dumps (the ones that NO$GBA requires) (and dsidumper could also be upgraded to produce such augmented BIOS dumps). Not sure about aeskeys.bin.

* rewriting the trainwreck that is the SD/MMC controller code. As an example, Flipnote Studio tends to softlock while saving due to bugs in that code. It was written mainly to get things working, and it shows, but now that we know that things are working, we can afford to rewrite it cleanly. Some other DSi parts (AES engine, ...) could use some cleanup, but they aren't as bad.

* adding features such as cameras and wifi connectivity

* ability to direct-boot DSi games, and work out something for DSiware

* the holy grail, eventually, merging all that back into regular melonDS]]>
melonDSi beta release -- by Arisotura Tue, 15 Oct 2019 22:21:19 +0000
Anyway, here we go: we present you melonDSi, in unfinished beta glory. The JIT beta will follow, but I first want to ensure Generic is comfortable with it.

The end goal will be to merge this with regular melonDS once it's in a better state, but, for now, we're providing it for the curious folks.

How to use melonDSi

BE SURE TO READ THIS BEFORE USING MELONDSI. You are using a beta release, provided with zero polishing and with a debug console. I expect that you know what you are doing to atleast some extent.

melonDSi will require several files to be dumped from a DSi. It's a bit messy right now, but we hope to come up with more user-friendly solutions in the future.

So, if you have a DSi, you might want to install unlaunch on it.

A quick rundown of the files melonDSi requires:

* bios9.bin / bios7.bin: DS-mode BIOS images. Those are the same as the ones from a DS. They shouldn't be required to get DSi software running, but melonDSi will refuse to start if they aren't present.

* bios9i.bin / bios7i.bin: DSi-mode BIOS images. Those can be dumped from your DSi using the good ol' dsbf_dump.nds.

* firmware.bin: SPI FLASH image. See above. dsbf_dump.bin yields a 'FWxxxxxx.bin' file which you need to rename. It should be 128K.

* aeskeys.bin: AES keys at boot time, dumpable using dsidumper.

* initmem9.bin / initmem7.bin: Boot-time contents of ARM9 ITCM and ARM7 WRAM, containing several other useful keys. See above.

* nand.bin: DSi NAND image, ideally dumped with fwTool. This should be dumped from the same console as dsikeys.bin and initmem9/7.bin.

* boot2_9.bin / boot2_7.bin: Second-stage bootloader, to be extracted from the NAND with twltool, using the following command:

twltool boot2 --in nand.bin

Gives you files arm9.bin and arm7.bin, to be renamed to boot2_9.bin and boot2_7.bin respectively.


Make sure that your DSi-mode ARM7 BIOS is good. Open bios7i.bin in a hex editor. Ensure that the first 32 bytes are the following:

06 00 00 EA 06 00 00 EA 1F 00 00 EA 04 00 00 EA
03 00 00 EA FE FF FF EA 13 00 00 EA 00 00 00 EA

Make sure that your NAND dump has the 'nocash footer' at the end. Open it in a hex editor and check that it has the string 'DSi eMMC CID/CPU' at offset 0xF000000. This is the beginning of the 64-byte 'nocash footer', which holds the eMMC CID and console ID.

Ideally make sure that direct boot is disabled.

Using melonDSi

As with regular melonDS, System->Run will boot the firmware, which should land you into the DSi menu if you did everything right.

You will need to recalibrate the touchscreen through the system settings app to have it work correctly.

melonDSi can boot some DSi-mode apps, but they currently don't work that well.

Booting games

melonDSi is able to boot DS-mode games from the DSi menu. They currently run with the DS-mode BIOSes and backwards-compatible touchscreen mode, but at the 133MHz clock speed and with 16MB of RAM.

DSi-enhanced games do not boot yet.

Direct boot is likely to break things as it hasn't been updated yet.

Other issues

Do not attempt using savestates, they can and will go wrong.

melonDSi is based on melonDS 0.8.1, so it will have the bugs present in that version.

Enjoy! Or atleast, try to.

melonDSi, Windows 64-bit
melonDSi, Linux 64-bit

If you want to help us take melonDSi further: here's our Patreon]]>
2D accuracy: it's a rabbit hole too -- by Arisotura Sat, 14 Sep 2019 19:01:18 +0000
Anyway, mosaic is a typical feature of old consoles with 2D engines, including the DS. It basically applies a pixelation effect to graphics, as shown here:

The basic idea is that the screen is split in a grid, whose dimensions are variable (configured by register 0x0400x04C on the DS). For each 'cell' in the grid, all the pixels are colored the same as the first (top-left) pixel. In reality, it's a bit more complex, as on the DS the effect can be applied per-layer and per-sprite, and you can even specify different grid sizes for BG layers and sprites, but fundamentally it's more or less the same thing, it pixelates shit.

Sounds simple enough, right?

It's a bit tricky to implement when you're trying to write a performant renderer, though. Which is more or less why blargSNES never supported it.

As far as melonDS is concerned, mosaic was implemented in version 0.5, but (among other silly bugs) it never worked quite right as far as sprites were concerned, especially when those use rotation/scaling. But, at the time, I didn't do much past the original implementation, mostly because I don't know of a lot of games that use the mosaic effect. The lack of test cases meant it stayed supbar.

Until, well, now.

First thing to do is to write some test cases for sprite mosaic. BG mosaic seems quite simple, even though I would still have to probe it extensively for edge cases, but sprite mosaic is a bit more oddball, as seen here:

Those snapshots were taken from Grey, my capture-card DS. This isn't some weird buggy emulator, sprite mosaic really is that weird.

The sprite being mosaic'd is a simple ball sprite. The cyan sprite behind doesn't have mosaic applied to it, but both share the same size and rotscale parameters. The sprites on the left have double-size mode enabled, which merely doubles the bounding box of rotscaled sprites.

I have yet to work out all the oddities, but, from what I have observed, it seems that sprite mosaic doesn't work as we thought. If vertical mosaic is simply done by adjusting the source Y coordinate when rendering BG layers or sprites, horizontal mosaic is a different topic. As far as sprites are concerned:

* horizontal mosaic is restrained to the sprite's bounding box (vertical mosaic is too, but only in one direction)
* seeing how it is affected by the presence of neighbor sprites, horizontal mosaic seems to be applied after all the sprites are rendered
* it seems to keep track of which pixels belong to each sprite, even when said pixels are transparent

As of now, none of the existing DS emulators get sprite mosaic right. Probably, none of the GBA emulators get it right either, since the GBA uses a similar renderer, and it doesn't look like anybody has ever figured out the mystery of sprite mosaic.

I wanted to be able to check my mosaic implementation against hardware for correctness in many situations, to make sure I'd gotten the logic right. But constantly dumping frames from Grey and hand-comparing them against melonDS would be tedious. I wanted to have a better tool.

So, I set to work. I opened ds_capture.exe in IDA, opened up the Linux DS-capture code example and WinUSB documentation, and set to work. And, in one evening, MelonCap was born.

This is pretty simple. Left screen is the output from melonDS, middle is what is captured from Grey, right is a color-coded visualization of the differences.

The precision isn't exactly optimal though: melonDS outputs RGB666 color, like the DS does, but the capture card degrades it to RGB565 due to technical constraints. Well, this is atleast better than RGB555.

This is pretty much made for graphical testing in specific use cases, as there's no synchronization mechanism, one has to ensure that both melonDS and the DS are rendering the same things.

Anyway, the example in the screenshot above shows something interesting. There's been all that effort put towards pixel-perfect 3D, but our 2D engine isn't pixel-perfect (neither is the 3D one, but... yeah). We figured pixel perfection with the 2D engine would be a given, and, yet... here we are.

The bottom screen has BLDCNT configured so that all layers/sprites will be dimmed, with BLDY=1. This is hardly noticeable, and is likely an oversight. However, the colors generated by melonDS in this situation don't quite match hardware output.

The function for the brightness-down effect is described by GBAtek as:

out = in - ((in * EVY) / 16);

However, a quick hardware test tends to show that the actual function is:

out = in - (((in * EVY) + 7) / 16);

There's probably more of this shit everywhere, so the 2D engine will have to be heavily tested for accuracy. Fun shit ahead.]]>