melonDS RSS http://melonds.kuribo64.net The latest news on melonDS. Status update -- by Arisotura http://melonds.kuribo64.net/comments.php?id=105 Sun, 17 Nov 2019 10:28:33 +0000

Couple things though.


On melonDS, things are going to be mostly on hold until I manage to get a proper apartment and am all settled in and all. But, two things. First, Generic said that the JIT branch is ready for a beta release, so I'll be doing that. Second, as a next direction to take the project, I'm thinking about adding support for AR cheat codes.

I'll also be trying to fix the remaining issues with the OpenGL renderer, but some are going to be tricky. Also, things related to this renderer: I need to come up with a not-too-hacky solution for dual-screen 3D and upscaling, and to add support for filtering in OpenGL mode (including xBRZ or something of that ilk).


But, blegh. The apartment. I visited one a while ago, now I'm going to have to put together the paperwork and hope it isn't too late. There are other apartments I can try, but, hey, I liked that one. So, here's hoping.


Oh also, this is Nov 17, birthday of the yellow vest movement. French protest movement that started one year ago, was originally against a gas tax raise but turned radical and against precarity in general, which seems to be a common theme these days (thinking about Hong Kong, Chile, ...). Despite the fact that the movement as it is will not bring revolution, it seems to want to never end... (what kind of protest movement lasts one whole year?)

But we can see that it's having an effect on protests in general, they turn more radical now. Like recently, a student from Lyon set himself on fire out of despair after being thrown into precarity. Had that happened two years ago for example, nobody would have cared, it'd just have been "another depressive sicko attempts suicide, film at eleven". But instead, this sparked several student-led protests and blockades, including one that broke into the Ministère de l'Enseignement Supérieur.

Dec 5 is going to be the general strike, too. I first thought that would be lame, as most of these attempts were a while back, but, now, within the different context, this looks like it's going to be quite something. May also cause some trouble on my end, but, we'll see.

Anyway, happy birthday to the yellow vests. I've been following the movement closely, and even taking part in it, as it gave me hope when I was at the worst of depression, so this means a lot to me.]]>
http://melonds.kuribo64.net/comments.php?id=105
When things turn out unexpectedly simple -- by Arisotura http://melonds.kuribo64.net/comments.php?id=104 Sun, 20 Oct 2019 10:02:42 +0000
Shows that taking a break does wonders.

Just yesterday, I idly looked at that bug again. Reproduced the freeze, then pressed F11 (debug key printing some status to the console) a few times to see if I could see a pattern. The ARM9 seemed to exhibit typical 'idle thread' behavior; the ARM7 however, seemed to get stuck in a loop. A quick glance at my IDB of the firmware code confirmed this:

AES wait routine

If you aren't fluent in ARM assembly, this function waits for the AES engine to finish whatever crypto operation it is doing, by checking the busy flag in AES_CNT.

But, in our case, the ARM7 got stuck inside this because, somehow, said busy flag remained set forever.

Some logging of AES operations shown that, while loading the game, the firmware would start a decrypt operation with a block count of zero. melonDSi didn't check for that case, and instead waited for data input that would never come (heh), causing the busy flag to remain set forever. You guess how this goes.

After quickly fixing this bug, I tried loading a cart game again. It still froze on a white screen, but the logs showed it was getting further. In particular:

UNIMPLEMENTED ARM9 NDMA3 START MODE 04, 04100010->021153A0 LEN=1 BLK=1 CNT=A4044000

That meant the game was actually booting, and attempting to load its data, using NDMA ('new DMA' implemented in the DSi) for whatever reason. NDMA start mode 04 is for cart slot DMA, which is triggered every time a data word is transferred from the cart.

This was quickly taken care of, too. A bunch of the NDMA start modes mirror those of 'old' DS DMA, so I just hooked those into the functions responsible for taking care of DS DMA. The only one that needs more extensive checking is GXFIFO mode, since it has a maximum block length of 112 words, I'm not sure how that interacts with the block subdivision system of NDMA.

Anyway, with this, I was finally able to boot DSi-enabled games.

Nothing terribly exciting to show tho. After all, most of the DSi-enhanced games out there don't have much extra features other than making use of the DSi cameras, so we have to do something to emulate them.

Loading DSi ROMs is also a bit finicky as they have to be one specific way for it to work (right now, with the DSi-specific data encrypted). And, some of the ROM dumps out there lack the DSi-specific data, so it would be good to add some verification and warn the user if their ROM is a bad dump.


Speaking of which, I am thinking of the ways I can make this evolve, now that it's atleast working to some extent. Namely:

* lowering the entry barrier, that is, upgrading dsidumper to dump all the required material in one go

* getting rid of some of the required files. boot2_7/9.bin can be sourced from the NAND directly, provided I can find a way to derive the required decryption keys. initmem7/9.bin could be made optional, the required data can also be sourced from certain particular BIOS dumps (the ones that NO$GBA requires) (and dsidumper could also be upgraded to produce such augmented BIOS dumps). Not sure about aeskeys.bin.

* rewriting the trainwreck that is the SD/MMC controller code. As an example, Flipnote Studio tends to softlock while saving due to bugs in that code. It was written mainly to get things working, and it shows, but now that we know that things are working, we can afford to rewrite it cleanly. Some other DSi parts (AES engine, ...) could use some cleanup, but they aren't as bad.

* adding features such as cameras and wifi connectivity

* ability to direct-boot DSi games, and work out something for DSiware

* the holy grail, eventually, merging all that back into regular melonDS]]>
http://melonds.kuribo64.net/comments.php?id=104
melonDSi beta release -- by Arisotura http://melonds.kuribo64.net/comments.php?id=103 Tue, 15 Oct 2019 22:21:19 +0000
Anyway, here we go: we present you melonDSi, in unfinished beta glory. The JIT beta will follow, but I first want to ensure Generic is comfortable with it.

The end goal will be to merge this with regular melonDS once it's in a better state, but, for now, we're providing it for the curious folks.


How to use melonDSi

BE SURE TO READ THIS BEFORE USING MELONDSI. You are using a beta release, provided with zero polishing and with a debug console. I expect that you know what you are doing to atleast some extent.

melonDSi will require several files to be dumped from a DSi. It's a bit messy right now, but we hope to come up with more user-friendly solutions in the future.

So, if you have a DSi, you might want to install unlaunch on it.

A quick rundown of the files melonDSi requires:

* bios9.bin / bios7.bin: DS-mode BIOS images. Those are the same as the ones from a DS. They shouldn't be required to get DSi software running, but melonDSi will refuse to start if they aren't present.

* bios9i.bin / bios7i.bin: DSi-mode BIOS images. Those can be dumped from your DSi using the good ol' dsbf_dump.nds.

* firmware.bin: SPI FLASH image. See above. dsbf_dump.bin yields a 'FWxxxxxx.bin' file which you need to rename. It should be 128K.

* aeskeys.bin: AES keys at boot time, dumpable using dsidumper.

* initmem9.bin / initmem7.bin: Boot-time contents of ARM9 ITCM and ARM7 WRAM, containing several other useful keys. See above.

* nand.bin: DSi NAND image, ideally dumped with fwTool. This should be dumped from the same console as dsikeys.bin and initmem9/7.bin.

* boot2_9.bin / boot2_7.bin: Second-stage bootloader, to be extracted from the NAND with twltool, using the following command:

twltool boot2 --in nand.bin

Gives you files arm9.bin and arm7.bin, to be renamed to boot2_9.bin and boot2_7.bin respectively.


BEFORE STARTING

Make sure that your DSi-mode ARM7 BIOS is good. Open bios7i.bin in a hex editor. Ensure that the first 32 bytes are the following:

06 00 00 EA 06 00 00 EA 1F 00 00 EA 04 00 00 EA
03 00 00 EA FE FF FF EA 13 00 00 EA 00 00 00 EA

Make sure that your NAND dump has the 'nocash footer' at the end. Open it in a hex editor and check that it has the string 'DSi eMMC CID/CPU' at offset 0xF000000. This is the beginning of the 64-byte 'nocash footer', which holds the eMMC CID and console ID.

Ideally make sure that direct boot is disabled.


Using melonDSi

As with regular melonDS, System->Run will boot the firmware, which should land you into the DSi menu if you did everything right.

You will need to recalibrate the touchscreen through the system settings app to have it work correctly.

melonDSi can boot some DSi-mode apps, but they currently don't work that well.


Booting games

melonDSi is able to boot DS-mode games from the DSi menu. They currently run with the DS-mode BIOSes and backwards-compatible touchscreen mode, but at the 133MHz clock speed and with 16MB of RAM.

DSi-enhanced games do not boot yet.

Direct boot is likely to break things as it hasn't been updated yet.


Other issues

Do not attempt using savestates, they can and will go wrong.

melonDSi is based on melonDS 0.8.1, so it will have the bugs present in that version.



Enjoy! Or atleast, try to.


melonDSi, Windows 64-bit
melonDSi, Linux 64-bit

If you want to help us take melonDSi further: here's our Patreon]]>
http://melonds.kuribo64.net/comments.php?id=103
2D accuracy: it's a rabbit hole too -- by Arisotura http://melonds.kuribo64.net/comments.php?id=102 Sat, 14 Sep 2019 19:01:18 +0000
Anyway, mosaic is a typical feature of old consoles with 2D engines, including the DS. It basically applies a pixelation effect to graphics, as shown here:



The basic idea is that the screen is split in a grid, whose dimensions are variable (configured by register 0x0400x04C on the DS). For each 'cell' in the grid, all the pixels are colored the same as the first (top-left) pixel. In reality, it's a bit more complex, as on the DS the effect can be applied per-layer and per-sprite, and you can even specify different grid sizes for BG layers and sprites, but fundamentally it's more or less the same thing, it pixelates shit.

Sounds simple enough, right?

It's a bit tricky to implement when you're trying to write a performant renderer, though. Which is more or less why blargSNES never supported it.

As far as melonDS is concerned, mosaic was implemented in version 0.5, but (among other silly bugs) it never worked quite right as far as sprites were concerned, especially when those use rotation/scaling. But, at the time, I didn't do much past the original implementation, mostly because I don't know of a lot of games that use the mosaic effect. The lack of test cases meant it stayed supbar.

Until, well, now.

First thing to do is to write some test cases for sprite mosaic. BG mosaic seems quite simple, even though I would still have to probe it extensively for edge cases, but sprite mosaic is a bit more oddball, as seen here:






Those snapshots were taken from Grey, my capture-card DS. This isn't some weird buggy emulator, sprite mosaic really is that weird.

The sprite being mosaic'd is a simple ball sprite. The cyan sprite behind doesn't have mosaic applied to it, but both share the same size and rotscale parameters. The sprites on the left have double-size mode enabled, which merely doubles the bounding box of rotscaled sprites.

I have yet to work out all the oddities, but, from what I have observed, it seems that sprite mosaic doesn't work as we thought. If vertical mosaic is simply done by adjusting the source Y coordinate when rendering BG layers or sprites, horizontal mosaic is a different topic. As far as sprites are concerned:

* horizontal mosaic is restrained to the sprite's bounding box (vertical mosaic is too, but only in one direction)
* seeing how it is affected by the presence of neighbor sprites, horizontal mosaic seems to be applied after all the sprites are rendered
* it seems to keep track of which pixels belong to each sprite, even when said pixels are transparent


As of now, none of the existing DS emulators get sprite mosaic right. Probably, none of the GBA emulators get it right either, since the GBA uses a similar renderer, and it doesn't look like anybody has ever figured out the mystery of sprite mosaic.


I wanted to be able to check my mosaic implementation against hardware for correctness in many situations, to make sure I'd gotten the logic right. But constantly dumping frames from Grey and hand-comparing them against melonDS would be tedious. I wanted to have a better tool.

So, I set to work. I opened ds_capture.exe in IDA, opened up the Linux DS-capture code example and WinUSB documentation, and set to work. And, in one evening, MelonCap was born.



This is pretty simple. Left screen is the output from melonDS, middle is what is captured from Grey, right is a color-coded visualization of the differences.

The precision isn't exactly optimal though: melonDS outputs RGB666 color, like the DS does, but the capture card degrades it to RGB565 due to technical constraints. Well, this is atleast better than RGB555.

This is pretty much made for graphical testing in specific use cases, as there's no synchronization mechanism, one has to ensure that both melonDS and the DS are rendering the same things.

Anyway, the example in the screenshot above shows something interesting. There's been all that effort put towards pixel-perfect 3D, but our 2D engine isn't pixel-perfect (neither is the 3D one, but... yeah). We figured pixel perfection with the 2D engine would be a given, and, yet... here we are.

The bottom screen has BLDCNT configured so that all layers/sprites will be dimmed, with BLDY=1. This is hardly noticeable, and is likely an oversight. However, the colors generated by melonDS in this situation don't quite match hardware output.

The function for the brightness-down effect is described by GBAtek as:

out = in - ((in * EVY) / 16);

However, a quick hardware test tends to show that the actual function is:

out = in - (((in * EVY) + 7) / 16);

There's probably more of this shit everywhere, so the 2D engine will have to be heavily tested for accuracy. Fun shit ahead.]]>
http://melonds.kuribo64.net/comments.php?id=102
melonDS 0.8.3 -- by Arisotura http://melonds.kuribo64.net/comments.php?id=101 Wed, 04 Sep 2019 15:20:40 +0000

So what does this release bring? Well, we have been trying to address the issues present in previous 0.8.x releases (or sometimes even older releases, heh).


For example, I fixed the bug that was introduced with the new support for Ctrl+K type hotkeys. Basically, using Shift/Ctrl/etc as regular keys mapped to buttons was no longer possible. So, support for key mappings with modifiers was restricted to hotkeys. Meaning that using right Shift as R (as done by the default key mapping) should no longer cause input problems.


I have been trying to fix the issues we had with the framerate limiter and audio output, too, with moderate success. As I haven't been able to come up with a one-size-fits-all fix, there are now three different sync modes you can use, individually or together:

* Limit framerate: the oldschool framerate limiter. Although this is a revamped version that tries to average over several frames, reducing the likelihood of limiting too aggressively on certain games that internally run at 30FPS and are otherwise able to run fullspeed.

* Audio sync: synchronizes emulation to the audio output system. Seems to result in a bit more fluctuation in the framerate, but should prevent any audio stuttering.

* VSync (in the video settings dialog): synchronizes video output to your monitor's refresh rate. This only works with OpenGL, and currently only works under Windows (OpenGL support under Linux still needs more love). Also, DS games/programs may alter their framerate by messing with VCount, which VSync would be ill-equipped to deal with (unlike the two other sync methods).

I think most of the audio issues came from not properly syncing, which resulted in semi-regular overflows or underflows in the SPU FIFO, causing stuttering. The current audio output system cannot be precise enough to prevent those, as it works with small audio frames.

Speaking of which, I have also been revising it to use a more standard output frequency, in case some bad audio driver doesn't appreciate the previous frequency of 47340Hz. Now, it will attempt to run at 48000Hz, but it also allows SDL to specify another frequency if needed.


There are also a few other fixes that were long due (like OpenGL initialization failing under OpenGL <4.2), and some accuracy improvements, as usual.


As promised, beta builds of the JIT and DSi branches are coming soon, so stay tuned! Those will be based off older melonDS versions (0.8.2 and 0.8.1 respectively), though.


Enjoy!


melonDS 0.8.3, Windows 64-bit
melonDS 0.8.3, Linux 64-bit

If you're feeling generous: here's our Patreon]]>
http://melonds.kuribo64.net/comments.php?id=101
We're alive -- by Arisotura http://melonds.kuribo64.net/comments.php?id=100 Fri, 23 Aug 2019 13:21:03 +0000

Also, on the DSi front:

I'm also unsure what to do. there are BIOS dumps out there that have been filled as much as possible, by dumping the lower half and adding back the known parts of the upper half that can be sourced from elsewhere, as seen there: http://problemkaputt.de/gbatek.htm#biosdumping . NO$GBA expects these particular dumps. I might want to adopt that method too, but I'm afraid such dumps are less complete than dsikeys.bin/initmem7.bin/initmem9.bin.

Opinions on this are welcome. I'll need to check how much of the AES keys is sourced from the BIOS, vs how much is needed to run shit like the DSi firmware.

For reference: dsikeys.bin contains all the AES keys, as extracted from the AES engine after an unlaunch boot. initmem7.bin and initmem9.bin are chunks of ARM7 WRAM and ARM9 ITCM that contain keys (RSA, Blowfish) and other shit needed for initialization, those are mostly (but not entirely) sourced from the BIOS.


On other fronts, all sorts of exciting things are being developed for melonDS! Those who follow repo activity may have noted that DLDI support is being worked on. Stay tuned!


Edit- comments about how "I don't give a shit about DSi emulation!! Work on the features I want!!" or "nobody cares about DSi" or whatever, are getting on my nerves. There are very good reasons to work on DSi emulation, so I swear that if this shit continues I'll be deleting comments and handing out bans.]]>
http://melonds.kuribo64.net/comments.php?id=100
Progress on the DSi front -- by Arisotura http://melonds.kuribo64.net/comments.php?id=99 Sun, 04 Aug 2019 00:09:35 +0000
So instead, I did what I typically do in these situations: procrastinate and work on something else.

In this case, the DSi front.

If you remember, this was at a point where it needed wifi initialization to be able to launch titles. I was stuck at a particular point of the init procedure, where it waited for something and I had trouble figuring out what. But, regardless, I set to work.

In the meantime, googling some of the strings present in the firmware's wifi code brought me to this Atheros driver codebase which is very close to what the firmware uses. This helped me a lot in figuring out how things work and what the firmware was expecting.

And, well...

melonDSi config melonDSi flipnote

It's possible to boot shit from the firmware now!

Couple issues with this though.

* the system settings app boots, but it's impossible to get further due to the lack of touchscreen support
* Flipnote boots but freezes when starting a new flipnote
* pure DS-mode games seem to be bootable, DSi-enhanced games are not

Also, some DS-mode games suffer from audio issues for whatever reason. Well, it's worth noting that atm melonDSi does not support dynamic ARM9 clock adjustment, so DS games are basically running in forced DSi mode.

Well, we're getting there, I guess.]]>
http://melonds.kuribo64.net/comments.php?id=99
The JIT -- by Arisotura http://melonds.kuribo64.net/comments.php?id=98 Sun, 21 Jul 2019 15:11:00 +0000
I meant to fix a few of the issues we had lurking around, so we could release 0.8.3. I managed to fix some issues that caused random crashes on exit, then began trying to fix the framerate limiter. Thing is, it is directly related to the audio output code and the issues that has. So, yeah, I still haven't found a solution to that tricky problem.

I also need to struggle to keep some motivation at times. Looking at the patterns in how I work, it is likely that I have ADHD. I have yet to get diagnosed and all, but this wouldn't surprise me.

Anyway!

As you may have noticed, Github user RSDuck sent a pull request for the JIT recompiler. This is a bit of a tricky situation though. I don't want to have to deal with a situation where people make third-party JIT builds, but at the same time, the JIT isn't quite ready for general use, as we're still ironing out various issues with it.

However, the JIT is said to give a performance boost ranging from 30% to 100%, which is fairly nice, especially as there's room for improvement. RSDuck also said that porting the JIT to other architectures, like ARM, should be easy, so we might finally have Switch builds that run at acceptable speeds.

Or, also, Android builds. But, dunno. On one hand, this is a market I want to address, but on the other hand, history from Dolphin has shown that the Android userbase isn't exactly the nicest to deal with, so... yeah. And don't get me started on Android itself and the fun that coding for it is.


But first, let's focus on what we have in front of ourselves.


I'm going to try as hard as I can to fix the audio issues for 0.8.3 (and also add a vsync setting).

We will also be releasing beta builds of the JIT and the DSi branch, for those of you who want to mess around with things.]]>
http://melonds.kuribo64.net/comments.php?id=98
welp. -- by Arisotura http://melonds.kuribo64.net/comments.php?id=97 Sat, 06 Jul 2019 14:31:06 +0000
It will require some amount of wifi support to be able to launch titles. The amount required makes the 0.2 wifi stub look tiny in comparison. Basically the DSi menu tries to communicate with the wifi CPU thing and upload firmware to it.

We can either HLE it given sufficient documentation (and have to deal with ass-shitting high-level wifi concepts) or LLE the damn thing.

We'll see.]]>
http://melonds.kuribo64.net/comments.php?id=97
the heat -- by Arisotura http://melonds.kuribo64.net/comments.php?id=96 Mon, 01 Jul 2019 16:49:00 +0000
..

Let's just say it's a bit hard to be productive when your outside temperature is a scorching, capitalism-induced 43°C.

It's back to more reasonable temperatures now, but regardless, we don't do much besides bathing at the beach or at the nearby river.

I kind of need to take a break at this point though. Like, there are many things I want to do with melonDS, but I have too much ideas, and there's only so much I can do at once.

I will need to release a quick-fix 0.8.3 though. Atleast to fix the bug I introduced when adding support for 'Ctrl+M' type keys'. Basically you can't press any key while keeping R pressed if you keep the default mapping (right Shift).

I also want to try and fix the framerate limiter, which, askskfdksgsf.

On the DSi side, there will be some alpha builds of melonDSi once I finish the DSi dumper. Right now, the dumper is able to extract AES keys from the AES engine. The rest will be a breeze: dumping the BIOSes, the memory holding RSA/Blowfish keys, the NAND, the eMMC CID and console ID. It's mostly just matter of copying memory to a file, except the NAND is going to be a bit more involved, but nothing insurmontable.

I gave melonDSi a small pause, because I got stuck on AES-CCM (needs hardware tests) and also didn't want to let the master branch rot. Right now, it's not in a very interesting state -- all it does is boot the DSi menu, but it's still not getting further. And also because I keep fixating on the lack of full DSi bootroms. I'm even considering getting in touch with someone who is doing the hardware haxing required to dump them, or attempting to do it myself.

melonDSi will eventually be merged into regular melonDS when it's ready. You can think of it like medusa vs mgba.

I'm also trying to optimize melonDS. I have several targets in mind, but we'll see. The plan is to get it to run on less powerful platforms, like the Switch, or Android.

Speaking of this, user RSDuck is working on a JIT. The first iteration will be for x64, but he said it will be easy to port it to other architectures. If we can get this going without sacrificing too much accuracy, it will be a great help on the performance front.]]>
http://melonds.kuribo64.net/comments.php?id=96