melonDS RSS http://melonds.kuribo64.net The latest news on melonDS. melonDS 0.7.3 -- by StapleButter http://melonds.kuribo64.net/comments.php?id=74 Sat, 05 Jan 2019 13:05:52 +0000

Something that should have been done long ago, and has finally been done: the emulator's main loop was rewritten to use absolute timestamps to keep track of cycle counts. Overall, it's more reliable (far less chances of desyncs), more accurate, and also a bit faster, and the code is cleaner. So we win on all fronts.

This should fix the recent flickering issues, like that of Colour Cross.

The downside is that this breaks savestate compatibility with older melonDS versions.


In the same vein, there's another fix to GX timings. We knew that vertex timings were different when a vertex completed a polygon, but we just found out that timings also differ based on whether the polygon passes culling and clipping. This will help with games that need their display lists to be running fast enough.


To hopefully fix the saving issues, we built a new database, resulting of a mix between the Advanscene XML and savelist.bin from the R4 Wood firmware. For better detection, the new romlist.bin indexes games by their serial code rather than by the ROM's CRC32, which also means that it will work with hacked ROMs. This means you will need to make sure to replace romlist.bin with the newer one if you're extracting this melonDS release over an older one.

There's also a particular fix for Pokémon Mystery Dungeon - Explorers of Sky, which uses 128K EEPROM. It should now save properly.

However, note that the correct savefile size for this game is 128KB. If you have a savefile which is 256KB, the game will read it fine but fail to save to it (because that size is detected as FLASH memory and not EEPROM). You can fix this by opening your savefile in a hex editor and deleting the upper half of it. This is especially important if you use a savefile coming from DeSmuME as it will create a 256KB file.


We also have a neat little pile of UI improvements.

melonDS now supports hotplugging joysticks. That being said, it still defaults to the first joystick available on your system. I'm not quite sure how to go about handling multiple joysticks.

Under Linux, the crashes coming from the input config dialog should be fixed.

The main window is smarter about remembering its size: it also remembers whether it was maximized.

There's a tentative fix for hiDPI under Windows, which should atleast make the screens scale correctly.

There's a new menu for setting the window size to an integer scaling factor (1x to 4x).

The setting for savefile relocation when using savestates is now disabled by default. While the feature may be useful, having it enabled by default was confusing for unaware users.


And finally, the misc fixes:

* fix STRD_POST (fixes music in Just Sing - Vol 2)
* 2D: fix blending bugs
* add support for 8bit reads to DISPCNT/BGxCNT (fixes The Wild West)
* make nocashprint also work in ARM mode


Enjoy!


melonDS 0.7.3, Windows 64-bit
melonDS 0.7.3, Linux 64-bit

melonDS Patreon if you're feeling generous]]>
http://melonds.kuribo64.net/comments.php?id=74
Some brighter news tho -- by StapleButter http://melonds.kuribo64.net/comments.php?id=73 Thu, 03 Jan 2019 00:55:26 +0000

Regarding the Colour Cross issue, I have an idea. It seems to stem from the GXSTAT busy flag, like, sometimes it's still set when it should be cleared because we aren't syncing the GX often enough.

I could go back to the old way (syncing it per-opcode), but I'm afraid this is going to be suboptimal. The better solution I have will require a bit of reworking how melonDS handles timing. As a bonus it would be more straightforward, easier to maintain and less error-prone.


I still have no idea about Spellbound though. I found that DMA timings can differ depending on how source/destination addresses are updated, but that doesn't apply in the Spellbound case. So, still no idea there.


I guess we can try releasing 0.7.3 after fixing Colour Cross, though.


I also have a few ideas for improving this site. A nice one to have would be an intro blurb before the blog posts, showing off melonDS and why it's awesome and you should totally download it, like the lolSNES site has.

Might also change the download page a bit, namely, to add downloadable extras like the latest romlist.bin or the dsbf_dump.nds copy that's been squatting the Kuribo64 uploader since forever.

Maybe even later, some form of compatibility list. Which will need some dedicated testers to fill it and keep it up to date.

If you have ideas how to improve the site, visually or technically, I'm open to all suggestions!


I'm contemplating running a sorta-buildbot on blargcity (the server), like we had for lolSNES. Looking at MinGW and how we can use it to produce 64-bit Windows and Linux builds.

Might even produce 32-bit builds if it's any worth doing. Noting that it wouldn't run on Windows XP as it is, libui uses several APIs that were introduced in Vista.

One fear I have over that is that having a buildbot might kill incentive to do proper releases. Kind of what happened with lolSNES.


Oh well.]]>
http://melonds.kuribo64.net/comments.php?id=73
Happy new year from melonDS -- by StapleButter http://melonds.kuribo64.net/comments.php?id=72 Tue, 01 Jan 2019 16:23:26 +0000
This year is not starting too well. I implemented an instruction cache in melonDS, and what are the results? Absolutely none whatsoever (other than being a bit slower). It achieved as much as farting in the wind. The problem games I previously mentioned are still giving me the finger.

My motivation is utterly sapped.

..

We hope this year will be better for you.]]>
http://melonds.kuribo64.net/comments.php?id=72
Updated romlist.bin -- by StapleButter http://melonds.kuribo64.net/comments.php?id=71 Mon, 31 Dec 2018 02:57:37 +0000
In the near future, the database will be changed to identify ROMs by game code rather than CRC, so it will also work with hacked ROMs and whatnot. But for melonDS 0.7.2 or 0.7.1, you can get this updated database.

-> Download here <-

Just overwrite your existing romlist.bin with this and you're good to go!

Remember, if your game is still failing to save, you may need to delete the savefile.]]>
http://melonds.kuribo64.net/comments.php?id=71
World of shit -- by StapleButter http://melonds.kuribo64.net/comments.php?id=70 Sun, 30 Dec 2018 16:30:00 +0000 dual-screen 3D shitting itself in Colour Cross.

The game draws all its animated background tiles/etc separately, giving each its own transforms. It sends unpacked GXFIFO commands and doesn't use DMA. All in all, not very efficient, but probably not a bad way to do it given what they're doing.

Anyway, the game expects to take a certain time to draw each screen (for the bottom screen, it spans two frames). If the code runs too slow or too fast, it shits itself. Given how geometry is sent, GX timings don't affect this.

It doesn't have to be precise, but there's a window within which this has to fall, so we can't make it absurdly fast or absurdly slow and hope to fix it.


On the other hand, we have the Spellbound issue which I mentioned in the previous post. There, the code has to be running slow enough, I can't see any other way out. Both DMA and GX timings are correct there. I could always try dumping the display list and running it on hardware within the same conditions to see how it goes, but I'm sure the issue is with code timings.


So I guess this is it. No amount of hodgepodging timings will get us out of this, as I feared. And I really want to avoid 'solutions' like game-specific hacks or 'toggle this timing setting and see if it fixes your game'.

So, we have to emulate the ARM9 instruction cache.

On the plus side, since code fetches are mostly sequential, this shouldn't be a big performance penalty. The cache can be checked only upon branches and on cache line boundaries (on the DS, that is every 32 bytes). The ARM9 caches have 4-way associativity, which means a measly 4 cache lines to check before we know whether the current address is cached. So I believe we can afford to emulate it without killing performance.

The data cache would be the one killing performance; because data accesses are far less predictable, the cache would have to be checked upon every memory access. For now, it appears we can get away with not emulating it, so let's pray.


For less performant devices, we could always have a 'performance' profile that uses the old timing model (2 cycles per code fetch in cached memory, always). It's less exact and more likely to break something else, but it seems to run most things reasonably well, so it's a good candidate for a performance/accuracy compromise.


I still have to look into Spellbound, because of course making code fetches taking 2c doesn't fix it. It needs to be atleast 5c to work correctly, but 5c breaks other things, no surprise there.

Quick testing confirms what I suspected there, it only ran correctly in older melonDS versions because the GXFIFO DMA was too slow, taking 3c per word. On hardware, transferring from main RAM to IO registers (GXFIFO or whatever else) takes 2c per word, which is also the case in melonDS after the timing renovation. So, well, we can't take it back.

(the difference looks puny, but when transferring hundreds of units, it quickly adds up)


In brighter news, well, 0.7.3 will come out soon-ish, once I come up with something for the ARM9 code cache.

There are also a few improvements making it more user-friendly. For example, I made savemem relocation when loading savestates disabled by default, as I figure that having it enabled by default can be confusing.

We're also trying to fix the input config dialog crashes under Linux. I wasn't able to reproduce them because I wasn't looking at the right place: they happen when mapping joystick buttons, and taking that into consideration, I reproduced it, and could fix it. It was quite silly, reason is that we use a SDL timer callback to sense joystick input, and were updating the UI directly from that timer callback. Forgetting that GTK is not thread-safe and the UI shouldn't be manipulated outside of the UI thread, even after I ran into similar issues with the main window. Using uiQueueMain() lets us get around this and fixes the crash.

I'm still not 100% sure there though, there was a crash under FreeBSD that appeared to be different.

There's also a little improvement to joystick input, it can detect if the joystick is disconnected/reconnected while melonDS is running.

It's still limited to one joystick though, I'm not quite sure how to go about handling multiple joysticks and whether we can trust the OS not to go and change their indexes.

I also want to work on network support, adding the interface selector and built-in DHCP/NAT, but maybe for 0.7.4. I already have enough on my plate here, and that's without getting into the 0.8 hardware renderer.]]>
http://melonds.kuribo64.net/comments.php?id=70
Short break -- by StapleButter http://melonds.kuribo64.net/comments.php?id=69 Mon, 24 Dec 2018 14:06:13 +0000
Especially when it seems that all you're throwing at the wall is pointless. That sure deals a blow to motivation.


I picked one of the available Advanscene databases for savemem type detection, not knowing what all the available ones mean and what the differences are. Well, this one seems to be a Swiss cheese, full of errors and missing games.

If you have any ideas for a better database (like, what the fuck are the differences between all the Advanscene ones), I'd love to hear about it. Among the suggestions are that of using AKAIO's database, so I might look into that.

Also, a possibility to consider: detecting ROMs by game code rather than by CRC.


We're getting more timing renovation victims, and this time they're goddamn real. They all seem to be the same thing, missing/corrupted geometry. I don't know about all the games, but I looked into one of them (Spellbound), and it looks, well, not very good. The developers coded their own software version of GXFIFO DMA (the base idea is to transfer a fixed-size chunk of display list data every time the GXFIFO is less than half-full), which works by relying on the GXFIFO IRQ set to trigger when it's less than half-full.

However, the way this is coded expects that by the time a DMA transfer is finished, the GXFIFO will already be less than half-full, so the next chunk will be started immediately. Of course, there's a point in melonDS where that doesn't happen, causing the game to think the transfer has completed, and send a SWAP_BUFFERS command in the middle of the display list, which works as well as you'd expect.

Something's up with our timings. I hope we don't need to emulate the ARM9 caches.


And the goddamn input config dialog under Linux. I have no fucking idea why that would crash. I can't reproduce the crash. It's working just fine under my setup (Ubuntu 16.04). I can't even tell if we're just hitting some obscure GTK bug.


argl.


and merry Christmas to you reader.]]>
http://melonds.kuribo64.net/comments.php?id=69
Hotfix release -- by StapleButter http://melonds.kuribo64.net/comments.php?id=68 Sun, 16 Dec 2018 13:26:39 +0000
This has since been fixed, and a hotfix has been pushed. If you downloaded melonDS 0.7.2 before this post, just redownload it to get the fixed version.]]>
http://melonds.kuribo64.net/comments.php?id=68
melonDS 0.7.2 -- by StapleButter http://melonds.kuribo64.net/comments.php?id=67 Sun, 16 Dec 2018 01:10:57 +0000
You have already gotten a glimpse of it, but let's go over all the changes since 0.7.1, because it's been a fast week.


So first of all, the issues we have seen pop up in NSMB, Pokémon, or Etrian Odyssey after the timing renovation, have been fixed. Nicely, the new timings uncovered some stealth GX bugs that would surely have bitten us another day under other circumstances.

So the claim that was made for 0.7.1 ("now your games run better than ever") is finally more than a phony advert :P


Second big thing is, as you probably guessed by now, microphone support. If your machine has a microphone connected and if you are using SDL 2.0.5 or more recent, you can blow or blare bullshit into it and it just works! If that is not the case, you can also opt to feed a WAV file or white noise as microphone input.

Note that this feature is still experimental. Quality of microphone input may not be optimal, especially when using a physical microphone. WAV input works better.

WAV and white noise modes send input when pressing a microphone hotkey (default is the key right next to right Shift, '?' on QWERTY keyboards). WAV mode can take any reasonably small file, encoded as 8-bit or 16-bit PCM, signed or unsigned, any number of channels (it will read the first channel).

All this can be configured in the new audio settings dialog, where you can also set the volume for audio output.


Which brings us to the new hotkey system. For now, aside from the aforementioned mic hotkey, there is only another one: 'Close/open lid', which simulates closing/opening the DS. Default key is Backspace.

Oh and the hotkey system is an extension of the regular input system, which means you can also assign joystick buttons to these hotkeys.


Speaking of the input system, Windows users may have noticed that the input config dialog was abysmally slow, taking several seconds to open and generally feeling quite laggy. With a quick little fix in libui, that is no more, and the dialog now feels a lot more normal.

Some attempt was made at fixing possible crashes with that dialog under Linux, but those crashes may need more investigation. In my current setup (Ubuntu 16.04), I am unable to reproduce them, or break the input config dialog in any way. I don't know which are caused by melonDS and which are obscure GTK bugs. While one of the stack traces that were reported showed something I could easily work around, the other pointed at some obscure bug where some function internal to GTK is getting a NULL value and crashing.


On a whim, I added support for nocash-style debug print, which enables homebrew to print to the emulator's console. Also, Windows users don't need to get a debug build to get console output -- running the release version of melonDS from cmd will dump the console output there.


We also have some welcome contributions from some fine Github comrades:

* FPS limiter toggle, courtesy abcdjdj
* flatpak manifest, by cpba
* Linux libpcap library names added to the libpcap loader by dogtopus
* and finally Aqueminivan renovating the readme, it looks cool now!


And, last but not least, a whole bunch of misc bugs were fixed:

* black screens in Puzzler World 2
* American Girl - Kit Mystery Challenge! screeching garbage audio in the house
* blending fail in Pokémon Mystery Dungeon - Explorers of Sky
* lack of background music in Club Penguin: Herbert's Revenge
* a few wrong entries in romlist.bin were corrected
* config dialogs could be opened multiple times


Merry Christmas!


melonDS 0.7.2, Windows 64-bit
melonDS 0.7.2, Linux 64-bit

melonDS Patreon if you're feeling generous]]>
http://melonds.kuribo64.net/comments.php?id=67
I am fucking relieved -- by StapleButter http://melonds.kuribo64.net/comments.php?id=66 Sat, 15 Dec 2018 13:23:01 +0000 NSMB's cannons putting on some weight to Pokémon characters getting magical growth to, less amusingly, dual-screen 3D shitting itself in Etrian Odyssey.

The NSMB and Etrian Odyssey issues also had the fun aspect that they didn't occur in the USA ROMs, for some reason.

Well, that's a thing of the past now.

I knew the issues appeared after the timing renovation, but that was about it. I figured that instead of blindly hodgepodging timings until the issues disappeared, and potentially getting into some whack-a-mole game, I would take the time to understand the issues.


So the first one was the NSMB cannon.

Technically, the cannon body is a billboard (3D sprite that always faces the camera). The 'big' version is just the same but with the cannon body scaled too big.

So I investigate how the cannon body is rendered. For that, we can use NO$GBA's debugger version, atleast until melonDS gets similar tools :P

Conveniently, the cannon body is the first polygon in the display list. Before it gets rendered, there are some transforms set up so it will face the camera. In this case, the second-to-last scale transform sometimes got wrong values that caused the billboard to appear too big, hence the bug.

So I look into the game's code to find out where and how that scale transform is calculated. Noting that debugging NSMB is made way easy by the awesome folks at the NSMB Hacking Domain who have a complete IDA database of the game.

The game feeds some transform commands to the GX, waits until it's done, then reads the clip matrix (projection and position matrices product), and derives the scale transform from it. In our case, the bug came from how melonDS handled the GXSTAT busy flag: there was a thin chance that, after submitting some commands, the game could manage to read GXSTAT before the busy flag was actually set. Thus it would decide that the GX was already done, and proceed to read the clip matrix while the transforms were executing, and get some of the values wrong.

The issue was unrelated to CPU/memory timings, it just happened to be a simple bug that slow enough timings kept hidden, until now. So, it was easily fixed.

The Pokémon issue is likely the same issue, and is likely fixed too. We just need to check it.


The Etrian Odyssey issue, however is different.

Quick investigation showed that the game was almost never swapping its screens, despite attempting to do dual-screen 3D.

The DS can only render 3D to one screen at the time, so how do you do dual-screen 3D? With a bit of trickery. For example, you would render a 3D frame to the top screen, and at the same time, capture that frame to VRAM. Then, during the next frame, you swap the screens, so that you're now rendering 3D on the bottom screen, and on the top screen you render the bitmap you previously captured. With this typical method, you get 3D on both screens, but you're limited at 30 FPS.

So obviously, if the game is not swapping the screens, the rendering is only going to be a trainwreck.

So, we look into the code responsible for swapping the screens and making that whole thing work. It runs upon VBlank, and basically does the following:

1. wait ~400 cycles, by running a subs/bcs loop 200 times
2. check the GXSTAT busy flag; if that is set, give up
3. swap screens, do more setup

The base idea there was likely that if a frame took too long to render, the game would avoid swapping the screens at the wrong time and causing flickering.

In our case, however, the SWAP_BUFFERS command took a bit too long and accidentally triggered that.

It was set to take 392 cycles as per GBAtek's data, but measurements on hardware showed that that duration is more like 325 cycles. Revising our code accordingly, Etrian Odyssey finally rendered normally.


So, as the title says, I am fucking relieved there. I thought we were faced with some problem that could only be solved by emulating the ARM9 caches, which, urgl. It turned out to not be the case. And melonDS 0.7.2 is totally gonna rock!]]>
http://melonds.kuribo64.net/comments.php?id=66
Sneak peek -- by StapleButter http://melonds.kuribo64.net/comments.php?id=65 Fri, 14 Dec 2018 15:47:00 +0000


I'll let you guess ;)]]>
http://melonds.kuribo64.net/comments.php?id=65