Opening your eyes to the world
I haven't been very active these days, mostly real life getting in the way, as usual. Anyway, I have been thinking of cool things I could work on for melonDS. I have some ideas in mind regarding input, like supporting multi-touch and other fun touchscreen-related features, but lately I felt like working on some actual emulation, especially in the DSi field. It's well known that DSi emulation in melonDS needs more love.

I was thinking of implementing things like SoundEx and microphone support, but Generic said he was going to implement that, so I'm letting him having fun there. Instead, I went for improving camera emulation. After all, cameras are the main thing DSi-exclusive games make use of, so DSi emulation with no camera support just feels incomplete.

melonDS 0.9.4 already has some very basic camera emulation, but it basically just does the bare minimum to pass the camera init sequence, and then returns a fixed stripe pattern (which is basically just the picture data register returning a fixed value). Nothing exciting.

For example, you can open the DSi camera app and get to the actual camera part of it, but trying to take a picture will result in a system error. Reason for this is that the camera can be set to output at multiple resolutions. It has two modes of operation, named 'preview' and 'capture', which makes sense in applications like digital cameras: preview mode would provide a low-res preview of what the camera sees, that can be displayed on a built-in screen, while capture mode would provide a full-res picture that can be saved. The DSi camera lets you configure two contexts, each with their output resolution, image format, and various other options (for example, flipping the picture...).

So I first had to add support for these modes of operation. I first implemented the registers necessary to handle the camera mode/context switch. But, of course this wasn't enough. The stub in place for picture transfer assumed a resolution of 256x192, but in this case, the resolution is changed to 640x480. This also means that the DSi camera app expects more picture data to come, and it will softlock if there isn't enough data incoming.

This is where my camera test homebrew came in handy. I modified it so that I could do things like switching between the front and back cameras, switch between preview and capture modes, and even use the picture cropping feature to view different areas of the picture in 640x480 mode (as obviously it wouldn't fit onscreen). Doing this let me get a good grasp on how these features were supposed to work.

Then I felt confident implementing all that into melonDS.



This is a preexisting still picture, not actual camera input. But regardless, we have working camera emulation now, and this is half the battle. The other half would be reading picture data from sources like an actual camera, feeding it into the emulator, and integrating all that into the UI.

Taking a picture in the DSi camera app also works, and thanks to the SD folder-sync system, I can easily retrieve the resulting picture. For example, the original file, and the resulting DSi picture.

There is also still a bunch of tidbits to take care of, as far as camera emulation goes. For example, all details and edge cases pertaining to the picture transfer hardware. It's tricky to work with, because from my understanding, picture data is transferred from the camera on a per-scanline basis, but the transfer hardware holds it in a buffer which is limited to 512 words, and it has no idea what resolution the picture is. It just knows to trigger a DMA transfer every N scanlines (N being configured in CAM_CNT), but if these N scanlines amount to more data than the buffer can hold, you're going to get blank lines in your picture.

The DSi cameras themselves also have a looooot of low-level details, many of which don't matter in the case of melonDS. It's like for the DS wifi hardware: we have to deal with these things to some extent because we're doing low-level emulation, and the games/apps we emulate run on the bare metal, but a lot of it isn't useful to our emulator, and in most/all cases a lot of it is initialized to some generic settings and never touched again.

But hey, this looks like a pretty good start. If you're interested, you can check it out in the camera_betterer branch, atleast until we merge it.
poudink says:
Apr 17th 2022
Nice, I've been waiting for this since the original DSi camera branch. DSi emulation definitely isn't complete without camera emulation.
Pk11 says:
Apr 18th 2022
Is the source of your camera test app shared anywhere? I've been meaning to get 640×480 mode working in my camera demo app but last time I tried I couldn't get it working right...
Arisotura says:
Apr 18th 2022
it's not online anywhere because it's an utter mess, but I could put it up somewhere

basically:
camWriteMcu(curdev, 0xA103, 2);
camWaitMcuClr(curdev, 0xA103, 0xFFFF);


the value you write in A103 will be 2 for capture mode (640x480) and 1 for preview mode (256x192) (although their specific resolutions depend on other MCU registers, but the 'standard' setup will provide these resolutions). by default the camera is in preview mode.

that's the gist of it. then for 640x480 mode you obviously need to adjust the DMA settings, and the scanline count setting in CAM_CNT (take into account that the transfer buffer can only hold 512 words). the camera app uses a DMA block length of 320 words combined with a 'scanline count minus one' value of 0.

also, make sure you don't have a transfer going at the time you try to switch modes.

I believe you can also read out register A104 to find which mode the camera is in: 3=preview, 7=capture.
Pk11 says:
Apr 18th 2022
Thanks! That was pretty simple lol, I didn't think to change the "scanline count - 1" and wasn't sure exactly what the DMA block length was (in retrospect very simple lol, just the word count) so before I'd gotten stuck just doing a DMA transfer length for 1024x480 which "worked" except it was actually sending 4 640x120 images with a bunch of useless extra pixels

Image taken by it:
https://bbs.ピケ.コム/image/1650324449.png
Pk11 says:
Apr 19th 2022
Oh, also, fyi my dsi-camera program freezes on initializing the cameras in melonDS, it could well be me doing something wrong but since it works on hardware and in no$gba it might be worth looking into, certainly not high priority though lol

(tested v1.0.0 and v1.1.0 using melonDS 0.9.3 and 529a6900896618fee9f5d85e3e5fa12d5973871c on macOS Catalina 10.15.7)
Arisotura says:
Apr 19th 2022
oh, thanks for the heads-up! I can see it, it seems there's something iffy with my I2C implementation. definitely ought to fix it, if it works on hardware.

other than that, glad you got it working!
Arisotura says:
Apr 19th 2022
well, I fixed the I2C comm bug, and your demo also revealed another silly bug with the raw YUV mode (that went unnoticed because for some silly reason Nintendo doesn't use that mode when taking a picture), so thanks there!

this also has me wondering what is the maximum resolution the camera can do. in theory the resolution registers can take any values, but in practice the camera has a maximum resolution of 640x480, so it would probably be blurry if you went above that (if it worked at all). also, the transfer hardware would enforce a maximum horizontal resolution of 1024 pixels in the default color format. I don't know if there would be any limit vertically, though.
Pk11 says:
Apr 19th 2022
Hmm it now crashes for me on loading anything in DSi mode using 852a0eae003acbd1b7112190617c40ee88944b50 (just realized I forgot to actually checkout the camera_betterer branch before lol)
https://haste.ピケ.コム/oyixoyitij.yaml

Maybe I put the just image in the wrong spot or something? I put "test.jpg" in my home folder, next to melonDS.app, and next to the melonDS executable though so I dunno.

--------

Since you mentioned going above 640x480 I was curious and tried taking some 1024x1024 images, it doesn't seem to get any additional image data, but it did have some somewhat interesting results:
https://bbs.ピケ.コム/image/1650394385.png
https://bbs.ピケ.コム/image/1650394390.png
https://bbs.ピケ.コム/image/1650394395.png

The first image you take seems to always have those exact pixels on line 589 (even on different consoles, though I didn't try on 3DS), then every image after that has a weird corruption of the first image. Oddly the corrupted image bit seems to vary by console, those images (from my Korean DSi) start at line 740, but on my Japanese DSi LL it starts at line 906, though that may be something to do with the image not the console, I didn't test super thoroughly.

I also did some testing with small resolutions and it seems to work fine for anything 4:3, but behaved weirdly when trying to do other aspect ratios (well, at least 1:1 as that's all I tried)
https://bbs.ピケ.コム/image/1650396625.png (128x128)
https://bbs.ピケ.コム/image/1650396630.png (256x256)
https://bbs.ピケ.コム/image/1650396635.png (192x144)
https://bbs.ピケ.コム/image/1650396641.png (32x24)

Not sure if any of that info's particularly useful for anything, but I found it somewhat interesting so figured I'd share it.
Pk11 says:
Apr 19th 2022
Wait, I'm dumb lol

I just noticed the console output said it rejected the image because it was over 128 MB lol, I guess that's what I get for using an Apple desktop background because it was the first image I saw xD

Scaled it down to 640x480 and it works now.
Arisotura says:
Apr 21st 2022
so Qt can't take images bigger than 128MB? that's good to know, I guess

also your tests there are interesting... setting up aspect ratios different than 4:3 might require altering some other parameters, like pixel clock or whatever, I don't know all the low-level details

and for resolutions bigger than 640x480: these might just end up reading 'out of bounds' within the MCU RAM or something of that kind
nocolx says:
Apr 21st 2022
MelonDS Emulator Crashes
Zeta says:
Apr 22nd 2022
Ive been trying to play DSI games, but they didn't work lol.
txmutt says:
Apr 25th 2022
Somewhat unrelated, but what is SoundEx? :o
Arisotura says:
Apr 26th 2022
it's the extra sound hardware in the DSi, it connects the DSP and the old DS mixer together, and also handles microphone input
bisaalz says:
May 5th 2022
Will nintendo dsi sound app work after fixing SoundEx???
Post a comment
Name:
DO NOT TOUCH