Hybrid MIDI Sound Emulation MUNT/FluidSynth

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Hybrid MIDI Sound Emulation MUNT/FluidSynth

Postby BBond007 » Mon Nov 12, 2018 6:08 pm

Greetings:

I have compiled MUNT (mt32emu_alsadrv - mt32d) for MiSTer Linux and I have adapted "MidiLink" to send it data via the ALSA sequencer API.

I am using a iMic USB sound device for sound output.

I started a new topic because this is not really "USB MIDI" anymore :)

I'm using CM-32 ROMS.

Tie Fighter (ao486) --> https://youtu.be/ddXy0UY7I5A
Leisure Suit Larry Enhanced (Minimig) --> https://youtu.be/bfeHWpN9eZc
Monkey Island 2 (ao486) --> https://youtu.be/bZfDPOXx-Wo

MUNT --> https://github.com/munt/munt

:cheers:
Last edited by BBond007 on Fri Nov 16, 2018 2:12 am, edited 4 times in total.

NegSol
Captain Atari
Captain Atari
Posts: 268
Joined: Sat Dec 05, 2015 9:22 pm

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby NegSol » Mon Nov 12, 2018 6:16 pm

Great stuff! Thanks for pushing the topic :mrgreen:

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3178
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby Sorgelig » Mon Nov 12, 2018 7:12 pm

I didn't get it. What device produces the sound?

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby BBond007 » Mon Nov 12, 2018 7:32 pm

Sorgelig wrote:I didn't get it. What device produces the sound?


Oh, good question...

My bad... I forgot to mention that I'm using USB sound device. I should add that info.
Last edited by BBond007 on Mon Nov 12, 2018 7:36 pm, edited 1 time in total.

NegSol
Captain Atari
Captain Atari
Posts: 268
Joined: Sat Dec 05, 2015 9:22 pm

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby NegSol » Mon Nov 12, 2018 7:36 pm

Yeah - some sort of internal mixing would be nice - but that can never be :) - in the old time you had to mix externally also - so actually it is really authentic :lol:

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3178
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby Sorgelig » Mon Nov 12, 2018 9:51 pm

i think, the output of softSynth can be routed back to FPGA for final mixing to HDMA/analog.
I don't know how much CPU time is required for MUNT and will it slowdown MiSTer binary. It would be good to use MUNT on a single core and different than MiSTer - this way it should be smooth for MiSTer main thread.

ghogan42
Atari freak
Atari freak
Posts: 50
Joined: Wed Oct 17, 2018 7:27 pm

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby ghogan42 » Mon Nov 12, 2018 11:42 pm

On some older projects using the rpi2 and rpi3 some people had buffer underruns and the like when running munt because of cpu speed. I compiled it myself for my own project and munt ran fine for me though. So I don't know. But Munt might be close to taking up a full core on MiSTer.

Obviously, Munt is the best option for retro systems. But fluidsynth can also be run with soundfonts that are similar to other roland/retro synths. So if there was a way to route the audio to the fpga for mixing that could be something to look at too.

Here is a thread about running munt on a rpi2 on Vogons: https://www.vogons.org/viewtopic.php?f=29&t=46899

And a thread about a rpi3 synth box that might be useful for scripts/compiling info: https://www.vogons.org/viewtopic.php?f=9&t=58174

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby BBond007 » Tue Nov 13, 2018 2:27 am

ghogan42 wrote:On some older projects using the rpi2 and rpi3 some people had buffer underruns and the like when running munt because of cpu speed. I compiled it myself for my own project and munt ran fine for me though. So I don't know. But Munt might be close to taking up a full core on MiSTer.

Obviously, Munt is the best option for retro systems. But fluidsynth can also be run with soundfonts that are similar to other roland/retro synths. So if there was a way to route the audio to the fpga for mixing that could be something to look at too.

Here is a thread about running munt on a rpi2 on Vogons: https://www.vogons.org/viewtopic.php?f=29&t=46899

And a thread about a rpi3 synth box that might be useful for scripts/compiling info: https://www.vogons.org/viewtopic.php?f=9&t=58174


Munt is good for a lot of retro stuff but fluidsynth I'm sure is a better choice for GM stuff. Munt has second device where they have arranged the instruments to be more GM-like, but it's definitely not ideal. I used to have a sysex command I would run that would do the same thing with a real MT-32 and that was about the same.

Sorgelig wrote:i think, the output of softSynth can be routed back to FPGA for final mixing to HDMA/analog.
I don't know how much CPU time is required for MUNT and will it slowdown MiSTer binary. It would be good to use MUNT on a single core and different than MiSTer - this way it should be smooth for MiSTer main thread.


It is typically using around 20 - 30% CPU while playing music. MiSTer menu seems to always use 50%. The CPU is typically idle around 20%. I did read then if run as root that MUNT is able to elevate its priority if necessary.

As a quick test just to see what would happen I SFTP'd 1GB HDF file while the synth was playing. Did not cause it to skip a beat...

I can also try and compile MUNT with better optimization.

This is while playing a MIDI file with MidiPlay workbench app:

Mem: 230624K used, 276080K free, 380K shrd, 4332K buff, 187500K cached
CPU: 53% usr 27% sys 0% nic 19% idle 0% io 0% irq 0% sirq
Load average: 1.60 1.27 1.13 3/74 14007
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
887 1 root R 22968 5% 49% /media/fat/MiSTer /media/fat/Minimig_2
6811 6731 root R < 15296 3% 30% mt32d
6725 972 root S 5048 1% 0% sshd: root@pts/0
8089 6731 root S 2828 1% 0% midilink MUNT


This is while playing LSL Enhanced:

Mem: 502292K used, 4412K free, 380K shrd, 1600K buff, 460560K cached
CPU: 44% usr 28% sys 0% nic 26% idle 0% io 0% irq 0% sirq
Load average: 2.05 1.86 1.51 4/73 14736
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
887 1 root R 22968 5% 50% /media/fat/MiSTer /media/fat/Minimig_2
6811 6731 root R < 15296 3% 23% mt32d
6725 972 root S 5048 1% 0% sshd: root@pts/0
14584 6731 root S 2828 1% 0% midilink MUNT

ghogan42
Atari freak
Atari freak
Posts: 50
Joined: Wed Oct 17, 2018 7:27 pm

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby ghogan42 » Tue Nov 13, 2018 2:43 am

Munt is good for a lot of retro stuff but fluidsynth I'm sure is a better choice for GM stuff. Munt has second device where they have arranged the instruments to be more GM-like, but it's definitely not ideal. I used to have a sysex command I would run that would do the same thing with a real MT-32 and that was about the same.

It is typically using around 20 - 30% CPU while playing music. MiSTer menu seems to always use 50%. The CPU is typically idle around 20%. I did read then if run as root that MUNT is able to elevate its priority if necessary.

I can also try and compile MUNT with better optimization.


Mem: 230624K used, 276080K free, 380K shrd, 4332K buff, 187500K cached
CPU: 53% usr 27% sys 0% nic 19% idle 0% io 0% irq 0% sirq
Load average: 1.60 1.27 1.13 3/74 14007
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
887 1 root R 22968 5% 49% /media/fat/MiSTer /media/fat/Minimig_2
6811 6731 root R < 15296 3% 30% mt32d
6725 972 root S 5048 1% 0% sshd: root@pts/0
8089 6731 root S 2828 1% 0% midilink MUNT


It's good that cpu usage is relatively low. Since we have a dual core it shouldn't be a problem right? You can use "taskset" or something to make mt32d run on the second processor if it interferes with MiSTer's normal functionality I guess.

And I know that softsynths use little cpu when not actually playing music, so if we wanted to, munt (or fluidsynth or other) could just always run in the background and then any core that wanted to could just output midi data to it without having to worry about how to start up and stop the softsynth. Although that would make it hard to offer multiple softsynth options since you don't want them all to be running. But then you could even just enable/disable it through mister.ini

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby BBond007 » Tue Nov 13, 2018 2:59 am

ghogan42 wrote:
It's good that cpu usage is relatively low. Since we have a dual core it shouldn't be a problem right? You can use "taskset" or something to make mt32d run on the second processor if it interferes with MiSTer's normal functionality I guess.

And I know that softsynths use little cpu when not actually playing music, so if we wanted to, munt (or fluidsynth or other) could just always run in the background and then any core that wanted to could just output midi data to it without having to worry about how to start up and stop the softsynth. Although that would make it hard to offer multiple softsynth options since you don't want them all to be running. But then you could even just enable/disable it through mister.ini



mt32d is using 4% just doing nothing, so I think it's best to kill that process if another synth is used.

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Re: Hybrid MIDI Sound Emulation (Roland MT-32/CM-32)

Postby BBond007 » Thu Nov 15, 2018 3:03 am

Here is a view showing CPU load while 'mt32d' is running.

I switched to a Turtle Beach USB sound dongle. I like the resulting sound a lot better than the iMic I was previously using.

Indiana Jones and the Fate of Atlantis game demo: --> https://youtu.be/wg4wTVM1ylA

Here the is the same thing (for comparison) with Adlib/Sound Blaster sound which I really think sounds great, but inferior to the hybrid emulation. --> https://youtu.be/HP6mt4N0vuI

And the real MT-32 --> https://youtu.be/pHaamnmIy2E

:cheers:

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Re: Hybrid MIDI Sound Emulation MUNT/FluidSynth

Postby BBond007 » Fri Nov 16, 2018 2:21 am

Quick Update:

Here is MidiLink / FluidSynth running a Sound Canvas SC-55 Soundfont.

While FluidSynth used a lot less of the ARM CPU (while running) than MUNT, both use 4% sitting idle so rather then having both running simultaneously on different MIDI ports, I've decided to have MidiLink manage those daemons and make sure only the one required is running.

Tie Fighter --> http://y2u.be/fKy1UIVX3L4

(for comparison)
Real Sound Canvas SC-55 MKII --> http://y2u.be/vZQ12AZ5ZWk
Real Yamaha TG100 --> http://y2u.be/hb9eK3czxk8
Hybrid MUNT Emulation (Roland CM-32 ROMS) --> http://y2u.be/ddXy0UY7I5A
Real Roland MT-32 --> http://y2u.be/ky90I_EWL6c

FluidSynth:
http://www.fluidsynth.org/

Roland SC-55 Soundfont used (link included):
http://y2u.be/_J8-CgvC-As

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3178
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Hybrid MIDI Sound Emulation MUNT/FluidSynth

Postby Sorgelig » Fri Nov 16, 2018 6:19 am

Try to play the games and see if you have controller lag while playing the MIDI music. Also if some games have loading in background - check the loading time (while playing the music).
As i've told already, any CPU loading (the CPU core where MiSTer is running) may affect the HPS/FPGA communications. This communication is based on polling (not events), so any CPU loading may affect the communication.

Soft-synths must run on a different CPU core than MiSTer binary for better experience.

htop utility shows which process is running on which core.

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Re: Hybrid MIDI Sound Emulation MUNT/FluidSynth

Postby BBond007 » Mon Nov 19, 2018 6:42 am

Sorgelig wrote:Try to play the games and see if you have controller lag while playing the MIDI music. Also if some games have loading in background - check the loading time (while playing the music).
As i've told already, any CPU loading (the CPU core where MiSTer is running) may affect the HPS/FPGA communications. This communication is based on polling (not events), so any CPU loading may affect the communication.

Soft-synths must run on a different CPU core than MiSTer binary for better experience.

htop utility shows which process is running on which core.


Tried htop, the menu is on its own core and never seems to drop below 99% (Unless I SFTP). Both process (munt and fluidsynth) elevate their priority by default. FluidSynth does not use much CPU but it uses 33% RAM for the soundfont I'm using.

Most of all these Amiga games (and many of the DOS) that support MIDI are mouse based Adventure games so I would assume the mouse would also have slowdown/lag?

Anyway, here is my latest test with a Anders Jensen MIDI --> https://youtu.be/YtJGwHw9Z1w

0:00 MUNT
4:20 FluidSynth
8:35 Network UDP MIDI to Roland Sound Canvas VA
Last edited by BBond007 on Mon Nov 19, 2018 6:45 pm, edited 1 time in total.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3178
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Hybrid MIDI Sound Emulation MUNT/FluidSynth

Postby Sorgelig » Mon Nov 19, 2018 7:48 am

htop is not relevant in this test.
Yes, mouse also have the same route as joysticks and will be affected. So, need to play some games and see if there any additional lag in inputs. You can just play MIDI in background and play some game which has no MIDI.

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Re: Hybrid MIDI Sound Emulation MUNT/FluidSynth

Postby BBond007 » Thu Nov 22, 2018 4:51 am

Sorgelig wrote:htop is not relevant in this test.
Yes, mouse also have the same route as joysticks and will be affected. So, need to play some games and see if there any additional lag in inputs. You can just play MIDI in background and play some game which has no MIDI.


After more testing, I'm not noticing any additional lag.

Do you know if routing the sound back into the FPGA would incur any additional CPU usage/load?

Is there a way for the FPGA to appear to be a ALSA PCM device?

Thanks

ijor
Hardware Guru
Hardware Guru
Posts: 3629
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: Hybrid MIDI Sound Emulation MUNT/FluidSynth

Postby ijor » Thu Nov 22, 2018 11:31 am

It might also be possible to have the whole thing on the FPGA side. No open source MIDI synthesizer core? Perhaps won't fit on the PC/486 core that hasn't much left space. But on others it might.
Fx Cast: Atari St cycle accurate fpga core

BBond007
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 115
Joined: Wed Feb 28, 2018 3:23 am

Re: Hybrid MIDI Sound Emulation MUNT/FluidSynth

Postby BBond007 » Thu Nov 22, 2018 6:37 pm

ijor wrote:It might also be possible to have the whole thing on the FPGA side. No open source MIDI synthesizer core? Perhaps won't fit on the PC/486 core that hasn't much left space. But on others it might.


Ideally, I think you would want to include two distinct synth cores because MT-32/CM-32 and General Midi modules (like Sound Canvas) differ greatly in operation and support. Some DOS ISA sound boards and Sound Canvas GM modules have a a "MT-32 mode" but this only works OK for games that use the default sound bank - most completely reconfigure the sound bank via sysex messages before starting.

I'm not aware of any MT-32/CM-32 implementation in FPGA. The videos I have seen demoing FPGA MIDI implementations don't sound anything like General MIDI and sound more like IBM PCjr or Tandy sound.

Sort of interesting, At first cursory glance at the MT-32 compatibility list, I thought that ST and Amiga supported more games in common, but it seems that 'Space Quest III' is the only one they share in common.

The Amiga and ST really don't have many supported MIDI games at all (compared to DOS) so for those platforms I see the ability to experience legacy sequencer software perhaps more of a benefit. I was messing around with Bars and Pipes yesterday with FluidSynth on the Minimig core and that was pretty interesting. Someone more musically inclined could definitely do something interesting with that setup or say Cubase on FX Cast.

If space is limited in the ao486 core, beyond a hardware MPU-401, perhaps the remaining space is better used for something else like CPU improvements or FPU.

Here is the start FPGA MPU-401 implementation by Electron Ash --> https://github.com/ElectronAsh/fpga_mpu401


Return to “MiSTer”

Who is online

Users browsing this forum: uigiflip and 3 guests