Anyone to convert a Z80 music player?

[Targhan]

Hi all,

I am an Amstrad CPC coder, and has released Arkos Tracker 2 lately, a cross-platform (Windows, Mac, Linux) tracker to compose for Amstrad CPC, MSX, Spectrum and Atari ST. It is still in alpha (which is why I didn't promote it outside the CPC scene), but is already very stable and, I hope, powerful.

There are three Z80 players:
- One "generic", that is the normal player, good balance between speed and memory.
- One "lightweight", smaller but also more limited.
- One "streamed", using a new format called AKY, which is some kind of compressed YM, with a very interesting compression ratio (basically, it is better than MXM, and 3 times smaller than what is done on CPC).

I have adapted all three players for all the hardware, except for Atari, as I don't know 68000. However, I believe only the AKY must be converted. It is both the simplest and fastest player, and the memory overhead is, I believe, almost negligible for an Atari ST.

So I'd like to ask if one of you would be interested in converting the player. The good news is that the format is fully documented. A good knowledge of the AY is required of course, but it is only about addressing the right registers (there is currently no SID in my tracker).

The code source of the players is fully available in the Arkos Tracker package, but if you're interested in this little project, it is better that we discuss about it here (or via PM). The format is effective, but quite simple to decode, so with a bit of guidance, the whole convertion can be done within a few evenings!

Thanks!

Targhan/Arkos.

[spiny]

Gwem, author of maxYMiser, would probably be a good person to talk too

http://www.preromanbritain.com/maxymiser/

[ggn]

Is the 6502 player source available? Then we could use something like mos2mot to bring it to 68000 and add platform specific stuff and optimisations with a couple of passes.

[Targhan]

Thanks for your answer.
The 6502 player is not available right now. Someone asked to do it, but I had no news since. But I think it's really more efficient to convert it from a Z80, the 6502 is really more limited!

[FedePede04]

i tried your program in windows, it look like it run great, could have been a couple more example in its native format...
i like you interface not the usual windows interface...

[Targhan]

Thanks! More example will appear with time. Anyone can also participate .

[dma]

Wow, nice set of features and cool non native interface.
It would be excellent if it could support the standard trackers copy/paste/block/transpose keyboard shortcuts (those present in PC FastTracker 2).

[ggn]

Just for the heck of it, I started converting the z80 code line by line, trying to accommodate for small changes but not seeing the big picture yet. If anyone wants to have a laugh, point your browsers at https://github.com/ggnkua/Arkos-Tracker-2-ST or https://bitbucket.org/ggnkua/arkos-trac ... rc/master/ if you prefer!

[Targhan]

Wow, nice set of features and cool non native interface.
It would be excellent if it could support the standard trackers copy/paste/block/transpose keyboard shortcuts (those present in PC FastTracker 2).

Thanks! Yes of course, copy/paste/undo/redo are on the TODO list (as stated in the Roadmap page). But first I wanted to have a fully finished engine, import/export and players. This should be ready quite soon.

Just for the heck of it, I started converting the z80 code line by line, trying to accommodate for small changes but not seeing the big picture yet.

Wow! That's great! HOWEVER: you should have asked me before starting, you should NOT convert the "stabilized" player, because it only means something on an Amstrad CPC (each instruction always takes the same cycles, so it's rather easy for us to have "stable" code. Too bad we don't have timers). But the code is uglier and slower because of this. So you should convert only the "PlayerAky.asm", and that's it. Do not bother with the other players (AKG, Lightweight)! I don't think they are relevant on Atari ST.

Don't hesitate if you have any question, I'll glad to help.

Also, do you think the AKY format can have a use on ST, on a game/demo? I hope so. It's must better than YM in term of compression, but it's a bit slower of course, since data has to be decoded.

Thanks again!

[FedePede04]

Also, do you think the AKY format can have a use on ST, on a game/demo? I hope so. It's must better than YM in term of compression, but it's a bit slower of course, since data has to be decoded.

when you say YM format do you mean leonard's STSound YM files, and if yes how do you compress the file,
and wont the file still be to big for a 8bit machine?

[ggn]

Hello!

Wow! That's great! HOWEVER: you should have asked me before starting, you should NOT convert the "stabilized" player, because it only means something on an Amstrad CPC (each instruction always takes the same cycles, so it's rather easy for us to have "stable" code. Too bad we don't have timers). But the code is uglier and slower because of this.

Okay, I understand your concern. But really I wouldn't worry about slower/uglier code as from what I've seen so far the code will definitely require an optimisation pass regardless which source is used. For now the real goal is to get something working really. Then we can look into optimising it.

Of course the real driving force will be people (coders, musicians) actually wanting to use the player in productions. THEN you'll start to feel at home ("why does the player take 3 scanlines? My effect uses 99.9999% CPU and I didn't budget anything for music!")

By the way, what do you mean by "stable" here? That it takes a fixed amount of clock cycles so people can synchronise with the CRT beam easier? This is actually a thing that ST people also do so it's also of interest.

To be honest the thing that matters to me here is that the player and tune format is stabilised and there won't be major changes to it. I assume that's the case?

So you should convert only the "PlayerAky.asm", and that's it. Do not bother with the other players (AKG, Lightweight)! I don't think they are relevant on Atari ST.

Well, I just took the latest version of the player I found in the archive

As I said above, people always require light/small/fast versions of a music player, so don't dismiss that idea so quickly . Can you outline the differences in a bit more detail? Are the other players missing features?

Don't hesitate if you have any question, I'll glad to help.

Well since you're offering help - In the source archive I also see an "accurate" player. Does this take a lot more time than the normal player?

There'll be plenty more questions when I'll actually try to feed a tune into this thing!

Also, do you think the AKY format can have a use on ST, on a game/demo? I hope so. It's must better than YM in term of compression, but it's a bit slower of course, since data has to be decoded.

It all depends on the people, doesn't it? But when you see talented people like dma-sc expressing interest, there's a good chance!

Thanks again!

No problem, but I'll reserve the "thanks" until there's something working! And again, don't worry about me picking the "wrong" source, by the time this is final it'll have very few similarities with the original

[explorer]

Maybe the release of AKY format file would help.

[Targhan]

when you say YM format do you mean leonard's STSound YM files, and if yes how do you compress the file, wont the file still be to big for a 8bit machine?

My format (AKY) treats the whole music, not as a "dumb" YM list, but as small sequences of notes. Since I created the tracker, it is easy for me to split the sound stream into smaller streams. There is no "packaging" like LZ, only a smart (I hope) organization of data. So the compression ratio is much higher than some compression format we use on CPC (AYM), or MXM (on MSX), plus the player is very fast, and requires no buffer. To give you an example, a complex song of 5 mns, without any repetiting patterns, can be encoded within 16kb.

Okay, I understand your concern. But really I wouldn't worry about slower/uglier code as from what I've seen so far the code will definitely require an optimisation pass regardless which source is used. For now the real goal is to get something working really. Then we can look into optimising it.

Yes, but the "Stabilized player" code is a bit more complex to manage some branching easier. So really, my advice is NOT to convert this player.

By the way, what do you mean by "stable" here? That it takes a fixed amount of clock cycles so people can synchronise with the CRT beam easier? This is actually a thing that ST people also do so it's also of interest.

Yes, by "stable", I meant indeed "fixed amount of clock cycles". So indeed, you may want to use the "stabilized player" if you are interested in such feature, finally .

To be honest the thing that matters to me here is that the player and tune format is stabilised and there won't be major changes to it. I assume that's the case?

Yes. The format is fully described in the package of AT2, so feel free to delve into it!

Can you outline the differences in a bit more detail? Are the other players missing features?

- Generic (AKG): full blown player. It is not "compressed", manages all the effects and features of AT2. Best compromise between speed and size.
- Lightweight: the same, but with less features so that the player is smaller (less effects, less instrument possibilities). Most music will work with this set of features.
- AKY: "streamed" player. Smaller, very fast, but the music size is larger.

In the source archive I also see an "accurate" player. Does this take a lot more time than the normal player?

It does take about one/two scanlines on a CPC, so it is significant for a demo. But really, only very complex song will require it (switching hardware sound quickly between the three channels. 99.99% song don't). I wouldn't bother, but it's up to you!

[Targhan]

Maybe the release of AKY format file would help.

My mistake! I thought it was in the package, it is not. Contact me (contact at julien-nevo dot com, or via the AT2 website) and I'll send it to whoever wants it. I'll also publish it in the next release.

[FedePede04]

My format (AKY) treats the whole music, not as a "dumb" YM list, but as small sequences of notes. Since I created the tracker, it is easy for me to split the sound stream into smaller streams. There is no "packaging" like LZ, only a smart (I hope) organization of data. So the compression ratio is much higher than some compression format we use on CPC (AYM), or MXM (on MSX), plus the player is very fast, and requires no buffer. To give you an example, a complex song of 5 mns, without any repetiting patterns, can be encoded within 16kb.

i have also thought of doing something similar when i export my score to ymt, but i have not done it yet, because i want to do a whole new YMT editor (so i have to wait )
but i also compress my YM stream i have a word at every line, telling me which register that have new data
so if the word is %1000000000000000 then is only reg0 that will be change and if it say %0010100000000000 then reg 2 and 4 have new datas and so on it can be optimize further but it is ok for now.

my compress format "my version of Outrun Splash wave Time : lenght 4:50:43m data change 300HZ size 313041 bytes"
if the file was uncompress size would be round 1396128 bytes. it still take a lot

[FedePede04]

just did a quick test, I change the sounds to 200hz instead of 300hz and the splash wav size was reduce to
228983 bytes from the 313041 bytes

[Targhan]

Well, it's still quite a lot (even for an Atari). My AKY format can come in handy, I guess .

[FedePede04]

you are right its a lot but it also depend which ST you are tagging, most STE today have 4 MB and 5 min is also a quite long score plus one could reduce the HZ a little more. i have not rand into space problem yet...
but i will definitely say there are room for you player, the more the better , i will look forward to see what ggn does.
I will pass making a player for it, i have been doing music apps for most of my life, so now i will much rather code something else.
But good luck with it...

[ggn]

Okay I have a question! (well I have a bunch of them but this is probably the most crucial right now) What does this snippet of code do?

Code: Select all

                ld a,11         ;Next regiser

                ld b,d
                out (c),a       ;f400 + register.
                ld b,e
                out (c),0       ;f600.
                dec b
                outi            ;f400 + value.
                ld b,e
                out (c),c       ;f680
                ex af,af'
                out (c),a       ;f6c0.
                ex af,af'


I've read some docs a bit and I understand that on the amstrad the psg is hidden behind some ports so you don't have direct access to it. I didn't get too deep into the docs so my guess for now is that the f400+register is a write to psg register select so you can select the register to write data to, while f400+value is the actual byte written to the register. But I think I should be safe and ask rather than make bad assumptions

Also, to give some context: on the ST the YM2149 is mapped directly to memory, so the register select/read data are aliased to $ff8800 and write data to $ff8802. So, a write to the chip would be something like

Code: Select all

move.b #register,$ffff8800.w
move.b #value,$ffff8802.w


(I'm sure that people are already typing up angry responses at my non-optimal code - relax people, it's only for illustration purposes )

So anyway, I'd like to be sure how I should map the above Z80 code to 68k.

[Targhan]

Well, what you see is the struggle we must endure to access our dear AY .

On a CPC, accessing the PSG must be done via the OUT instruction (because it is OUTside the Z80):
#f400 + PSG register
#f6c0 (validate the register)
#f600 (sleep)

#f400 + value
#f680 (validate the value)
#f600 (sleep)

Yay!

[ggn]

So hi again,

Today I finished porting over the player code at sort-of-blind mode. It definitely doesn't work right now but at least it assembles without errors - woohoo . Of course during this first pass I did get an overview of the file format, and then I opened the editor and played with the export functions a bit. So I do have some observations/questions/suggestions/concerns I'd like to mention.

1. First of all, it's really nice you can export in binary and source format, that will help debugging the player since I can read the data directly in text format instead of having a hex editor open.
   a) Might I suggest here that you add an option to export values as hex as well?
   b) Also, on ST assemblers instead of db and dw it's customary to use dc.b and dc.w, so something else to consider there
2. An issue that's important: the exporter seems to require that the base address of the tune is a hardcoded absolute address, user selectable. While porting the player I also got vibes from this: the code would load some values from the song data and use them directly. Depending on the application on ST, people could need either absolute addresses (i.e. demos that take over the system) or a position independent, to be determined during runtime.
   This opens up multiple trains of thought:
   a) First of all for the ST, people could simply export at base address 0, so all addresses can simply be offsets as well. That way we can calculate the actual address the tune is loaded using tune_start_address+loaded address. Of course this might take a few more cycles. It might probably not matter much but it never can hurt to think about performance.
   b) Another thing to consider here is that the ST deals with 24-bit addresses, which more or less implies the above. A solution here would be to convert all dw addresses here to dl (or dc.l in assembler terms) so instead of loading offset+adding base address, simply load the absolute address as a 32bit value
   c) Directly related to the above: If we align the tunes to 64k blocks then we can simply force the tune to be loaded like that during startup. Then we can pre-load the upper 16 bits of our address register and then simply load the "offset" in the lower 16 bits, so we have the address by utilising a faster load from ram.
   d) Another trick that could be done here would be to relocate the offsets on runtime during init, sort of parse the tune and fixup all offsets (of course this might have some difficulty if - for example - the tune is loaded between two 64k blocks)
3. Of course from the above I can think of another issue: the tunes are restricted to "only" 64k. How easy or difficult is it to hit that hard limit would you say? Do you need hours of tune data till this is reached? Or (for example) 10? Would it make sense to consider raising that limit to more than that?

That's all for now. I would love some comments from Targhan, but also from other ST programmers that could potentially use this player in their productions. I don't consider any of the above as a hard or impossible problem, but the answers from this thread will shape the player's code - so it's better to discuss this before any major work is done.

[Targhan]

Wow, great! Please find the AKY format attached. It should have been spread within the package, it will be on the next version.

Let me answer your questions now:

Might I suggest here that you add an option to export values as hex as well?

Why not, I'll add it to the TODO list.

Also, on ST assemblers instead of db and dw it's customary to use dc.b and dc.w, so something else to consider there

Already done! Go to the Settings > Source Profile. Either use the "68000 profile", or create your own. Don't forget the command line tools too.

the exporter seems to require that the base address of the tune is a hardcoded absolute address, user selectable.

If you export as a source file, you don't have to encode an "ORG" to set the encoding address, so the music is can be located anywhere, at compile time. However, all the encoded addresses are absolute, because on a CPC, and probably on all the 8-bit computers, using relative address and having to add a base address is much slower, so this is not something I will do (even though you could have an ORG at 0, and somehow "parse" the data and add them an offset.
On possibility could be to generate a relocation table, that is, a list of pointers to all the addresses that needs to be added an offset, for the song to be relocated.

Did I answer your questions?

[Targhan]

Oh, forgot the last question:

Of course from the above I can think of another issue: the tunes are restricted to "only" 64k. How easy or difficult is it to hit that hard limit would you say? Do you need hours of tune data till this is reached? Or (for example) 10? Would it make sense to consider raising that limit to more than that?

On an 8-bit we won't go beyond 64kb of course. But really, I don't believe a song could reach 64kb. I already encoded an 8 minutes song within 20kb, and it was a pretty complex song. I think we are safe within this limit. If needed, breaking it shouldn't be too hard, we could simply encode 24/32 bits addressing instead of 16. Using the Source Profile, you could use a 24/32 bit mnemonic instead of a 16 one. But it won't work, because some values are encoded as several 8-bit values on a row . The encoder needs to know on how many bytes to encode the addresses. But it can be done.

[ggn]

Yup, thanks for the replies. I was mostly concerned about song length and limits, how much you can cram into 64k. I'll forge ahead with a 64k aligned buffer (because that's the easiest - adding support for non aligned buffers is mostly trivial and will take a slight more amount of cpu time). I just aim to raise awareness for programmers that might be interested in using the player, better clearing some things out early before it's too late .

[ggn]

So for anyone that cares: I just finished a 2nd pass at the source code, tidying up things and taking care of loose ends. Which in practice means:

a) It's now ready for debugging (so far I've been writing code at blind, so tons of bugs expected!)
b) It is most certainly not optimal! For now it's simply mirroring the behaviour of the z80 routine in order to have a reference routine up and running
c) After (a) is done then the player can be reworked into something really optimal. At least that's my guess! Running the code under a debugger will also have the benefit of identifying lots of unneeded code and also enable doing things in a more 68000 friendly way

At least one musician apart from dma-sc expressed interest in it so hopefully this is not an exercise in futility (or at the very worst case the sndh archive will have some new tunes )