Support in Steem - teaser

Moderators: DrCoolZic, Moderator Team

User avatar
npomarede
Atari God
Atari God
Posts: 1116
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: Support in Steem - teaser

Postby npomarede » Sun Apr 05, 2015 9:00 pm

Hi
<boring legal>
there was some talks lately with MAME regarding its license. From what I read, redistribution and modifications can only be made if you include the *whole* source tree (which is very big). And even so, I'm not sure the MAME license is compatible with GPL and allows to include MAME code in some other projects.
This is certainly boring stuff, and I don't know if MAME really tries to enforce its trademark, but maybe you'd better write your own decoder from scratch with Jim's help.
</boring legal>
Apart from that, nice work in MFM support.

Nicolas

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1835
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Support in Steem - teaser

Postby Steven Seagal » Sun Apr 05, 2015 9:10 pm

JimDrew wrote:As I said in my PM, you don't need to delay anything to duplicate weakbits with a single rev. During the decoding of the track, you will know if the last MFM bits are an even MFM byte. If not, then by wrapping back around to the start of the track data will automatically introduce a different result every virtual revolution. If the track data does decode to an even MFM byte, you can repeat the first bitcell for either even or odd virtual revolutions. Of course, any bitcell time that is outside of the normal clocking window (<3.5us or >8.5us) is a weakbit and you can take the next bitcell value (if also invalid), add them together, and then divide by a random divider and use both of those values for decoding. This is exactly how a real data separator behaves when an invalid bitcell time is encountered.


Maybe I don't get it right. Aren't weak bits possible because of some sort of drive jitter, that slightly changes delays from rev to rev? And doesn't a sync mark like in Dungeon Master sector 0-7 realign the controller on the right bit?
In the current dev build, the 1-rev version of DM doesn't work anymore, but the 4-rev version works, even if we use only rev 1.
The 1-rev version decodes to $60, $70 or some garbage instead of $68, $E8. I don't say the image is wrong, might be Steem.

NFA over index is easy with .scp images. You will see a NFA at the start of the track and at the end of the track. So, when a NFA is encountered at the end of the track, the start has to be checked to see if the NFA continues. You only need a single revolution here too.


For Turrican you must read a correct data sector over the IP, I haven't checked "NFA" yet. If we wrap 1 rev or go from rev 2 to rev 1 the game fails to load. Because of this we must always start reading data on rev 1.

These methods are used with the Amiga emulators, and because of this 100% of every type of copy protection can be loaded without any "hacks" required.

Nice job. Can't wait to do more testing!


Is it because the Amiga works with tracks and has no tricky WD1772?
Some debugging and I'll release another beta.

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1835
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Support in Steem - teaser

Postby Steven Seagal » Sun Apr 05, 2015 9:36 pm

npomarede wrote:Hi
<boring legal>
there was some talks lately with MAME regarding its license. From what I read, redistribution and modifications can only be made if you include the *whole* source tree (which is very big). And even so, I'm not sure the MAME license is compatible with GPL and allows to include MAME code in some other projects.
This is certainly boring stuff, and I don't know if MAME really tries to enforce its trademark, but maybe you'd better write your own decoder from scratch with Jim's help.
</boring legal>
Apart from that, nice work in MFM support.

Nicolas


Yes, and when you play a game using MAME you must certify that you legally own the original rom. Which we all do, of course. I have thousands of roms in the attic.
You're right, I'll write my own decoder. It will look a bit like MAME's, but with different variable names.
And so there's no reason to thank MAME in the release notes.

User avatar
npomarede
Atari God
Atari God
Posts: 1116
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: Support in Steem - teaser

Postby npomarede » Sun Apr 05, 2015 9:45 pm

Steven Seagal wrote:Yes, and when you play a game using MAME you must certify that you legally own the original rom. Which we all do, of course. I have thousands of roms in the attic.
You're right, I'll write my own decoder. It will look a bit like MAME's, but with different variable names.
And so there's no reason to thank MAME in the release notes.

ROM's owners are most likely not here anymore to claim their rights, but emulation word is a small word, and maybe MAME/MESS authors are also looking at ST emulation projects sometimes and could argue about reusing their code. Note that maybe they could accept it, it's always possible to dual license some code, so you could try contacting them to reuse the MFM part of MAME.

As for renaming variables and keeping the same code structure, honestly it's really easy to tell it's the same, that would not stand as a valid defence :)

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: Support in Steem - teaser

Postby JimDrew » Mon Apr 06, 2015 1:03 am

Steven Seagal wrote:Maybe I don't get it right. Aren't weak bits possible because of some sort of drive jitter, that slightly changes delays from rev to rev?


No, that is a myth that came from those that don't understand how the drive's magnetic pickup system works. When a magnetic flux transition is shorter than 1/2 of the smallest value (4us in our case with 3.5" DSDD drives) or 50% larger than the largest value, then the current clock (which is not not available as reference) is used as the bitcell time that is passed on the RDData line. This process repeats as long as the magnetic flux is not valid. When the magnetic flux is valid, the clock is reset. The result is that if you take a weakbit area and add all of the times together, it will equal the same time that normal valid bitcells would be. Because the internal clock runs and is used as a reference, the value will be different for each revolution when an invalid value is found. This is why the data changes on each pass.


Steven Seagal wrote:And doesn't a sync mark like in Dungeon Master sector 0-7 realign the controller on the right bit?


Yes, but other incoming invalid bits will cause the alignment to change.

Steven Seagal wrote:In the current dev build, the 1-rev version of DM doesn't work anymore, but the 4-rev version works, even if we use only rev 1.
The 1-rev version decodes to $60, $70 or some garbage instead of $68, $E8. I don't say the image is wrong, might be Steem.


Yeah, that is odd! :)
I am the flux ninja

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1835
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Support in Steem - teaser

Postby Steven Seagal » Mon Apr 06, 2015 9:55 am

JimDrew wrote:
No, that is a myth that came from those that don't understand how the drive's magnetic pickup system works. When a magnetic flux transition is shorter than 1/2 of the smallest value (4us in our case with 3.5" DSDD drives) or 50% larger than the largest value, then the current clock (which is not not available as reference) is used as the bitcell time that is passed on the RDData line. This process repeats as long as the magnetic flux is not valid. When the magnetic flux is valid, the clock is reset. The result is that if you take a weakbit area and add all of the times together, it will equal the same time that normal valid bitcells would be. Because the internal clock runs and is used as a reference, the value will be different for each revolution when an invalid value is found. This is why the data changes on each pass.



I sure don't understand this, will need some time.

Here's the doc by DrCoolZic perpetuating the myth about weak bits:

Atari Floppy Disk Copy Protection
By Jean Louis-Guérin (DrCoolZic) Revision 1.3a – November 2014


(...)

4.3 WD1772 Detection of Border Bits
With the above information it is now easy to understand that if a bit reversals happens close to the border of an inspection window (also called Ambiguous area) it will be detected into the first or the next inspection window based on small variation of the drive rotation speed between two read-sector commands and this will therefore result in pseudo random values returned (fuzzy bits).
For example having a reversal 5μs apart from the previous one can be interpreted as a reversal after 4μs or a reversal after 6μs based on small frequency fluctuation of the rotation speed between two reads. One might argue that it is not possible to make sure that these “marginal reversals” will be positioned correctly due to the fact that the rotation’s speeds of different drives are somewhat different and therefore precise reversals timing on a floppy diskette cannot be guaranteed. But in practice this is where the frequency and phase correction of the WD1772 DPLL comes into play. As explained above the inspection window will have it size (i.e. frequency) and position corrected based on the input reversals stream after reception of only a few reversals. Therefore the DPLL of the FDC automatically adjust the frequency of inspection windows for any acceptable (about 10%) variation of drive speed and adjust the phase so that a “normal reversal” will be perfectly in the middle of the inspection window and a “marginal reversal” will be perfectly at the border of the inspection window.
This also explains why, in most cases, “fuzzy bits” are used in “compensating pair”: for every two subsequent fuzzy bits the first reversal is placed at one extreme (e.g. at the beginning) of the inspection window and the “compensating reversals” of the next fuzzy bit at the other extreme (e.g. at the end) of the inspection window. By using this kind of “compensating bits” we guarantee that the frequency and the phase of the inspection windows are (almost) not affected.


User avatar
Steven Seagal
Atari God
Atari God
Posts: 1835
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Support in Steem - teaser

Postby Steven Seagal » Mon Apr 06, 2015 11:01 am

Image
Image

One version fails, one works, like Dungeon Master. Again it might be a problem in Steem.
EDIT: it was, see later posts.

BTW I fixed the sound issue, embarrassing that it was in the demo featured with v3.7.0 :oops:

Image
Image
Image

I have much trouble figuring out what exactly happens with $C2 and $A1 sync marks, especially when a $C2 is followed by a $A1 with (Jupiter's Masterdrive) or without (Albedo) 1 bit between both marks, looks like there's only 1 sync?
And is data transfer delayed when next bit will complete the $4489 MFM word? Much confusing.
So it's a hack for study.

Image
Teasin' Teramis
Here's it's an improvement hopefully. $FB DAM is valid only if preceded by 3 $A1, not $C2 $A1 $A1 (TR 75, S247).
EDIT: wrong! there are 3 x $A1 after all
Last edited by Steven Seagal on Fri Apr 10, 2015 8:28 am, edited 1 time in total.

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: Support in Steem - teaser

Postby JimDrew » Mon Apr 06, 2015 8:30 pm

Good news! The *only* program I have found so far that does not load is Power Drift. Now, I will have to check the image by making a disk and testing it on my ST to see if the image has an issue.

Jupiter's Master Drive, GhostBuster II, Obitus, and even KickOff2 (NFA) now work just fine with the latest beta.

I will continue to do testing, but so far Power Drift is the only one that hangs (on the protection check, reading track 0, head 0, sector 0 over and over forever).

Nice job!
I am the flux ninja

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1835
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Support in Steem - teaser

Postby Steven Seagal » Mon Apr 06, 2015 9:31 pm

Good news indeed, better than expected.
Just to clarify, a beta will soon be "released" but Jim got first look as SuperCard author.

User avatar
kodak80
Captain Atari
Captain Atari
Posts: 425
Joined: Sat Nov 09, 2013 12:05 am
Location: Brisbane, Australia
Contact:

Re: Support in Steem - teaser

Postby kodak80 » Mon Apr 06, 2015 10:07 pm

Steven Seagal wrote:Good news indeed, better than expected.
Just to clarify, a beta will soon be "released" but Jim got first look as SuperCard author.


Looks like Jim has helped get things sorted with SCP disk images. I look forward to the next beta release being available.

In my testing, on the older beta, I found that setting an SCP disk to Read-Only stopped it from adding it to disk A. I was able to add the disk image to disk A once I changed the setting back.
Atari 1040 STE | Atari 1040 STFM | Atari 1040 STF | Kryoflux & Supercard Pro Flux boards
Admin of Atari ST Review Magazine Archive: http://www.ataristreview.com

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: Support in Steem - teaser

Postby JimDrew » Mon Apr 06, 2015 11:31 pm

Other than providing some moral support (and a bunch testing), I really didn't do anything. Steven has done a great job with this. Now, I would like to be able to select a compressed file (we know that 7-zip [.7z] compresses .scp images incredibly well). If we could select a compressed file, Steem could decompress it to a tempA or tempB file (depending on the floppy drive) when inserting the virtual disk and leave the compressed file alone. This would greatly reduce the storage requirements. We do this for the Amiga emulators, and it works great!

I made a disk from the Power Drift image file and it works fine in my ST. So, that protection check really is failing under Steem. :(
I am the flux ninja

User avatar
Stefan jL
Atari God
Atari God
Posts: 1223
Joined: Thu May 09, 2002 3:21 pm
Location: Sweden
Contact:

Re: Support in Steem - teaser

Postby Stefan jL » Tue Apr 07, 2015 6:20 am

JimDrew wrote: I am going to make an option to strip additional revolutions in image files so that the size of the files can be reduced dramatically. If I can get Steven to support compressed files, we can get the file size down under 2MB.


With "ZCP" filename extension? :)
Image

User avatar
kodak80
Captain Atari
Captain Atari
Posts: 425
Joined: Sat Nov 09, 2013 12:05 am
Location: Brisbane, Australia
Contact:

Re: Support in Steem - teaser

Postby kodak80 » Tue Apr 07, 2015 6:34 am

ZIP or 7 Zip support would be great with the SCP images in Steem but only if this doesn't slow down the SCP support being added.

Jim, I would love a way to take my existing SCP images which are mostly 5 rev and extract a single rev copy which I could then use with Steem and keep my 5 rev ones as an archive.

I don't think we need a new file extension as these are zip files containing the SCP file and not a new file type.
Atari 1040 STE | Atari 1040 STFM | Atari 1040 STF | Kryoflux & Supercard Pro Flux boards
Admin of Atari ST Review Magazine Archive: http://www.ataristreview.com

User avatar
Stefan jL
Atari God
Atari God
Posts: 1223
Joined: Thu May 09, 2002 3:21 pm
Location: Sweden
Contact:

Re: Support in Steem - teaser

Postby Stefan jL » Tue Apr 07, 2015 6:38 am

I just thought it would be nice to be able associate to for example Steem... just double click the file and Steem starts.
Image

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1835
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Support in Steem - teaser

Postby Steven Seagal » Tue Apr 07, 2015 8:12 am

JimDrew wrote:I made a disk from the Power Drift image file and it works fine in my ST. So, that protection check really is failing under Steem. :(


It's no protection/MFM issue but a simple WD1772 status register issue ("TR00") which I just fixed.

Now disk A loads but disk B seems to be bad? Track 1 is garbage? Doesn't look very good in Aufit...

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1835
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Support in Steem - teaser

Postby Steven Seagal » Tue Apr 07, 2015 8:24 am

Steem (WIN32) supports zip and rar, which is more than what you may say of some other emulators.
7zip has better compression but is slow, which counts for big files like SCP images. Even with rar there's a noticeable delay.
It isn't as straightforward to use as unrar.dll, documentation is poor, and there are compiler issues.
Besides it would add overhead to Steem, there is much code (.obj files) involved apparently. Is Steem an emulator or an archive manager?
So I don't know.

User avatar
exxos
Hardware Guru
Hardware Guru
Posts: 4933
Joined: Fri Mar 28, 2003 8:36 pm
Location: England
Contact:

Re: Support in Steem - teaser

Postby exxos » Tue Apr 07, 2015 8:53 am

I did some tests with RAR, RAR5, and 7zip last year. Really 7zip can compress huge files marginally more than RAR, but as you say it takes a long time. Mostly my archives are in the GB range, but its like 10mins in RAR and 2 hours in 7zip just to gain like 100MB less archive size. On floppy images etc , smaller files isn't what I have tested, but I can't imagine there being overall much difference. 7zip can eat a lot of ram and CPU power so IMHO I wouldn't bother with 7zip and just stick to rar / rar5 and zip.
4MB STFM 1.44 FD- VELOCE+ 020 STE - Falcon 030 CT60 - Atari 2600 - Atari 7800 - Gigafile - SD Floppy Emulator - PeST - various clutter

http://www.exxoshost.co.uk/atari/ All my hardware guides - mods - games - STOS
http://www.exxoshost.co.uk/atari/last/storenew/ - All my hardware mods for sale - Please help support by making a purchase.
http://ataristeven.exxoshost.co.uk/Steem.htm Latest Steem Emulator

User avatar
Stefan jL
Atari God
Atari God
Posts: 1223
Joined: Thu May 09, 2002 3:21 pm
Location: Sweden
Contact:

Re: Support in Steem - teaser

Postby Stefan jL » Tue Apr 07, 2015 11:32 am

Steven Seagal wrote:One version fails, one works, like Dungeon Master. Again it might be a problem in Steem.
People with hardware and good will could confirm this...


If you want me to test the Union demos SCP's on real hardware then post links to them please :)
Image

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: Support in Steem - teaser

Postby JimDrew » Tue Apr 07, 2015 1:35 pm

I will look into making a utility that will strip multiple revolutions into a single revolution, maybe I can do this right now with the existing copier (since it does batch conversion already).

7 zip is far superior for flux images than RAR! Most images end up being 1/2 the size compressed with 7 zip compared to RAR, and decompression with 7 Zip is dramatically faster than RAR. I find compression speed is nearly identical because although 7 zip is slower, it creates smaller files so the time is about the same.

I made a feature in the copier called "perseveration mode". With this checked, images contain the exact flux data. If you turn this off, the data is altered to allow better compression without sacrificing the operation of the disk. With the preservation turned off, I can compress an image that is normally 13MB to under 2MB! We do this with the Amiga emulators, and it works perfectly. Decompression on a file of this size is basically instant.

Steven, you could look at the WinUAE source code. It has code for handling just about every compression type known to mankind.
I am the flux ninja

User avatar
exxos
Hardware Guru
Hardware Guru
Posts: 4933
Joined: Fri Mar 28, 2003 8:36 pm
Location: England
Contact:

Re: Support in Steem - teaser

Postby exxos » Tue Apr 07, 2015 4:28 pm

With some stuff I grabbed off the net..

7z.jpg


7zip can shave about 10% off smaller files, but as soon as it starts getting larger, it gets somewhat pointless. I used to use it all the time to do my backups, until one day I wanted to extract a single file from a 7z archive which took all day as it wanted to extract the whole archive first. I much prefer winrar, its a common no fuss solution IMHO.
You do not have the required permissions to view the files attached to this post.
4MB STFM 1.44 FD- VELOCE+ 020 STE - Falcon 030 CT60 - Atari 2600 - Atari 7800 - Gigafile - SD Floppy Emulator - PeST - various clutter

http://www.exxoshost.co.uk/atari/ All my hardware guides - mods - games - STOS
http://www.exxoshost.co.uk/atari/last/storenew/ - All my hardware mods for sale - Please help support by making a purchase.
http://ataristeven.exxoshost.co.uk/Steem.htm Latest Steem Emulator

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: Support in Steem - teaser

Postby JimDrew » Tue Apr 07, 2015 5:40 pm

I am not sure what version of 7 zip you are using, but I don't get that. You can always extract single files from a .7z archive no different than .zip or .rar archives - instantly. I use Total Commander to interface to all of the various compression routines, reading file systems, etc.

I just tested Wipeout.scp - as a .7z file it is 15,861,670 bytes long, and as a .rar file it is 20,520,188 bytes long.
I am the flux ninja

User avatar
exxos
Hardware Guru
Hardware Guru
Posts: 4933
Joined: Fri Mar 28, 2003 8:36 pm
Location: England
Contact:

Re: Support in Steem - teaser

Postby exxos » Tue Apr 07, 2015 6:04 pm

I downloaded the one on their site V9.2 and use that one. I had not had it installed for about a year as I stopped using it.

Can you send me one of those file you say you can get from 13MB to 2MB ? The only time I saw compression like that was when the files were basically full of zeros.

When I used to use ICE PACKER to compress disczine articles, we used to put loads of spaces on the end of the text files as it actually reduced the file sizes a lot. So I wonder of your 13MB files just have a lot of zeros in or something giving those results ? I did spend a fair bit of time compressing all sorts when I first installed 7zip, but overall it was just like similar results to what I posted. I had it on Ultra settings, if I changed much else it actually made the files larger than the rar ones. Its wanting to use about 700MB RAM as it stands. any more compression options and it wants to use over 2GB RAM which is just getting a bit silly. I actually have 16GB ram in my PC, but 7zip just dies saying out of memory after a few seconds of working :roll:
4MB STFM 1.44 FD- VELOCE+ 020 STE - Falcon 030 CT60 - Atari 2600 - Atari 7800 - Gigafile - SD Floppy Emulator - PeST - various clutter

http://www.exxoshost.co.uk/atari/ All my hardware guides - mods - games - STOS
http://www.exxoshost.co.uk/atari/last/storenew/ - All my hardware mods for sale - Please help support by making a purchase.
http://ataristeven.exxoshost.co.uk/Steem.htm Latest Steem Emulator

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: Support in Steem - teaser

Postby JimDrew » Tue Apr 07, 2015 6:53 pm

The files that compress from 13MB to <2MB are captured using the SCP without the "preservation mode" enabled. This alters the bitcell data so that it can be compressed better. The image still works when writing it to a disk (or in Steem, or other emulations that support .scp images), but it does not contain the exact flux data from the original disk.

I made an .scp image of Power Drift disk A. The file size generated with either the preservation option selected or not is ~13MB. However, compressing the file with the preservation mode off using the "normal" mode under 7-zip generates a file size of 1,230,847 bytes. With the "ultra" mode it generates a file that is only 1,084,757 bytes long!


http://www.cbmstuff.com/downloads/PD.7z

http://www.cbmstuff.com/downloads/PD2.7z
I am the flux ninja

User avatar
exxos
Hardware Guru
Hardware Guru
Posts: 4933
Joined: Fri Mar 28, 2003 8:36 pm
Location: England
Contact:

Re: Support in Steem - teaser

Postby exxos » Tue Apr 07, 2015 7:14 pm

20150407201257.jpg


:shrug:
You do not have the required permissions to view the files attached to this post.
4MB STFM 1.44 FD- VELOCE+ 020 STE - Falcon 030 CT60 - Atari 2600 - Atari 7800 - Gigafile - SD Floppy Emulator - PeST - various clutter

http://www.exxoshost.co.uk/atari/ All my hardware guides - mods - games - STOS
http://www.exxoshost.co.uk/atari/last/storenew/ - All my hardware mods for sale - Please help support by making a purchase.
http://ataristeven.exxoshost.co.uk/Steem.htm Latest Steem Emulator

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: Support in Steem - teaser

Postby JimDrew » Tue Apr 07, 2015 7:39 pm

Weird... when I use RAR I get a file size of 1,378,232 bytes. In any case, you can see that turning off the preservation option really makes the compression quite spectacular.
I am the flux ninja


Social Media

     

Return to “SuperCard Pro Disk Copier”

Who is online

Users browsing this forum: No registered users and 2 guests