SuperCard Pro

Moderators: DrCoolZic, Moderator Team

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

SuperCard Pro

Postby JimDrew » Fri Dec 20, 2013 7:30 pm

First of all, thank you to the Atari-forum owner/admin for this forum area! I hope that it will be informative and helpful for anyone interested in disk copying, disk imaging, and disk protections in general.

A little background on me. It is likely that most people in the Atari world don't know who I am because I never released any commercial products for the Atari. I did know a few people who did (like David Small), and I worked at Central Point Software during the creation of Copy II ST, while I was there writing Copy II 64/128. I spent a couple of decades developing disk copying software and hardware for the CBM computers (PET, VIC-20, C64/128, and Amiga) and last year I decided to get into retro computing and make a modern version of the popular Supercard hardware product line that I first released in 1985.

SuperCard Pro started out as just a hardware device that could read and write flux level transitions. It grew into much more than that before it was finally released. It has a USB port and drivers available for PC, Mac, and Linux boxes, but it can also be a stand alone device controlled by either of its dual serial ports (300 baud through 1 megabaud). So, you don't actually need a computer with a USB port to control it. Even a simple microcontroller can control it. The hardware has a standard 34 pin floppy drive interface port, some status LEDs, 512K of static RAM, a 40 MIPS PIC processor, and a sd-media slot. There is also a 3.5" floppy power connector that lets you power a 3.5" floppy drive directly using a USB port's power. This makes it a compact unit that most people are putting into old CD-ROM drive cases to make a nice portable unit.

The flux capturing has a 25ns resolution, making it nearly twice the resolution of Kryoflux. The floppy bus is fully bi-directional. One of the planned features is making the SuperCard Pro a floppy drive emulator like HxC, using flux image files so that it will emulate a real disk (copy protected disks would load just like a real disk). It will be possible to control the functionality of the floppy drive emulation via the serial port(s), where one could be used for a LCD display/button matrix. One idea I had was a small app that would talk to the board via serial (even bit-banged serial) and be able to swap disks, show track data, etc. This app could be made for the ST, Amiga, PC, etc.

SuperCard Pro can create flux level images (.scp extension). I have made the image file format public information, and I would be happy to work with any developer to integrate this format into an emulator, or any kind of tool. I am particularly interested in adding conversion support for .stx/.stt, and I have collected quite a bit of info from this site... so thank you for that!

So why another disk copier? Well, I wanted something better than what was available for resolution and I didn't want a closed file format. The reality is that 99.99999% of the people want to backup their software that is no longer in production, and they want to be able to use that disk with an emulator. The remainder of people are strictly interested in "preservation". I personally do not care about preservation, although SuperCard Pro provides the best solution for this due to its resolution and functionality. So, my goal is to make something that everyone will want to use due to its simplicity, ease of use, and ease to get information and support for. So far, I believe I am achieving my goal.

I know very little about the Atari ST. I had a 520ST back in 1989 with the Atari color monitor and Starglider - that's my total experience until a few weeks ago when I found a really nice 1040STFM machine with a monochrome monitor and a bunch of commercial midi apps (the machine came from a big recording studio in Canada). I got an Atari color monitor and a few protected games (Dungeon Master & Chaos Strikes Back) and now I am an Atari wiz! LOL! At least the ST is easy to use.

I have made the connection with DrCoolzic, and we have shared some info. We have many different views on copy protections because we have come from different angles at the same problem, which is proving to very informative for me. I have always dealt with data in either flux or MFM/GCR, never as any type of decoded data. So, it is interesting to see the differences in terminology and understanding of protections when looking at things from a non-raw level.

So... the SuperCard Pro board comes with software (written in VB6 actually) that allows you to copy and image a disk. There is also an extremely powerful disk analyzer that lets you read/write individual tracks, decode data as MFM or GCR, and soon will let you decode to the sector level. There are lots of disk formats out there, so this is a work in progress for sure! As it stands, Dungeon Master, Chaos Strikes Back, and the commercial midi programs I have all image and copy perfectly. So, I am not sure if that is some type of milestone or not. I am told that Turrican is nasty, so I purchased that from the U.K. and will test it when it arrives.

If you guys have any suggestions for this product, I am all ears. I really like the public to shape the direction and features of everything I do. Sometimes that can be dangerous though, with often times wild requests... but I am happy to hear any ideas and will do what I can to implement ideas that will best benefit everyone.

I have a website with an online store and our own forum area as well: www.cbmstuff.com


Thanks for listening!

Jim Drew


P.S. Special thanks goes out to DrCoolzic for helping make this forum possible!
I am the flux ninja

Dal
Administrator
Administrator
Posts: 4049
Joined: Tue Jan 18, 2011 12:31 am
Location: Cheltenham, UK
Contact:

Re: SuperCard Pro

Postby Dal » Fri Dec 20, 2013 9:01 pm

Nice first post - Welcome aboard and thanks for looking to bring your product to the Atari.
TT030: 4MB/16MB + Crazy Dots, Mega"SST" 12, STacy 2, MegaSTE, STE: Desktopper case, IDE interface, UltraSatan (8GB + 512Mb) + HXC floppy emulator. Plus some STE's/STFM's

kristjanga
Captain Atari
Captain Atari
Posts: 400
Joined: Sat Jul 25, 2009 3:35 pm

Re: SuperCard Pro

Postby kristjanga » Fri Dec 20, 2013 9:09 pm

perhaps this could make a working copy of Crown Of Creation 3D?

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: SuperCard Pro

Postby JimDrew » Fri Dec 20, 2013 10:34 pm

I will go out on a limb and say yes. When I was selling Supercard Ami for the Amiga, we sold a LOT of them to the Atari ST clubs because they used an Amiga 500 with that product to copy everything for the ST. There has never been an Amiga or ST disk I have heard about that did not copy with that device... but there were tens of thousands of disks released, so anything is possible.

Where can I get an original of Crown of Creation 3D? Is this program known to be particularly difficult (or impossible) to copy? Like I said, I know very little about the ST, so please bare with me while I get up to speed. If you guys know of any programs that are "impossible" to copy, please let me know the names and I will try to buy them (usually from eBay). I can also arrange for any disks sent to me to be returned with a backup and SuperCard Pro image file.

Thanks!
I am the flux ninja

kristjanga
Captain Atari
Captain Atari
Posts: 400
Joined: Sat Jul 25, 2009 3:35 pm

Re: SuperCard Pro

Postby kristjanga » Sun Dec 22, 2013 12:51 am

JimDrew wrote:I will go out on a limb and say yes. When I was selling Supercard Ami for the Amiga, we sold a LOT of them to the Atari ST clubs because they used an Amiga 500 with that product to copy everything for the ST. There has never been an Amiga or ST disk I have heard about that did not copy with that device... but there were tens of thousands of disks released, so anything is possible.

Where can I get an original of Crown of Creation 3D? Is this program known to be particularly difficult (or impossible) to copy? Like I said, I know very little about the ST, so please bare with me while I get up to speed. If you guys know of any programs that are "impossible" to copy, please let me know the names and I will try to buy them (usually from eBay). I can also arrange for any disks sent to me to be returned with a backup and SuperCard Pro image file.

Thanks!

some one here must have a original disks
this is a game for the Atari Falcon 030
this game has some nasty copy protection that has never been cracked, and has been impossible to copy
the only way I can think of that would copy those disks (which are two I think) is to make a raw image of the disks and then writing that back to another disks
If we could get those disks and if that would work this game would be saved from extinction, as my guess is that floppy disks must get damaged sooner than later

you can make a raw image with supercard?

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: SuperCard Pro

Postby JimDrew » Sun Dec 22, 2013 2:45 am

Yes, SuperCard Pro generates flux level images. You can dump a disk to an image and then recreate an identical disk to the original using the flux data.

If someone has originals of anything they want duplicated and imaged, I would be happy to do it!
I am the flux ninja

User avatar
Brume
Red eyes
Red eyes
Posts: 4044
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: SuperCard Pro

Postby Brume » Sun Dec 22, 2013 10:01 am

Congratulation and welcome Jim Drew :D This card looks very promising!

What kind of file will it generate? Does it need to be converted to write back the flux (like raw -> IPF with Kryoflux card)?

Also, does it can produce .ST or .MSA images for unprotected disks?

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: SuperCard Pro

Postby JimDrew » Sun Dec 22, 2013 6:56 pm

SuperCard Pro generates .scp (flux level) images. The image file format is public info (posted in my developer area on the cbmstuff.com forum). You can go from disk to .scp or .scp to disk, there is no middle step. You can also go from .scp to (insert image format here). I am working on a converter routine that will convert .scp to .stx and .stt. I guess I can write a converter routine also for .ST images for unprotected disks. .stx/.stt format will also work for unprotected disks as well. Once the .scp file has been generated with the hardware, anyone can use the software (no hardware required) to convert from .scp to whatever image file format the image was created for. The .scp files are the generic flux data, with information about the disk type imbedded in the header, so you can't accidentally do things like convert a C64 flux image to a .stx image.
I am the flux ninja

User avatar
Stefan jL
Atari God
Atari God
Posts: 1178
Joined: Thu May 09, 2002 3:21 pm
Location: Sweden
Contact:

Re: SuperCard Pro

Postby Stefan jL » Tue Dec 24, 2013 7:47 pm

I was just about to buy an KryoFlyx but now i think a SuperCard Pro is better to have :)
Image

User avatar
Brume
Red eyes
Red eyes
Posts: 4044
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: SuperCard Pro

Postby Brume » Mon Dec 30, 2013 11:07 pm

Have ordered one :)

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: SuperCard Pro

Postby DrCoolZic » Tue Dec 31, 2013 9:42 am

KryoFlux is a good product but unfortunately not very useful for average Atari users. The reason is that currently you generate raw stream file that you send to SPS people and that's it. You have to wait for IPF files that never come back! As I mentioned I have sent images of about 100 FD and never got anything back after two years.
After experimenting (for only one day!) with SuperCard Pro I have been able to make copy of Dungeon Master immediately :)
And with my program I can see exactly the flux transitions and analyze and the results are more 'crisp' (precise) thanwith KF
Jim is currently writing a converter to stx (pasti) format that will make the dump usable in Steem immediately.
I am also working on an STX writer.

So yes it seems that for Atari users SuperCard Pro is much more useful.

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2595
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: SuperCard Pro

Postby AtariZoll » Tue Dec 31, 2013 9:49 am

Great news, and I hope that this will move currently little frozen Atari SW on floppies preservation ahead. From some reason Kryoflux project for Atari ST works not well. May be that they just don't like approach of majority here - about legal issues and sharing.
I don't plan to buy it from simple reason: don't have any rare SW, what is not already imaged in good Pasti images. But I can make list, and actually I can send via post some my floppy disks to Jim Drew, without charge and need to return them, because they just stay here and 'collect dust' :D - if that will help development.
I like idea of converting to popular formats. Converting to STT is not much interesting, I think. That format is not much used. Instead it, convert to STX. Of course, that will be not easy, knowing limited informations about that format. (I'm one of who dealt with it http://atari.8bitchip.info/STXdesc.html ) . And yes, we need conversion to ST format, and myself need it not only for unprotected titles - for doing hard disk runnable fixes from original.
What would be useful is conversion to format used by HxC floppy emulator. But I guess that better idea is to use SuperCard Pro self in that purpose.
Btw. , what is usual scp image size of some DD DS (800K) floppy ? According to resolution: some 50MB ?
I'm not against GMO, I'm against that children play with fire.

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: SuperCard Pro

Postby DrCoolZic » Tue Dec 31, 2013 11:01 am

Hello Peter
Yes you have provided a lot of useful information on the STX format along with Markus. I took it fromthere and had some mail exchange with Ijor to complete the description see viewtopic.php?f=47&t=19904#p242579

Most of the information aaprt from the timing information was already described by you or Markus I just made a nice doc that is "almost" blessed by Ijor :)

I have not done yet much images but the size is smaller than I thought. For Dungeon Master it is around 10MB (only one revolution)
For Fire and Forget (3 revolutions) about 28MB
So this is about 10MB per revolution

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: SuperCard Pro

Postby JimDrew » Tue Dec 31, 2013 8:26 pm

That is actually about 5MB per side. Both sides of the disk are always imaged, even if the disk is a single sided type.... but the size does vary quite a bit, all depending on the contents.
I am the flux ninja

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2595
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: SuperCard Pro

Postby AtariZoll » Wed Jan 01, 2014 9:10 am

Happy New Year :D

Great work, as usual DrCoolZic .
I was thinking little about converting raw scp images to popular formats. IPF is certainly better than STX, and there is support now in Steem SSE, while people working on Hatari support too. In case of STX we have support in Steem and Saint, and as I know Hatari support is under work too.
I'm not against STX, but it appeared that it emulates not correctly floppy rotation - see threads about MicroProse Golf case - where it fails in emulators because too fast data load with STX image. With IPF it was better. I don't know how well IPF format is documented, and is it able to support all Atari copy protections. Additionally, it would be nice if conversion to some compacter format would allow writing to floppies - and STX is not such format.
In any case, conversion to STX may be compact too - producing images only little larger than ST images, because protection is usually only in 1-5 tracks. For instance much used Copylock 1990 is always on track 0 of side A. So, it is enough to have track dump only of that track, with extra infos (timing in first place). Other tracks can be imaged as unprotected ones. Actually, I converted (shrinked) many STX images in that way. And they are perfectly usable. When you know where is protection ... and DC STX images are mostly such.

As there are plans to use SuperCard Pro (variant) as HW floppy emulator - I think that compact format should serve it well .

One thing more: there is no any checksum/CRC in STX images, and it should be for all track records. Likely, it is possible to add that extra info (4 bytes) without problems at track record end.
I'm not against GMO, I'm against that children play with fire.

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: SuperCard Pro

Postby DrCoolZic » Wed Jan 01, 2014 10:43 am

Yes happy New Year to all 8)

I know that you do not like too much the STX format! :oops:
This format is far from being perfect for historical reasons (new protections discovered after initial releases) but it has the advantage of being able to store information for all known protections I can think of. We could redesign a cleaner format but what for? There are thousands and thousands of games already saved using this format. I know that Nicolas is working on providing support in Hatari so this format will be supported by all major Atari emulators.

IPF is a strange “beast”. The format is design to be able to support any kind of protections but in fact the protections are not “defined” per se in the file format but in the accompanying library. So for example you will not find any timing information in the file but this timing information is coded in the library. The idea behind IPF is that there is a limited number of protections used by reproduction machines (like the Trace machine) and you just need to find out and describe all these protections. The problem is that when you want to add a new protection like the Dungeon Master weak bits you have to modify the IPF library to handle this new protection. Creating IPF file is tough because you need to know all the protections supported by the current lib and with this information you have to select the right one based on what you detect in the source… But the advantage is that you should get a “perfect image”.

The IPF format is described in http://info-coach.fr/atari/software/pro ... tation.pdf and information about the IPF/CAPS library can be found in http://info-coach.fr/atari/software/projects/IPF.php

STX has never been designed to be able to write FD but only for emulation. Product like SCP or the old Discovery Cartridge are designed for that. Also it is not clear that in future you want to create Floppy Disks? If SCP provides floppy drive emulation in future (like HxC) it seems a better route?

I am not convinced that adding CRC/Checksum is very useful. This idea come from the past when files where transmitted over modems… in the last decades I have never seen a file to get corrupted? I do not know if the Pasti library would care if extra bytes were added at the end of the file but it is easy to experiment.

Can you summarize the problem with Microprose Golf? What do you mean by STX does not emulate correctly floppy rotation? Is it a problem with the pasti library or with the STX format.
I need to understand why some programs can’t be imaged with Pasti: is it a problem with the imager or a problem with the format?

User avatar
npomarede
Atari God
Atari God
Posts: 1114
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: SuperCard Pro

Postby npomarede » Wed Jan 01, 2014 2:28 pm

DrCoolZic wrote:Yes happy New Year to all 8)

I know that you do not like too much the STX format! :oops:
This format is far from being perfect for historical reasons (new protections discovered after initial releases) but it has the advantage of being able to store information for all known protections I can think of. We could redesign a cleaner format but what for? There are thousands and thousands of games already saved using this format. I know that Nicolas is working on providing support in Hatari so this format will be supported by all major Atari emulators.

IPF is a strange “beast”. The format is design to be able to support any kind of protections but in fact the protections are not “defined” per se in the file format but in the accompanying library. So for example you will not find any timing information in the file but this timing information is coded in the library. The idea behind IPF is that there is a limited number of protections used by reproduction machines (like the Trace machine) and you just need to find out and describe all these protections. The problem is that when you want to add a new protection like the Dungeon Master weak bits you have to modify the IPF library to handle this new protection. Creating IPF file is tough because you need to know all the protections supported by the current lib and with this information you have to select the right one based on what you detect in the source… But the advantage is that you should get a “perfect image”.

The IPF format is described in http://info-coach.fr/atari/software/pro ... tation.pdf and information about the IPF/CAPS library can be found in http://info-coach.fr/atari/software/projects/IPF.php

STX has never been designed to be able to write FD but only for emulation. Product like SCP or the old Discovery Cartridge are designed for that. Also it is not clear that in future you want to create Floppy Disks? If SCP provides floppy drive emulation in future (like HxC) it seems a better route?

I am not convinced that adding CRC/Checksum is very useful. This idea come from the past when files where transmitted over modems… in the last decades I have never seen a file to get corrupted? I do not know if the Pasti library would care if extra bytes were added at the end of the file but it is easy to experiment.

Can you summarize the problem with Microprose Golf? What do you mean by STX does not emulate correctly floppy rotation? Is it a problem with the pasti library or with the STX format.
I need to understand why some programs can’t be imaged with Pasti: is it a problem with the imager or a problem with the format?


Hello, as we had a look with AtariZoll to microprose golf and why it didn't work with pasti or Hatari at that time, it's an error in the pasti dll, not in the stx format per se (as Hatari can run it in ST format now, it's not a problem of imaging format, but of timing when pasti emulates the WD1772).

In any case, for emulating any disks, several parts are needed :
1) an image format that can store all clock/data info, preferably in a compact/optimized way when the track or sector is detected as being "standard"
2) a part that will take the image format and return each byte and their duration as you would have got them when doing read sector, read track or read address fro example.
3) a part that emulates the mechanical part of the drive : seek, restore, rotation speed, ...

Pasti handles all 3 points ; as seen on MP Golf, part 3 is not always correct, as the game doesn't work in pasti mode. If we consider we want a format that is open and extensible enough, to handle any kind of disk layout we could encounter later, then I think STX is also limited when it comes to fuzzy bits and variable length bytes :
- fuzzy bits are handled with a mask and a random ; but is this really the case ? Maybe most protections only test that the same sector return different bytes, but a more careful analysis could show there's a repeatable pattern in those "random" bits, in which case STX is not good enough.
- variable length byte : this is only handled on a per sector basis in STX format. But nothing would prevent a disk to be recorded with variable lenght bytes in the gaps between sector, and to test this by doing a read track command and measuring at which pace bytes are coming from the WD1772.

My feeling about IPF, is that this format requires special tools to convert the raw stream in a more compact/optimized version that will often take ~800 KB per disk. Without such tool, it can be hard to decide if a sector should be stored as variable length byte or not, for example. The capslibray also includes a very good emulation of the WD1772 for part 2) and 3). From reading the libray code, I don't have the feeling that protection scheme are handled at the library level, but are stored in the IPF.
In the case of Dungeon Master, maybe IPF should be extended a little, with some code to handle it in capslibrary, but apart from that it seems much more versatile than STX to me.

There're also internal formats used by HxC or MESS, but I don't know them that much.

In case we want to create a new format (not sure of that) instead of extending current ones, that would only fix point 1) . In the case of WD1772 emulation, I think a set of library functions are also needed to get a higher level of abstraction and directly simulates reaching the next AM, returning the content of the sector, the content of a track, ... Leaving all those functions to the emulator would create duplicate work by different emulators' author (for example, this is what I will have to do to handle STX in Hatari : part 3) is already there but part 2) should be completely written from scratch). That's why I think the capslibrary used with IPF also provides a very solid basis ; I already discussed a few errors/limitations in it with SPS, so it can be fixed/improved either by them or by anyone, since the code is available.
On the contrary, there's no source version of Pasti, so at the moment, should you want to have pasti support for non windows OS, then you're stuck, or need to write it from scratch, as I intend to do for Hatari (and could be considered as wasting some time, trying to guess how all internal pasti cases are handled)

So, when it comes to converting a raw stream (made with kryoflux or supercard), I think IPF has many advantages. What we could need IMO is a tool to create those IPF files from ST/MSA/kryoflux stream/supercard stream.

Nicolas

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2595
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: SuperCard Pro

Postby AtariZoll » Wed Jan 01, 2014 2:40 pm

I think that STX (Pasti) is great achievement. Best thing is that can make images of copy-protected floppies without special equipment. Such images are actually not real floppy images, but mostly informations for emulators. This is why format is not good for writing on floppies.
Another problem is that in some cases you can not know is it damaged floppy data or protection - I guess that in case of some stream image it is well visible in most cases if damaged (aged floppy) data is somewhere.
Surely that it is best supported among Atari ST emulators, so conversion to it is certainly welcome. But as said, if aim is writing onto floppies + HW floppy emulation maybe is better to go on something other, what prevents not making SW for conversion to STX. Just someone needs to do it :D

Checsksum could be placed at end of every track record - I just made quick test by adding 4 to track record length at start, and adding 4 bytes at end - and it works fine in Steem. Now people mostly stores images in (ZIP) archives, what is probably main reason for less data corruption. In past there was more problem, and floppy transfer was weak point.

The problems with Pasti images, which I know about:
It can be unsupported protection or some error by imaging - but I don't know details. There are some threads here talking about - Ijor self said for one game that can not be imaged with Pasti, some race/driving one, on 3 floppies, if remember correct.
There are problems with emulation, and not images being bad, in some cases. And it can be bad ST HW emulation (mostly CPU) or not perfect floppy emulation in Pasti.dll .
For instance Warp has some tricky code at start (with purpose to make cracking harder, of course), and it fails because bus/address error stackframe writing is not 100% properly emulated in most of emulators. As I know, only Steem SSE emulates it well. There are 2 Titus games with tricky CPU code too: Knight Force and Crazy Cars II .
Case where Pasti floppy emulation is not perfect is game Rubicon: it has actually buggy floppy reading code, where both floppy drives are selected at once. On real HW with 2 drives it harms not, if second drive is empty. And with Steem's floppy code works well too. But if using Pasti (so in case of STX image) game will not work. I informed Ijor about it some 16 months ago. Possible that only mentioned 1 title is involved. But there are many games with really silly bugs, so who knows what we will see yet ...
Microprose Golf has pretty dumb code in intro: it reads further data while playing audio sample. Normally loading takes more time than audio playback, so will finish it before all data is loaded. The problem is that there is no any check about loading, and it may overwrite audio playback code. So, if reading of floppy is faster, it will cause crash, as it happened in Steem, Hatari, with Pasti or without it. Even in max accuracy mode Pasti reads floppy too fast.

I hope that above may be interesting to Jim Drew too - may see with what silly problems can encounter :D
I'm not against GMO, I'm against that children play with fire.

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: SuperCard Pro

Postby JimDrew » Wed Jan 01, 2014 4:44 pm

The problem with .STX, .IPF, etc. is that they interpret the contents of the disk, and so you are limited only to what the libraries 'know', which is actually very little compared to the number of different types of copy protections released. If you want full 100% compatible read/write, the easiest method is certainly flux level. Yes, the files are large, but you can trim most everything to a single revolution per track, and really we don't care about file size anymore - maybe 10 years ago we did, but hard drive space and RAM is not an issue anymore. You could always keep the images compressed and decompress them when the disk is 'mounted'. If you make each track padded to be the largest size possible (something that SCP will be able to do during image creation), then you have the ability to write track data back too. In the case of the WD1772 based disks, padding is probably not necessary since tracks are preformatted and sector headers are found and new data written immediately after, so the track can't really 'grow'. There are many disk formats that write the entire track when sectors are changed, so these need the ability to change track length.

The FPGA Arcade and a few other emulation projects are implementing .scp flux image files. The data is just clocked in like the real floppy. The WD1772 emulation would just fetch the flux transitions and convert them to clock and data bits. This would let you have 100% compatibility with any type of copy protection, now and in the future - no learning required. I can't imagine it be that much work to change your existing WD1772 emulations to support flux directly. There is already a bunch of work done to parse through files, build tables, etc. With the .scp file format, you just need a track number to look up the offset to the track data header, which gives you info about the rotation speed, track length, and offset to the flux. From there you either copy the entire track to a circular buffer or fetch the track data from the file directly.
Last edited by JimDrew on Wed Jan 01, 2014 5:01 pm, edited 1 time in total.
I am the flux ninja

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: SuperCard Pro

Postby DrCoolZic » Wed Jan 01, 2014 4:48 pm

I am not sure I understand the above comments. I am not particularly a defender of the Pasti format, I am just saying that it exists, it is used extensively, and it does the job. It has some ugliness but I can’t think of a protection that cannot be described with the format. I am not talking about the emulation library but just the format. When I started to documents Pasti I found many things I did not like and at that time I have been thinking of defining a new clean format. But I quickly stopped because the question is for what reasons? Just to make it nicer? To handle protections not handled currently?

I have nothing against the IPF format (not officially documented other than my doc). The sources of the lib have been published and could be modified to better fit Atari requirements and/or protections but it is and it must stay under the control of SPS because the CAPS/IPF library is the heart of their complete solution. SPS have limited resources and Atari is not a priority for them.

You are right when you say that the protections are stored in the file. But the interpretation of the protection is usually done in the library. For example if you have a sector with shorter bit cells you do not store any timing information in the file to indicate this (as you would do in most emulation format), but you will have a “type” telling the library that this sector uses a protection with shorter bit cell and that different timing should be returned …

Now let say that you discover a new protection like the macrodos protection that uses four segments with different cell timing. You will need to define a new type to store in the file but more importantly you will need to write the code in the library that will return the right timing values for this new protection and this can ONLY be done by SPS.

Writing IPF is difficult as it requires that you know the protection used and that you know how to describe this protection with IPF. SPS has a program called CTA to do this and they compare this operation “to breaking encryption due to the huge number of formats to validate against and the fact that due to the way MFM encoding works data must be shifted on the flux level to align correctly”. Even with this program you need trained people to be able to generate IPF files.

I have no problem if someone want to come with a new and better format for Atari protected disk and want to provide routines for emulation.

Meanwhile I hope to be able to write (and read) Pasti format with my tool very quickly. Supporting partially IPF is feasible (for example for non protected disk) but not high priority.

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: SuperCard Pro

Postby JimDrew » Wed Jan 01, 2014 4:56 pm

Speed issues are not surprising. When I created the first color Mac emulation, I ran into this a lot with Apple's stuff. It's amazing just how many race conditions existed in OS7.x/OS8.x. Audio playback vs. CD-ROM data was a nightmare, and my solution was to emulate the hardware exactly. Just as emulating a WD1772 exactly and fetching real flux transitions would solve any type of speed issue. If I clocked out the flux data generated with SCP to a real WD1772 it would think there is a floppy drive connected. :) In fact, that is what I am doing with the floppy drive emulator portion of SCP... using the flux images to spool out the data. So, SCP's floppy drive emulation is flux based, not .stx/.stt/.dsk/.hfe/.adf/etc. It uses the same exact data as a real floppy drive. One of the main reasons why there is 512K of static RAM on the board is for buffering entire tracks at a flux level while reading/writing the sd-media card. I will probably support other formats later on (by converting them to flux), but not initially. I still have a ways to go with that portion, primarily because I am working on the copier for many different disk formats.
I am the flux ninja

User avatar
npomarede
Atari God
Atari God
Posts: 1114
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: SuperCard Pro

Postby npomarede » Wed Jan 01, 2014 5:06 pm

JimDrew wrote:The problem with .STX, .IPF, etc. is that they interpret the contents of the disk, and so you are limited only to what the libraries 'know', which is actually very little compared to the number of different types of copy protections released. If you want full 100% compatible read/write, the easiest method is certainly flux level. Yes, the files are large, but you can trim most everything to a single revolution per track, and really we don't care about file size anymore - maybe 10 years ago we did, but hard drive space and RAM is not an issue anymore. You could always keep the images compressed and decompress them when the disk is 'mounted'

Just to know, you said a typical disk is 5 MB per side, or 10 MB per floppy, which I agree is not that much compared to a standard 800 KB floppy. Does it compress well if you zip it for example ? Space is not an issue, but nevertheless, if we were to dump the entire collection of Atari ST games, the smaller it is, the better (especially when you have hosting limitation on the bandwidth).
The FPGA Arcade and a few other emulation projects are implementing .scp flux image files. The data is just clocked in like the real floppy. The WD1772 emulation would just fetch the flux transitions and convert them to clock and data bits. This would let you have 100% compatibility with any type of copy protection. I can't imagine it be that much work to change your existing WD1772 emulations to support flux directly. There is already a bunch of work done to parse through files, build tables, etc. With the .scp file format, you just need a track number to look up the offset to the track data header, which gives you info about the rotation speed, track length, and offset to the flux. From there you either copy the entire track to a circular buffer or fetch the track data from the file directly.

Handling flux level is certainly not that much work, especially once the work to add STX support to Hatari is made (which roughly means "writing functions that returns byte and their duration by parsing the STX file"). Here I would need to have function to assemble the flux transition to create bits and assemble those bits to create bytes+duration. This would require MFM decoding, do you plan to write some libraries yourself to handle the flux level informations and provide the parsed result to the upper level emulator ?

Nicolas

User avatar
npomarede
Atari God
Atari God
Posts: 1114
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: SuperCard Pro

Postby npomarede » Wed Jan 01, 2014 5:15 pm

DrCoolZic wrote:Now let say that you discover a new protection like the macrodos protection that uses four segments with different cell timing. You will need to define a new type to store in the file but more importantly you will need to write the code in the library that will return the right timing values for this new protection and this can ONLY be done by SPS.

Macrodos is already supported, I tested Hatari with a partial dump of Damocles that SPS sent me some times ago, and the protection passed succesfully. That's why I had the feeling capslibrary is pretty much complete right now, and could certainly already support a sector split into 16 chunks of bytes at different speeds ? Do you mean the IPF format doesn't allow to define a sector as a variable list of blocks of bytes, each one with its own duration ?
Of course new protection schemes for Atari ST not yet imaged with KF could require to extend the IPF format later if current IPF is limited.

Nicolas

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: SuperCard Pro

Postby DrCoolZic » Wed Jan 01, 2014 5:26 pm

npomarede wrote:
DrCoolZic wrote:Now let say that you discover a new protection like the macrodos protection that uses four segments with different cell timing. You will need to define a new type to store in the file but more importantly you will need to write the code in the library that will return the right timing values for this new protection and this can ONLY be done by SPS.

Macrodos is already supported, I tested Hatari with a partial dump of Damocles that SPS sent me some times ago, and the protection passed succesfully. That's why I had the feeling capslibrary is pretty much complete right now, and could certainly already support a sector split into 16 chunks of bytes at different speeds ? Do you mean the IPF format doesn't allow to define a sector as a variable list of blocks of bytes, each one with its own duration ?
Of course new protection schemes for Atari ST not yet imaged with KF could require to extend the IPF format later it current IPF is limited.

Nicolas

Yes I know that macrodos is supported. I was taking this as an example of what needs to be done.
Yes any kind of bytes (or even bits) timing variation can potentially be described. But each time a new type of block needs to be described the lib needs to be modified.
I am not convinced that all protections used by Atari are handled today (at least Dungeon Master is not).

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 548
Joined: Mon Nov 04, 2013 5:23 pm

Re: SuperCard Pro

Postby JimDrew » Wed Jan 01, 2014 5:29 pm

Flux images vary greatly. For example:

Dungeon Master, non-compressed: 11,376,322 bytes - compressed (.7z): 4,664,106 bytes
Winter Games, non-compressed: 8,277,290 bytes - compressed (.7z): 2,342,786 bytes
Word Perfect, non-compressed: 12,751,533 bytes - compressed (.7z): 2,601,272 bytes

7zip seems to do the best job, followed by RAR, and then standard ZIP. I did not play with any of the compression options so it is possible there is some magic there that can further reduce the size.

I am not a PC programmer, although I did write about a million line assembly program (x86) called FUSION - the Mac emulation for the PC. I don't know any of the Windows stuff though, which is why the SCP software was written in VB. I can certainly help with structures, suggestions, etc. I do NOT write code in C - I never have, and I never will. C doesn't make much sense to me. I code strictly in assembly for anything I touch (with exception of Windows, where I have used VB a few times). I know Z80, 6502, 68K, 808x, 80x86, 805x, PIC, AVR, and other assembly languages extensively. I develop flight control and weapons guidance systems as my daily job, which is all assembly coded for speed, size, and the fact that you can't rely on a compiler to generate the same code with compiler revisions. I created dozens of libraries for the Amiga, but never for the PC. So, other than providing structures and ideas, I won't be much help.
I am the flux ninja


Social Media

     

Return to “SuperCard Pro Disk Copier”

Who is online

Users browsing this forum: No registered users and 1 guest