Emulation support

[JimDrew]

I had an idea I thought I would run by everyone... would anyone be interested in being able to read and write real ST disks with an emulator? I could program a complete WD1772 FDC emulation into the SuperCard Pro hardware and emulators could pass the FDC commands just like the ST does. I dont know who does the development of the various emulators, but if this is of interest it's something I can look at adding.

[Brume]

Just one word: YES

[alexh]

Looks interesting. How does the device stack up against Kryoflux? I'm interested to see one being used.

[JimDrew]

I created it, so I am biased - but I did so because of the shortcomings of the Kryoflux hardware, and quite frankly because of the attitudes of the people behind that project. I can't stand CLI based applications. I wanted something easy to use. I wanted something FAST (15 seconds to image a C64 disk vs. minutes with the Kryoflux). I wanted something that could be a stand alone device, controlled by anything from Timex Sinclair ZX-81 to a PC/MAC. I wanted something expandable, so it has a sd-media card slot and a fully bi-directional bus so it can become a floppy drive emulator with just a firmware change. I wanted a high resolution copy/image (25ns capture vs. 41.66ns capture of Kryoflux). I wanted the file format open for everyone to use and there are already several developers writing software for it. The SuperCard Pro hardware is far superior, less expensive, has a lifetime warranty, free software & firmware upgrades, and a very receptive developer with decades of experience in magnetic media duplication and imaging.

The response has actually been a bit better than I had anticipated, so I placed an order for the second production run already, which is much earlier than expected. There appears to be no issues with the hardware, so the v1.1 board will continue to be used. v1.0 was the original design and never produced. v1.1 added the bi-directional floppy interface to make it capable of being a floppy drive emulator like the HxC.

[jok]

Hi Jim,

very interesting hardware!

I am using a Kryoflux for over a year. What I really like about it: simple uncomplicated reading of unprotected disks in "normal" formats on PC (ST and Amiga) and ADF writeback to disk. Also IPF writeback is often handy. But IPF is a dead end in my opinion due to the requirement of a third party to create these files for the user (which they won't if they have the disk already in their collection).

Therefore I am really welcome a platform with an open format, which can directly store flux images and write them back without any intermediate requirements. Hopefully the emulator authors include the format asap, so testing of created images can be done directly on PC.

I am def. getting one of your device. If you will include an ST WD-FDC emulation as mentioned in another thread, that would be extremely cool!

[DrCoolZic]

Looks interesting. How does the device stack up against Kryoflux? I'm interested to see one being used.

I am modifying the prototype (should be ready today) so it can read SuperCard Pro raw csp format.

Here you go:
Below is the first reading of scp file with my tool:

First decoding of an SCP File !!!.JPG

And yes it looks like Dungeon Master ....

Be aware that this is very preliminary as pieces are hard coded but nevertheless cool
Note that CRC error is detected (sector 6 with red border) but not fuzzy bits (orange border) as it requires by definition at least 2 revolutions and the sample only provided one

[JimDrew]

jok,

I do need to make a converter to .ST format. I have the information on .stx from DrCoolZic, but I don't have any info on .ST file format. I am assuming it is just disk sector data sequentially like most other disk emulation formats. I will have to go find that info.

If any of the emulation authors want to support the raw flux images, I would be certainly happy to support them in anyway I can. That would give you 100% compatibility for disks at that point.

DrCoolZic,

Glad to hear progress is going well... have you figured out what disk image I sent to you yet?

[DrCoolZic]

Not too difficult as there is the name in the zip file

I now read it completely and I am modifying the internal data structure of my prog so it can accommodate both scp and KF files more easily
I might have a small problem using the file because from what I see it has only one revolution. Fuzzy (aka weam) bits requires a least 2 rev but this is good enough for test

[DrCoolZic]

Forgot to post what I am using for ST STT ....

[JimDrew]

Fuzzy (aka weak) bits requires a least 2 rev

No... that's a myth. If that were the case, then the image file I sent you would not create a working disk, and it certainly does. There are ways to determine where weak bits are located without having to read the disk multiple times. Also keep in mind that reading and writing of weakbits is *extremely* drive dependent. Some drives will not read (or write) weakbits at all.

Most all disk dumps with SuperCard Pro only need a single revolution, weakbits or not. The ONLY time you need a multi-rev dump is when the track does not start/stop on the index pulse. In these cases, we need 2 revolutions. The writing portion is then responsible for determining where the write splice is located (somewhere in the 2nd revolution) and writing out the disk as one full revolution + a partial revolution so that the data overlaps perfectly and shuts off the writing. There is no need to look at disk structures, decode anything, etc. The only thing you have to figure out is exactly where the writing was turned off on the original. Numerous write splices can occur on a disk and I see this quite often (especially with ST disks apparently). As long as you are not shutting off the write in the middle of a sector or some other valid data, you can shut it off anywhere. I locate a valid write splice and make the copy as accurate as the drive speed variation allows.

Thanks for the disk image file format info!! I figured that .ST files were just straight sectors.

[DrCoolZic]

Fuzzy (aka weak) bits requires a least 2 rev

No... that's a myth. If that were the case, then the image file I sent you would not create a working disk, and it certainly does. There are ways to determine where weak bits are located without having to read the disk multiple times. Also keep in mind that reading and writing of weakbits is *extremely* drive dependent. Some drives will not read (or write) weakbits at all.

Most all disk dumps with SuperCard Pro only need a single revolution, weakbits or not. The ONLY time you need a multi-rev dump is when the track does not start/stop on the index pulse. In these cases, we need 2 revolutions. The writing portion is then responsible for determining where the write splice is located (somewhere in the 2nd revolution) and writing out the disk as one full revolution + a partial revolution so that the data overlaps perfectly and shuts off the writing. There is no need to look at disk structures, decode anything, etc. The only thing you have to figure out is exactly where the writing was turned off on the original. Numerous write splices can occur on a disk and I see this quite often (especially with ST disks apparently). As long as you are not shutting off the write in the middle of a sector or some other valid data, you can shut it off anywhere. I locate a valid write splice and make the copy as accurate as the drive speed variation allows.

Thanks for the disk image file format info!! I figured that .ST files were just straight sectors.

Humm here again I guess it is a problem of terminology. Copying successfully a disk that contains fuzzy bits is certainly possible with one revolution
Detecting fuzzy bits (as described in patent 4849836) requires 2 or more revolutions by definition (see below)

Personally I do not like the term Weak bits because it can lead to confusion with weak bits
Normally weak bits are the result of magnetization problems (as described for example in the SpinRight document https://www.grc.com/srphysics.htm) that can occurs over time. There are also ways of creating artificial weak bits (meaning bits with a weaker flux transition energy) for example as described in a patent 4849836 (http://info-coach.fr/atari/documents/ge ... 849836.pdf) figure 3, 4, and 5. However I am not aware it has ever been used (at least on Atari).

What has been used in Dungeon Master is what is described in the same patent in figure 1 and 2.

This patent defines what I call fuzzy bytes (meaning indeterminate bytes - see http://en.wikipedia.org/wiki/Fuzzy_concept ): bytes that reads with different values when reading in the same data several times (see patent figure 6)

With my definitions you can says that weak bits, No output flux area, what I call border bits (pattent figure 1-2), random bits (from unformatted area), etc … results in fuzzy bits ...

[DrCoolZic]

ooops I have the admid rights and I did a mistake I have edited my previous post instead of quoting it !!!
so go to viewtopic.php?f=102&t=25848&p=242778#p242728

[DrCoolZic]

SuperCard Pro - KryoFlux - Dungeon Master Analyzer outputs

SuperCard Pro

dm-scp.JPG

KryoFlux

dm-kf.JPG

[JimDrew]

Detecting fuzzy bits (as described in patent 4849836) requires 2 or more revolutions by definition (see below)

By the definition of the patent - because that's what they state in the patent. However, (with certain drives) you DO NOT need to read two revolutions to determine if an area has weak bits. I have an option for showing the raw or 'fixed' (defined area) of weakbits for the c64 disks and I will add that to the MFM disks. With this option enabled, it will show you exactly where the weakbits are located. We have had weakbits since the early 80's with the Commodore disks, so the patent of 1989 was a bit behind the times. Weakbits are areas lacking flux transitions that are within the allowable clocking period. So, if the controller is expecting a 8us pulse and there isn't one for 50us, that would generate a random value being returned for the flux pulse duration. However, the value is not actually that random. In the case of the MFM drives, the read line pulse returns the duration of time between the last valid clocking up to and including the next valid clocking. So, bad data time + 1st good data time. The apparent floating of data happens (random values) because the valid clocking appears due to the AGC either picking up adjacent track data or the AGC is so high it is causing a feedback loop and generating garbage. Weakbits can be difficult because some drives won't read them properly, which may require the multi-rev comparison to get any difference between the bytes. Finding a drive that will read weakbits correctly is really important for duplicating disks. I have a list for the 5.25" drives already, and I working on a list of the 3.5" drives.

Nice job on the .scp file conversion for your program! That data looks familiar!

Here is my Dungeon Master output. You can see that I look at the data as either flux or raw MFM, not decoded.

dm.jpg

[JimDrew]

One other thing that I need to make more clear in the .scp file format documentation...

The bit cell times stored are based on the speed of the drive. The speed must be factored into the bit cell time when you are looking at flux data as a real time value. For example, with 360 RPM drives (3.5" HD drives spin this fast) the track time is ~166.6667ms per revolution. That is much different than a 300 RPM drive's 200ms per revolution. So, to calculate the true bit cell time, I use this formula:

D = 1000000 / IndexTime
F = FluxDuration
V = Int(F * 25 * D)

V = true flux duration (in nanoseconds), based on the rotational speed of the drive. IndexTime is the index to index time that is returned in the structure (TDH_INDEXTIME). FluxDuration is the bit cell time.

Using this formula you can convert between different speed drives. Of course, reading and writing on the same drive requires really no correction. But, as you can see from your own example between the Dungeon Master dumps done with SuperCard Pro and Kryoflux, the drive speeds were different. In order to re-create the disk correctly, every single bit cell must be adjusted proportionately to ensure the same number of bits appear on the copy as the original had.

[JimDrew]

Looking at the Dungeon Master data a bit more... I think that 'fuzzy bits' and 'weak bits' are very different. It appears that the 'fuzzy' bits are only fuzzy to the WD1772 controller. True weak bits have their values outside of the normal 2us to 12us range common with FM/MFM/GCR formats (all over the place). It looks to me that fuzzy bits are just bits that are varied in the data rate between the 4us and 6us range. Since the WD1772 controller is only expecting to see 4us/6us/8us durations, anything between these values is going to do odd things, all depending on how the controller clocking works.

[AtariZoll]

I had an idea I thought I would run by everyone... would anyone be interested in being able to read and write real ST disks with an emulator? I could program a complete WD1772 FDC emulation into the SuperCard Pro hardware and emulators could pass the FDC commands just like the ST does. I dont know who does the development of the various emulators, but if this is of interest it's something I can look at adding.

That's certainly very useful. You can contact Steven Seagal (Steem) and people working on Hatari.
And btw. many emulators (Amiga, Sinclair ..) use Simon Owen's http://simonowen.com/fdrawcmd/ for work with floppies on standard PC floppy drives. Interesting is that noone did it for some Atari ST emulator (and it would be not so hard, I know).
"I have the information on .stx from DrCoolZic, but I don't have any info on .ST file format. I am assuming it is just disk sector data sequentially like most other disk emulation formats. I will have to go find that info."
Yes, ST format is just raw sector content in order. Same as some other ones (DSK for instance, if I remember correct). MSA is used a lot too with Atari ST, and it has header with disk GEO info + possible RLL compression. There is STT format too (created by Steem authors), not much used, which supports some simpler protections.