Trouble using FloImg or FdRawCmd

WinSTon, Nostalgia, MSA Converter, FloImg, Makedisk and all the others.

Moderators: Mug UK, Moderator Team

Lautreamont
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 103
Joined: Fri Jan 27, 2006 9:11 pm
Location: Friceland

Postby Lautreamont » Thu Sep 21, 2006 9:08 pm

1- Fastcopy
Daeghno already found out the problem with FastCopy:
http://www.atari-forum.com/viewtopic.php?t=7822&highlight=fastcopy+iii
Don't use the "quick" (or "fast" ?) format option.

2- Reading ST disks on a PC
All the tests and programs I've done on Linux months ago agree with obo.
I believe there's an hardware incompatibility problem between the WD1772 and the Nec or Intel PC chip. I often can't read the first sector (header and consequently data because I don't know how to read data without header) of any track when the sectors density is >= 10.
I believe the WD1772 and the PC chip do different things when they meet a new track, and the PC chip is slower, or does more, I don't know, I have only sketchy docs on the WD1772.

Although, sometimes the READ ID command can be succesful after 55 or more tries on some of the "first sectors" (and consequently, the READ command is succesful too). It can be a matter of hours to image a single disk!


More than 80% of my disks (most of them formatted with Fastcopy) are unreadable on a PC

obo
Atari freak
Atari freak
Posts: 51
Joined: Mon Nov 10, 2003 12:38 am
Location: Nottingham, UK

Postby obo » Thu Sep 21, 2006 10:14 pm

ijor wrote:Those are precisely the extra dummy headers, they must be "strange". The whole point is to insert a header alone, without the actual sector. Because the idea is that the next sector would be very close after the seek with verify. And then it must have an unused/strange sector number.

The logs from Pera show a dummy sector 66 (header only, no data field) in some cases, which I've seen on a couple of other ST disks but always assumed was part of a a simple copy protection. On one such disk there was a sector 1 immediately after it, which could be accessed using normal sector commands but was too close for a repeated read-id to catch, so it didn't appear in the scan. That sounds very much like what you suspected the index problem might be connected to?

Pera's tests with fast-header on and off also show a key difference. The first result is the same as leonard's disk: the 1st sector shown in the scan was the 2nd sector on the track, and the sector position suggested the 1st sector was very close to the start of the track (~30 bytes). The second result showed all sectors correctly, with the 1st sector shown as being ~90 bytes from the start of the track. I think the latter case was with fast-header enabled, which might mean his log names were reversed?

In second place, this option is the one that controls the creation or not of the dummy header. It doesn't control the gap sizes. But the header should actually be helpful for the gap problem. This is because now the first real sector is further from the index.

This fits with the second test above, which is an extra ~60 bytes further into the track. 60 bytes would about cover the overhead of the ID header plus gaps, with the next real ID header starting about where the data field would be for the dummy sector.

I'm still a bit confused about the FastCopy options though, and how they fit the tests. It sounds a lot like the FH option that Pera changed was having an effect a lot like the separate 'extra header' option. Any thoughts?

Si

obo
Atari freak
Atari freak
Posts: 51
Joined: Mon Nov 10, 2003 12:38 am
Location: Nottingham, UK

Postby obo » Thu Sep 21, 2006 10:29 pm

leonard wrote:I made some fast test with read-track command, reading 8192 raw bytes. I noticed the bit stream is not valid very fastly (about 1000 bytes). the clock is lost and some dummy bits seems to appairs , shifting all the rest.

The first 512 bytes of the data will be the data field for the first visible sector on the track, which will actually be sector 2 on track 0 of the problem disk due to the hidden sector. The next 2 bytes are the data field CRC, and if you don't lose sync at this point you'll get the gap values in the run up to the next sector.

What you think is lost sync might just be valid data that is in the wrong bit position in the bytes you're viewing. Remember that you're dealing with a bitstream as it comes from the drive, with the clock bits removed. The address mark detector isn't enabled to help align the values in bytes, so after the end of the first 514 bytes there are no guarantees about alignment. You need to work through the data bit-by-bit, using a shift-register method to compare values in a sliding window against what you expect to see for address marks. Finding 2 or 3 0xa1 values followed by 0xfe or 0xfb should be enough to indicate normal id and data address marks, and you can then decode the field information and check CRCs.

Unfortunately I haven't been home long enough to try it myself tonight, but I'll have another try tomorrow to see what the quality of the data is like.

Is there a way with the driver to to a "read track" , but beginning at the last sector (or at a user specifyed sector?).

Unfortunately not, as the PC controller's read-track command always starts from the index pulse. The parameters for the command almost suggest it should be possible, but they're simply sharing a read/write parameter structure with other commands and most (except size) are ignored.

It's even more unfortunate that the read-track command starts by reading the data field for the 2nd sector, and we want the details for the 1st sector. The data we want will be almost 6K into the buffer - knowing that (and assuming 10-sector ST disks) we could start processing further through the bitstream buffer.

I mean, on my first track, I know sectors 2,3,4,5,6,7,8,9,10 are ok. If I can run a "read track" command, beginning at sector 10, the sector1 will come "closer" in my raw buffer, and I get more chance to decode it correctly. I hope my explains are not too confuse.

I see what you mean, but I'm not sure that starting later would help much. It's just as likely to lose sync at the end of the data field for sector 10 as anywhere else, so I don't think it necessarily makes it worse. Disk2FDI author Vincent probably has a better idea of how likely it is, as he's done much more of this than me.

I don't succeed in testing it with your driver: the read-track don't take the "sector start" into account. Do you see a way to do what I want?

I do see exactly what you mean, and what you want. The drive is passing all parameters through to the controller, and it's the controller that ignores the starting sector - that's just the way the command is documented to behave, so there's nothing my driver can do to improve it.

Si

ppera

Postby ppera » Fri Sep 22, 2006 12:25 pm

I thought that my log file names are clear:

10st_NOFH - 10 sector/track, no fast header (set) it is where all sectors are readable on PC

10st_FHtr0reached - 10 s/t , fast header (set) - it is onlyone which was formatted to end (start actually), and PC can't see first sectors on tracks

Other 2 was set as MS DOS,
But wrote some ID at start, with sector #66 - despite, such disk is probably readable well on PC.

I performed scan of 'FH' floppies on Atari with my simple program TRACC. Unexpectedly, I didn't see some strange ID numbers at beginning of tracks - it just showed and read all 10 sectors.

1: are we correct about used technique - that there is some ID at start of track?

2: or ID is so close to index pulse that even Atari's FDC can't see it by scan ?
It would be in accord with my opinion, that no index is watched by sector read or seek - so, by normal read ST will see that ID's, and can use them with seek (with verify flag on)...

In any case, I don't see why then author of FC Pro didn't use skew of 1 by such floppies.

Maybe I will try to add this trick to my format programm, and will check how 10 s/t floppy with skew of 1 will read.... And then comes hard work: adding warning that those floppies will not read on 500000000 computers :D
Last edited by ppera on Fri Sep 22, 2006 12:42 pm, edited 1 time in total.

ppera

Postby ppera » Fri Sep 22, 2006 12:40 pm

ijor wrote:Well, I wasn't talking about how/why it fail (have no idea why). I was just commenting about the reason why it starts from the last track. Even when you used 80 tracks, it make sense that the program uses always the same direction.

By me, format should first check start of floppy (system area), and if there is some bad sector abort immediately with info. Don't see real advantage in reversed format. Of course that it can not be reason for bug. I just have impression that (mostly) German authors liked to use non-standard tecnhiques, often without good result.

....
Those are precisely the extra dummy headers, they must be "strange". The whole point is to insert a header alone, without the actual sector. Because the idea is that the next sector would be very close after the seek with verify. And then it must have an unused/strange sector number.

Btw, I'm not quite sure I understand how/why disabling "Fast access headers" makes a difference for you in reading the disks with the PC. In first place, that option seems to be ignored when using 10 or more sectors per tracks. At least it is ignored in the version of FcopyPro that I'm using.

In second place, this option is the one that controls the creation or not of the dummy header. It doesn't control the gap sizes. But the header should actually be helpful for the gap problem. This is because now the first real sector is further from the index.


I formatted 9 sec/track floppies as PC format. And 10 s/t as Atari format. And 'fast access headers' is not ignored by 10 sec/tr - Leonard started this thread with such floppies.

But it looks that FC Pro left some of ID even by PC format. Because of bigger gaps, it is visible by 9 s/t (my theory) - actually I left switch on, to see will it turn off if MS DOS is selected. So, I need at least on test more - MS DOS with FH off...
I'm getting tired of FC pro...

User avatar
leonard
Moderator
Moderator
Posts: 658
Joined: Thu May 23, 2002 10:48 pm
Contact:

Postby leonard » Fri Sep 22, 2006 1:37 pm

I do see exactly what you mean, and what you want. The drive is passing all parameters through to the controller, and it's the controller that ignores the starting sector - that's just the way the command is documented to behave, so there's nothing my driver can do to improve it.


thanks obo for details. Sorry for my lack of knowledge of PC driver, but is this possible (or not):

Start a "read sector" (not a read track) on sector 10 (the last on the track), and then , when data is deconded, change the sector size "on the fly". I beleive it's not possible but let's ask. (the same way disk2fdi change the floppy drive "on the fly". So we could read raw 8192 bytes, starting at sector 10, track 0. Possible? (I guess not...)
Leonard/OXYGENE.

obo
Atari freak
Atari freak
Posts: 51
Joined: Mon Nov 10, 2003 12:38 am
Location: Nottingham, UK

Postby obo » Fri Sep 22, 2006 3:07 pm

leonard wrote:thanks obo for details. Sorry for my lack of knowledge of PC driver, but is this possible (or not):

No worries - the general PC controller operation isn't too different from the WD1772, but there are a number of extra quirks and restrictions.

Start a "read sector" (not a read track) on sector 10 (the last on the track), and then , when data is deconded, change the sector size "on the fly". I beleive it's not possible but let's ask. (the same way disk2fdi change the floppy drive "on the fly".

The size code passed for the sector read command must match the sector being read for the command to begin, and once it has started there's no way to change how much the controller will transfer. The controller will ignore anything written to it when it's busy, and it doesn't even have an interrupt command like the WD1772!

The disk2fdi trick only works because the drive selection circuitry is separate from the controller, and it's possible to change selection without the controller's involvement. After the change it's simply sampling from the data line on the other drive, and you get its data instead.

Si

obo
Atari freak
Atari freak
Posts: 51
Joined: Mon Nov 10, 2003 12:38 am
Location: Nottingham, UK

Postby obo » Fri Sep 22, 2006 3:14 pm

Lautreamont wrote:Although, sometimes the READ ID command can be succesful after 55 or more tries on some of the "first sectors" (and consequently, the READ command is succesful too). It can be a matter of hours to image a single disk!

I can imagine that small variations in rotation speed could bring some of them just into range, though the 30 bytes we've seen with some ST disks seems a little too far away - wouldn't hurt to try though.

Coindentally, I've just been sent some logs for a different system that are also suffering from the same index issue. Even stranger is that the scan of the disk shows all sectors on the track, including the sector 1 quite close to the index (about 45 bytes). The scan consistently shows all sector 1s on each track, so it's not just that some of them are visible. However, simply attempting to read the sectors is failing every one of them with sector-not-found. I might resort to the multiple retries for him, in the hope that it might manage it eventually.

Si

obo
Atari freak
Atari freak
Posts: 51
Joined: Mon Nov 10, 2003 12:38 am
Location: Nottingham, UK

Postby obo » Fri Sep 22, 2006 3:31 pm

leonard wrote:I made some fast test with read-track command, reading 8192 raw bytes.

I wrote a quick version at lunchtime too, which you can download from: http://obo.homeip.net/DiskTest01.zip

I've included source code so you can tweak it if you feel the urge - it was a bit of a rush job so I'm not promising it'll be bug-free! By default it reads 16K from track 0 and decodes what it finds. With my problem disk it does show a lot of what's on the disk, and I can sometimes see the header of sector 1, but I've yet to see manage both the header and data. Typical results look like this:

Code: Select all

 1051: C=80, H=00, R=02 N=03
 1095: Data (1024 bytes)
 2198: Data (no header)
 5601: C=00, H=00, R=01 N=02   <--- sector 1 header (but no data)
 8407: C=80, H=00, R=05 N=03
 9518: C=80, H=00, R=03 N=03
 9563: Data (1024 bytes)
11900: Data (no header)
12459: C=80, H=00, R=04 N=03
12504: Data (1024 bytes)
13559: C=80, H=00, R=02 N=03
13603: Data (1024 bytes)
14706: Data (no header)

When a header is found it shows the CHRN values, and when data is found close enough to a header the size is reported, otherwise an unsized data field is reported. Not every header and data field is seen, and running the program again will usually give different results. The 16K of data means there should be 2.5 tracks worth of data present, so you might see the same header more than once.

The value on the left shows the data offset where it was found, which reveals something very strange. The start of the data file is the data field for the first visible sector, as expected. However, the first decoded header always seemed to be header for this sector - that should be almost a full revolution away, but the file offset shows different. It's behaving as though it stopped transferring in the middle, and later continued.

I'd be interested if some of you could run this on your own FastCopy disks to see what happens. I've put an output line to shout if it does manage to read a complete sector 1 from track 0, to help you spot if it works.

Si

User avatar
leonard
Moderator
Moderator
Posts: 658
Joined: Thu May 23, 2002 10:48 pm
Contact:

Postby leonard » Fri Sep 22, 2006 3:57 pm

Hi obo,

this is the same progrma I wrote exept you add the bitstream parsing, which is cool. I get the same result as my program (even without the bitstream shift, I saw sometimes I lost sync just by looking at an hexa viewer :-))

with your program, sector 1 is successfully decoded, say, 4 times on 5, which is quite really good.

I guess the success depends on the hardware drive (sync loss depends on rotation speed variation I guess?)

this is a good news, 4/5 times is cool to read drives. I start to think of an interface which read sector by sector. If a sector is not found (RNF), the routine will automaticcaly try to "read track" trick.
Leonard/OXYGENE.

ijor
Hardware Guru
Hardware Guru
Posts: 3898
Joined: Sat May 29, 2004 7:52 pm
Contact:

Postby ijor » Fri Sep 22, 2006 4:32 pm

obo wrote:This fits with the second test above, which is an extra ~60 bytes further into the track. 60 bytes would about cover the overhead of the ID header plus gaps, with the next real ID header starting about where the data field would be for the dummy sector.


60 extra bytes sounds a bit too much for the extra header. I don't know how accurate are your measurements, but the difference should be less than 40 bytes.

I'm still a bit confused about the FastCopy options though, and how they fit the tests.


Run FcopyPro under Steem's debugger. Enable Pasti. Set a Pasti breakpoint on format commands. "Play" with the format options and attempt a format. The format might work or not, but you'll be able to see the format buffer.

But again, "Fast Access Headers" are completely ignored (at least on the FcopyPro I'm using) when using 10 or more sectors per track. What makes a change, and controls the gap, is "Fast format".

I can imagine that small variations in rotation speed could bring some of them just into range, though the 30 bytes we've seen with some ST disks seems a little too far away - wouldn't hurt to try though.

Coindentally, I've just been sent some logs for a different system that are also suffering from the same index issue. Even stranger is that the scan ...


I made the following test:

Formatted a "fast format" disk using FcopyPro on the ST. 10 sectors per track. Then I copied a standard DOS 720k over it using a sector copier without reformatting. The idea was to get a plain DOS disk, from the logical format point of view, onto a physical "FcopyPro fast format".

A modern PC running Windows XP, a different old PC (Pentium I) running old DOS. In both cases I got the same result: Disk can be read completely. Tried copying big files back and forth successfully. However disk access seems much slower than normal.

Tried makedisk. It reads boot sector and autoconfigures for 9 sectors per track. Read whole disk. Slowly. Now forces the physical geometry on makedisk for 10 sectors per track. Now makedisk fails.

So it seems to me that a different way of reading the sectors and/or different timeouts and/or different retry logic might be very helpful. Of course that my tests might not match all the PCs. But can't be just an unique case when I got the same results on two very different PCs.

ijor
Hardware Guru
Hardware Guru
Posts: 3898
Joined: Sat May 29, 2004 7:52 pm
Contact:

Postby ijor » Fri Sep 22, 2006 4:44 pm

ppera wrote:Other 2 was set as MS DOS, But wrote some ID at start, with sector #66 - despite, such disk is probably readable well on PC.


When using 9 sectors per track, then the first gap seems to be always much bigger.

1: are we correct about used technique - that there is some ID at start of track?


Again, it depends on the options and the track layout.

2: or ID is so close to index pulse that even Atari's FDC can't see it by scan ?


It doesn't matter where it is located. The WD1772 would always see it.

By me, format should first check start of floppy (system area), and if there is some bad sector abort immediately with info. Don't see real advantage in reversed format. Of course that it can not be reason for bug. I just have impression that (mostly) German authors liked to use non-standard tecnhiques, often without good result.


Most people formatting ST disks doesn't matter which track is bad. They want a full disk and will discard any disk with bad sectors (disregarding where the bad sectors are located).

Again, the big plus of starting from the last track is to verify that all tracks were accessed. If you start from track 0 and try to format more tracks than your drive can access, then you won't detect it (not without some extra verification).

obo
Atari freak
Atari freak
Posts: 51
Joined: Mon Nov 10, 2003 12:38 am
Location: Nottingham, UK

Postby obo » Fri Sep 22, 2006 5:01 pm

leonard wrote:I get the same result as my program (even without the bitstream shift, I saw sometimes I lost sync just by looking at an hexa viewer :-))

I started with the hex editor too, which was enough to show it was probably worth decoding properly. I say "properly", but my parser is very crude, and I've since thought it would be much better to use a 32-bit shift register to match the full A1A1A1FE for IDAM in one go. You probably don't need to worry about deleted DAMs either, so A1A1A1FB would be the start of the data fields.

with your program, sector 1 is successfully decoded, say, 4 times on 5, which is quite really good.

That's much better than I expected too. Was that showing the "*** Sector 1..." message each time, or just showing the header in the output?

I guess the success depends on the hardware drive (sync loss depends on rotation speed variation I guess?)

It's probably one of the factors affecting it, which makes it very unpredictable. I rely on the same effect for +3/CPC, to create a flakey sector that returns different results every time, as needed to fool a copy-protection.

I start to think of an interface which read sector by sector. If a sector is not found (RNF), the routine will automaticcaly try to "read track" trick.

I'd highly recommend using multi-sector reads for these tracks. You can ask for 10 sectors starting at sector 1, and if any of them fail you can use IOCTL_FD_GET_RESULT to determine which sector failed. If it's failing as a missing sector you can perhaps note that you'll need to find it later using the raw scanning, then use a multi-sector read for the remaining sectors.

I agree with Ijor and Pera that real-disk support for game disks is not worth it, as very few would be possible, and even those would require a slower scanning method to identify correctly. Optimising for regular 10-sector ST disks will handle any normal data disks, and hopefully most demos too. The raw reading still isn't a guaranteed method to handle problem disks, but if it can be used to improve compatibility then it's probably still worth doing. :)

I've had great success with using real disks in SimCoupe, with only 2 game disks unable to be booted from directly (one of them due to this index problem). The DMA accesses mean it uses little CPU, and run in a separate thread allows the emulation to stay running while waiting for the floppy to respond. Shout if I can be of any assistance...

Si

ppera

Postby ppera » Fri Sep 22, 2006 5:41 pm

I made special version of format program for Atari with fake ID at track start. And made so, that such floppy is fine readable/writeable on PC too.
I just didn't decreased most of Gaps, and those couple little. 10 sector/track plus fake ID with his Gaps fit fine on one track.

Speed: I don't know how to test speed gain of access. But sequencial read with skew of 1 was slover, what means that ST can't catch sector 1.
Of course... Fake ID must be set not on track beginning (as it is supposedly in FC pro), but before sector #1 .. I will try it tomorrow...

In attachment is PRG for Atari. Ir is made in rush, and not checks bad sectors, so use good floppies (if there is still some)

Ijor:
"It doesn't matter where it is located. The WD1772 would always see it. "

I talked there about scanning, not sector read. Are you sure? My scan on Atari showed not those IDs with FC Pro formated floppies, but formatted with mine I see well them.
You do not have the required permissions to view the files attached to this post.

ijor
Hardware Guru
Hardware Guru
Posts: 3898
Joined: Sat May 29, 2004 7:52 pm
Contact:

Postby ijor » Fri Sep 22, 2006 6:22 pm

obo wrote:I agree with Ijor and Pera that real-disk support for game disks is not worth it, as very few would be possible, and even those would require a slower scanning method to identify correctly...

The DMA accesses mean it uses little CPU, and run in a separate thread allows the emulation to stay running while waiting for the floppy to respond.


CPU time is usually not a problem on modern PCs. The main problem is sustaining real-time timing. It doesn't matter if you have free CPU time, emulation can't run (at real time) if you need to wait for the floppy too much.

If the programs reads trackwise then the problem is not so big. Emulation would keep running until the whole track is read. But if reading is done sectorwise, then it is much more difficult. You would need either to buffer the whole track previously, or read from the real floppy one sector at a time. Even with standard formats it would be difficult to do this at real time.

Of course, for most cases it doesn't matter too much if emulation must stop or slow down at certain times. It is mostly a cosmetic issue. But then the whole point of getting the original drive seeking sound and rythm would fail.

ppera wrote:Ijor:
"It doesn't matter where it is located. The WD1772 would always see it. "
I talked there about scanning, not sector read. Are you sure?


Yes, I am sure.

My scan on Atari showed not those IDs with FC Pro formated floppies, but formatted with mine I see well them.


It means that those floppies don't have the extra header ID (once again, I never get them on 10 sectors per track). Or your software is missing them.

ppera

Postby ppera » Fri Sep 22, 2006 7:15 pm

ijor wrote:It means that those floppies don't have the extra header ID (once again, I never get them on 10 sectors per track). Or your software is missing them.


It seems as correct - I see well IDs on floppies with 9 sec/tr.
But then, why PC can't read first sectors from tracks with such floppies?
When I check off 'fast header access' switch then PC reads fine all sectors...

Does it means that FC Pro is even bigger crap than I thought at first - too much bugs for 2 day testing... Author did not checked MS DOS format on PCs - or expected that users know what not should be set then...
Uf, this is really waste of time, even thinking about it. I will not test it more, amen.

obo
Atari freak
Atari freak
Posts: 51
Joined: Mon Nov 10, 2003 12:38 am
Location: Nottingham, UK

Postby obo » Fri Sep 22, 2006 11:26 pm

ijor wrote:60 extra bytes sounds a bit too much for the extra header. I don't know how accurate are your measurements, but the difference should be less than 40 bytes.

The scanning measurements are usually accurate to within a couple of bytes, with interrupt processing being an unavoidable factor. As a sample, here's the first few tracks from a high-density PC disk:

Code: Select all

500Kbps MFM, 512 bytes/sector:
  0:0   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
         12527: 168 651 651 651 650 651 652 651 650 652 651 651 651 652 651 651 651 651
  1:0   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
         12525: 167 650 651 652 650 651 651 651 650 652 652 651 651 652 651 651 651 651
  2:0   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
         12526: 167 651 651 651 651 652 651 651 651 652 652 651 651 651 651 652 652 650

You can see each first sector is 167-168 bytes from the start of the index pulse, and all the sector-sector spacings are 650-652 (header+gaps+data) apart. With 16us per byte at 500Kbps, the measurements are only varying by about 50us on my P4 2.8 here.

I was thinking the 60 byte difference for the extra header might be accounted for by the standard IBM header+gaps:

Code: Select all

12*0x00(sync) + 3*a1+0xfe(idam) + cyl + head + sector + size + crchigh + crclow + 22*0x4e(gap2) + 12*0x00(sync) = 56 bytes

That would match if the next real sector header is placed where the data field would be for the dummy sector. It would be big enough to allow the the controller to finish the seek+verify, and prepare to read the first sector on the new track. If it was too small there would be a risk of missing the first sector.

Run FcopyPro under Steem's debugger. Enable Pasti. Set a Pasti breakpoint on format commands. "Play" with the format options and attempt a format. The format might work or not, but you'll be able to see the format buffer.

I keep forgetting about that option for looking at the ST workings! :)

A modern PC running Windows XP, a different old PC (Pentium I) running old DOS. In both cases I got the same result: Disk can be read completely. Tried copying big files back and forth successfully. However disk access seems much slower than normal.

The speed does seem to depend on a lot of factors, including the head stepping rate, how long the driver waits for the head to settle after seeks, and whether multi-sector reads are used to ensure the controller is ready for the next sector in good time.

So it seems to me that a different way of reading the sectors and/or different timeouts and/or different retry logic might be very helpful. Of course that my tests might not match all the PCs.

My sample utility (http://simonowen.com/fdrawcmd/DiskUtil.zip) was designed for SAM disks, but it also nearly the same 10-sector format used by the ST. It formats using a 1-sector skew (which won't be optimal for the ST), but it can read/write that at full speed on almost all systems. The utility should also do pretty well with 2-sector skewed ST disks too, which will be a touch slower. It includes source if you'd prefer to build your own EXE or tweak the format skew/gaps.

Si

ijor
Hardware Guru
Hardware Guru
Posts: 3898
Joined: Sat May 29, 2004 7:52 pm
Contact:

Postby ijor » Sat Sep 23, 2006 1:31 am

Hi Simon,

obo wrote:I was thinking the 60 byte difference for the extra header might be accounted for by the standard IBM header+gaps:

Code: Select all

12*0x00(sync) + 3*a1+0xfe(idam) + cyl + head + sector + size + crchigh + crclow + 22*0x4e(gap2) + 12*0x00(sync) = 56 bytes

That would match if the next real sector header is placed where the data field would be for the dummy sector.


No, you are computing wrong by counting one serie of zeros twices. With the extra header the 3 A1 bytes would follow immediately after those 56 bytes. But without the extra header, they will be preceeded by 12 zeros. So the difference between both cases would be 44 and not 56. And because FcopyPro uses smaller gaps here, then the difference would be about 40.

Anyway I'm pretty sure that those disks didn't have the extra header. Pera seems to confirm that above. The 60 extra bytes you see, are for the bigger initial gap when disabling "Fast Format". Not for an extra header.

A modern PC running Windows XP, a different old PC (Pentium I) running old DOS. In both cases I got the same result: Disk can be read completely. Tried copying big files back and forth successfully. However disk access seems much slower than normal.
The speed does seem to depend on a lot of factors, including the head stepping rate, how long the driver waits for the head to settle after seeks ... My sample utility ...


I'm not sure I expressed myself correctly, or if you got my point or not. I was talking about a disk formatted with FcopyPro with "Fast Format", with the first (real) header starting at about 16 bytes from the index.

My point wasn't complaining about slowness. My point was that the slowness suggest that the (my) PC does can read those sectors depending how you read them. Because it seems that either was missing it in the first revolution and/or required more retries and/or it required the sector to be read individually.

It also shows that even when Makedisk have troubles and cannot read them, it doesn't necessarily mean it can't be read.

ppera

Postby ppera » Sat Sep 23, 2006 8:30 am

Yes, fast format switch is culprit in FC Pro. It makes too short first Gap for PC. But I really don't want to bother more with FC Pro and his floppies.
Instead, try to make some use from things learned here.

I made version of formatter which puts extra ID before sector #1 on each track - and it allowed finally ST to read fast floppies with 10 s/t and skew 1.

I need to tune it more - it looks that track size is on limit, and sometimes goes over index.

I think that we should try to improve format on PC - writing of 10 s/t floppies formatted on it is pretty slow, while floppies formated on ST writes much faster. I will play little with Gap value, but don't think that it will help much.

Maybe is worth to try to put fake ID before sectors #1 by format - if it is possible at all with PCs FDC....

Some thought about why PC misses first ID, if it is too slose to index (start of track) : not because of index pulse itself, but because there is not enough 0x4E byte in flow. There are 0x4E's at end of track, but bitstream is losen by end of format, so it is not contingous flow. I don't see way, how standard PC with one floppy can read such ID...

ppera

Postby ppera » Sat Sep 23, 2006 10:41 am

Some speed results with FloImg:

800K floppy with skew of 2 - full write: 51 sec
800K floppy with skew of 1 - full write: 47 sec (but I observed couple short slowdowns) - this 2 is formatted on PC

800K floppy with skew 1 and fake ID before sectors #1, formatted on ST:
full write 44 secs

All it is according to theory, that skew of 1 should be faster for some 9 % than skew of 2.

On PC format was with fdrawcmd & gap3 value of 32.

User avatar
leonard
Moderator
Moderator
Posts: 658
Joined: Thu May 23, 2002 10:48 pm
Contact:

Postby leonard » Sat Sep 23, 2006 5:51 pm

Hi again,

I tried some code to read sectors by using raw read track trick, and then "bitstream" parsing. By retrying several times, I could read my first fcopy disk.

But, now a strange things happend: when trying my new program with other floppies, I get a strange result: I have some disk (classic 10 sectors disk), which can't be read with the rawtrack trick (even with tons of retry). BUT, these tracks read perfectly using classic "sector read" (ie disk are ok with FloImg).

My question: How can it possible? Is the "rawtrack" command produce bad data (even retrying 100 times)? Or is the way of parsing the bitstream wrong? (it fail with the DiskTest01 of obo too)

btw, a question: when writing my bitstream search stuff, I wondered something: we all agree we have to search pattern with a "bit precision" (not byte) in a raw track. But, is it possible that "0xa1" for exemple is stored in more than 8bits? (I mean, 9 bits?). so depdening of the drive speed some inner bits can be dupplicated?

all in all, why some tracks are readable with "read sector" and are not using the "read track" ??
Leonard/OXYGENE.

obo
Atari freak
Atari freak
Posts: 51
Joined: Mon Nov 10, 2003 12:38 am
Location: Nottingham, UK

Postby obo » Sun Sep 24, 2006 2:28 am

leonard wrote:But, is it possible that "0xa1" for exemple is stored in more than 8bits? (I mean, 9 bits?). so depdening of the drive speed some inner bits can be dupplicated?

The bitstream will be consistent in all places except the write-splice points, so the 0xa1 will always be 8 clean data bits (plus the appropriate clock bits for the address mark).

It might help if you start by thinking how the track is formatted initially. The controller writes from index to index in a single pass, using a single data rate of 250Kbps. The MFM encoding uses 1 clock bit per data bit, so the raw bitrate is 500Kbps on ST double-density disks. This bitstream is clean and consistent, except for the wrapping point at the end of the track. If you run the raw reading utility on a freshly formatted disk you should get very good results, as the hardware has no problem staying in sync right up to the end of the track. There's still a chance it could trip up at the wrapping point though, which would prevent it finding the real first sector on the track.

The bigger problems start when you write data into sectors on the track, when the controller is writes a new section of bitstream. The new section itself has a consistent bitcell width, but the alignment of the start and end points is unlikely to exactly match up with the original content. Ideally the PLL will cross the seam and lock on to the new data correctly, but it could just as easily start syncing with the clock bits instead - that would give is useless data that wouldn't contain what we need for decoding. Writing to a 10-sector track gives 10 sets of start and end write-splice points to handle, and the sync could be good or bad after each one! Subtle real-world variations in rotation speed seem to affect just how each one is handled, for different results each time. Some could also be consistently good or bad, so there may be some that are never read correctly.

The Amiga doesn't have a traditional floppy controller, and reads/writes full tracks in a single pass. This makes it particularly well suited to the Disk2FDI method of reading, since there are no write-splice points to throw the sync off. The 2-drive method does still add another complication of the index points on the 2 drives not being in sync, so you don't know where on the track you're starting the read from. The solution for adfread is to look for a long enough section of bitstream without invalid breaks (too many zero bits, which is invalid MFM).

all in all, why some tracks are readable with "read sector" and are not using the "read track" ??

When reading/writing normal sectors, the controller uses the address mark detector to handle this. When enabled, it watches for specific patterns with missing clock bits, which can only occur in address marks. Once seen it locks on to the correct part of the bitstream, so data following is valid.

The problem with our raw track reading is that the whole track is read as a big data field, and the address mark detector remains disabled to prevent it reacting to bitstream patterns that would correspond to address marks. The data+crc for the first visible sector is guaranteed to be corrected synced for the start of the read, but the rest of the track is down to chance as covered above. Incidentally, the WD1772 controller leaves the address mark detector enabled during its diagnostic track read command. This has the benefit of re-syncing for each address mark, but has a drawback that normal data field patterns can cause unwanted triggers that throw it into returning clock bits instead of data bits.

I did some Googling for the small gap4a index issue, and it seems to come up a lot with TRS-80 disks too. A suggested solution is to modify a floppy cable so the index line is on a switch, so it can be turned off. Without it the controller isn't blind, so it can see the first sector. Not a nice software solution, but might still be worth considering if all else fails!

Si

User avatar
leonard
Moderator
Moderator
Posts: 658
Joined: Thu May 23, 2002 10:48 pm
Contact:

Postby leonard » Sun Sep 24, 2006 7:13 am

ok obo I see the point with the "format track" and then "write sector".

I still don't enderstand something. I have a disk with 10 sectors, and these sectors are readable using "read sector" command. when doing read-track, I can retreive some sectors headers and some data (A1A1A1etc..), but not for all sectors. I enderstand with your explain that maybe the fact the index mark detector is not active could explain that. so suppose index mark is not active, why can't I find "A1A1A1" pattern for some headers? Suppose index mark is not active, how the A1A1A1 could be read? Still don't enderstand why finding that pattern. (maybe shifted, but consistant).
Leonard/OXYGENE.

ppera

Postby ppera » Sun Sep 24, 2006 10:02 am

I can try to add support for FC Pro 'too short track lead' floppies in FloImg.

I think that best strategy is to autodetect such sloppies (not big deal) and read sectors 2-10 normally (sectors by pos on track, not by # ) and then with readtrack read and then decode first sector on track - it should be much easier and reliable than decoding all sectors.

There was some other thread where was talk about problems with FC Pro floppies...
Maybe will be useful for more people.

User avatar
leonard
Moderator
Moderator
Posts: 658
Joined: Thu May 23, 2002 10:48 pm
Contact:

Postby leonard » Sun Sep 24, 2006 11:38 am

I'm writing such a program actually but I have tons of error, on many floppy. Really strange, some tracks can be read without trouble, some other never suceed (even with 30 or 50 retry !!).

Btw, How can I know which read-sector fail? When I execute "read data" for 10 sectors, if it returns true then I know it's ok, and if it returns false maybe some sectors are ok. How can I know what sectors are ok ?
Leonard/OXYGENE.


Social Media

     

Return to “Other emulators & tools”

Who is online

Users browsing this forum: No registered users and 6 guests