Issues known with CTR files

Post all your Kryoflux related topics in here. From questions about the hardware through to disks you've managed to image up and, probably most importantly, write back without any problems :)

Moderators: DrCoolZic, mr.vince, Moderator Team

User avatar
Brume
Red eyes
Red eyes
Posts: 4101
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Issues known with CTR files

Postby Brume » Sun May 11, 2014 11:25 am

I don't know if it concerns KF or Steem, but I wanted to open a thread about CTR support into Steem.
I've recently imaged a lot of original games into KF/CTR format. Almost all of them work fine, and adding CTR support into Steem SSE was a great idea (thanks KF team for the .dll and Steven Seagal for the support). It's very useful!

But a few of CTR images doesn't seem to work. That's the case of Out Run, for instance. It refuses to boot properly.
So I've also converted the RAW stream files (made with KF) into .STX format (with Aufit) = the game works fine.
I've imaged the game with two different drives and used another source, since I have both copies of the game = same result. CTR doesn't work, while .STX runs perfectly.

So it's definitively an issue with the CTR file.

The archive can be downloaded here:
Outrun - https://mega.co.nz/#!6INgQQ4T!67Aq7dNwP ... Uexu3UBfJ4

I've also included the SuperCard Pro file into the archive. This one seems to work. Please note I had to image the disk in 'Splice' mode (3 revolutions). The 'index' mode didn't image the disk correctly (at least it didn't start with Steem when it was converted with Aufit into .STX format).

User avatar
npomarede
Atari God
Atari God
Posts: 1178
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: Steem & CTR files

Postby npomarede » Sun May 11, 2014 11:52 am

Doesn't work for me either under Hatari.
From the traces, I see it reaches the point where it reads sector 9 on track 79, which should give a CRC error (that the case for the STX image).

I think IFW should be able to see if there's a problem with this CTR image.

Nicolas

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Steem & CTR files

Postby IFW » Sun May 11, 2014 12:08 pm

Known problem.
It uses a weak bit protection that uses a very specific sequence of FDC commands resulting in reading the same revolutions (and this sector data) twice where it should have read weak bits instead.
It wouldn't work from a real disk either, if it read the same data every 5 revolutions ;)
The reason it works with other formats (like it would from e.g. IPF) is the weak bit area found after analysis.
With CTR you get what you read in 5 revolutions, "played back" like a record as if it were coming from the disk, so no data gets modified - but in this case it should be.
I am considering adding support for this case to the CTR decoder, but it must work 100% and very fast (almost real time), otherwise it would mess up perfectly good data randomizing it... which is not something you want to happen :)
This won't be an issue from using stream files.

User avatar
npomarede
Atari God
Atari God
Posts: 1178
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: Steem & CTR files

Postby npomarede » Sun May 11, 2014 12:20 pm

By the way, Outrun uses TOS function to read sectors and check for the protection, so it's likely that if the game had accessed the FDC directly then it would have used a different sequence of FDC commands and it would have worked with CTR :)

Edit : Sarnau descibed it here http://www.sarnau.info/atari:hls_protection , so those games are likely to show the same problem :
    Roadrunner
    OutRun from Sega
    Music Construction Set
    Metrocross from Epyx
    Skyfox
    The Bards Tale
    Pinball Wizard

User avatar
Mug UK
Administrator
Administrator
Posts: 11210
Joined: Thu Apr 29, 2004 7:16 pm
Location: Stockport (UK)
Contact:

Re: Steem & CTR files

Postby Mug UK » Sun May 11, 2014 2:31 pm

Outrun also left the disassembly labels inside the main program in case you really want to go sniffing around and see what's happening in their protection code :)
My main site: http://www.mug-uk.co.uk - slowly digging up the bits from my past (and re-working a few): Atari ST, Sega 8-bit (game hacks) and NDS (Music ripping guide).

I develop a free Word (for Windows) add-in that's available for Word 2007 upwards. It's a fix-it toolbox that will allow power Word users to fix document errors. You can find it at: http://www.mikestoolbox.co.uk

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Steem & CTR files

Postby IFW » Mon May 12, 2014 10:11 am

Thanks, I already have a full trace of exactly what happens - they sure knew how to confuse emulation 30 years later on ;)
There are several versions of this protection compiled from C originally, then at least one later title used assembly code, probably to avoid the need for the C startup code.
The FDC command sequence is identical in all of them, only the code differs slightly depending on the compiler used - so it was more than likely added by the duplicator as a generic protection and part of the duplication service offered (many of them offered free copy-protection added to masters).
Many original, first releases of Gremlin, US Gold, Atari, Hewson games are affected, so as I said I am looking at whether this can be reliably supported or not - without breaking the legitimate data for other games.
Last edited by IFW on Mon May 12, 2014 10:16 am, edited 1 time in total.

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Steem & CTR files

Postby IFW » Mon May 12, 2014 10:14 am

btw: no, it's not HLS, that's very different and works just fine.
HLS was by an US duplicator/firm, this protection is EU based.

User avatar
Brume
Red eyes
Red eyes
Posts: 4101
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Issues known with CTR files

Postby Brume » Mon May 12, 2014 11:40 am

Thanks to everyone for your answers. Very interesting and not 'Steem only' related, so I've moved the thread.

I have a similar case: Gunship. I still haven't imaged into .STX format, but the disk works fine while the CTR crashs just after the intro. I'll post the archives tonight.

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Issues known with CTR files

Postby IFW » Tue May 13, 2014 1:00 pm

The games using this or similar protections work now from CTR.

User avatar
Brume
Red eyes
Red eyes
Posts: 4101
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Issues known with CTR files

Postby Brume » Tue May 13, 2014 1:13 pm

Wow, that was fast IFW, thank you.
I suppose it requires new IPF/CTR decoder library that will be available soon?

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Issues known with CTR files

Postby IFW » Tue May 13, 2014 2:12 pm

Yes, it does.

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Issues known with CTR files

Postby IFW » Tue May 13, 2014 5:51 pm

New library+headers attached.
You do not have the required permissions to view the files attached to this post.

User avatar
npomarede
Atari God
Atari God
Posts: 1178
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: Issues known with CTR files

Postby npomarede » Tue May 13, 2014 9:53 pm

I confirm that with this new version Hatari devel version can now also run the OutRun CTR dump posted by Brume, as well as RoadRunner for example (which uses a similar protection scheme)
:cheers:

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Steem & CTR files

Postby DrCoolZic » Wed May 14, 2014 6:36 am

IFW wrote:btw: no, it's not HLS, that's very different and works just fine.
HLS was by an US duplicator/firm, this protection is EU based.

Even if the name is not HLS at least is the description of the protection given by Markus correct?
If not can you provide more inforation about this magic sequence of FDC command?

Thanks Jean

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Issues known with CTR files

Postby IFW » Wed May 14, 2014 3:56 pm

The protection described by Markus is completely different.

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Issues known with CTR files

Postby IFW » Wed May 14, 2014 5:24 pm

One word could describe this: there is no magic involved at all, it could be described simply as "unlucky" :)
It's simply weak bits, but the order and timing of commands ensures that the exact same revolution is being read twice during the check and that of course always fails for weak bits, since the data is the same from the same revolution sampled...

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Issues known with CTR files

Postby DrCoolZic » Wed May 14, 2014 7:00 pm

So if I understand correctly this is purely a simulation problem due to an unusual sequence of FDC command.

I have looked at the track 79 and apparently sector 9 is only 128 bytes long (although it take the room of a 512 bytes sector) and has the data segment mostly unformatted resulting in fuzzy bytes and therefore CRC error.
So nothing realy special "from a flux transition point of vue".

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Issues known with CTR files

Postby IFW » Fri May 16, 2014 10:06 am

Nothing special at all.

User avatar
Brume
Red eyes
Red eyes
Posts: 4101
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Issues known with CTR files

Postby Brume » Sat May 17, 2014 8:38 pm

npomarede wrote:I confirm that with this new version Hatari devel version can now also run the OutRun CTR dump posted by Brume, as well as RoadRunner for example (which uses a similar protection scheme)
:cheers:


That's great Nicolas, can't wait to test the final version of Hatari!

I've tested the .CTR file of Outrun + Steem SSE + new dll provided by IFW: the game still doesn't work.

I guess Steven Seagal will have some work with that ;)

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1988
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Issues known with CTR files

Postby Steven Seagal » Sat May 17, 2014 9:34 pm

I'd like to download outrun.ctr first, but the upload site says I should "upgrade" my browser... :(

User avatar
Brume
Red eyes
Red eyes
Posts: 4101
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Issues known with CTR files

Postby Brume » Sat May 17, 2014 10:07 pm

OK, I've uploaded on another place. Try this one:
http://djmvcu5oku.1fichier.com/

I'm a little bit curious: what browser do you use? which version?

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1988
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Issues known with CTR files

Postby Steven Seagal » Sun May 18, 2014 10:14 am

Thx. I tried with IE (8?) and opera up-to-date, XP.

About the game, I don't understand what the emulator is supposed to do.
"Lock track" happens when capsimg calls the callback function, here it locks track 79 only once.
Capsimg doesn't call anything when it gets on a new rev.
Wouldn't it be simpler if the lib knew how many revs were sampled, or at least did a callback at each IP so the emulator knows what to do?

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Issues known with CTR files

Postby IFW » Sun May 18, 2014 1:31 pm

During track decoding, the library makes note whether a track is constantly changing at each revolution or not and relays this information back to the host.
If you are using the WD177x emulator in the library as the host, it would recognize this, and for any affected track would call the track change call back at each index signal passed.
So what you describes is exactly what should happen :)

If that does NOT happen, that means that you did not enable this behavior in the first place - this is useful for analysis tools or writing, not for emulation.

So I think the flags you pass to the load or lock functions say, that you want pure data without change.
Could you post the flags you are using?

User avatar
Steven Seagal
Atari God
Atari God
Posts: 1988
Joined: Sun Dec 04, 2005 9:12 am
Location: Undisclosed
Contact:

Re: Issues known with CTR files

Postby Steven Seagal » Sun May 18, 2014 5:13 pm

Hi, I use DI_LOCK_DENALT|DI_LOCK_DENVAR|DI_LOCK_UPDATEFD|DI_LOCK_TYPE on CAPSLockTrack().
The image isn't loaded in memory at once in Steem.
Each time a track is locked, the previous track (or the same in case) is unlocked just before.

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: Issues known with CTR files

Postby IFW » Mon May 19, 2014 11:04 am

The flags look fine.

The problem is unlocking a track; when you do that, state information gets invalidated and recreated the next time you lock the track again. The only state a host program would be interested in is the current random seed associated with the track, since you don't want to read the same data twice from weak bits normally ;)

Locking a track can be expensive btw., which is mostly only noticeable if your PC is not very fast, and you try a game that stream audio/video from disk; then you'll notice. The only games I can think of on ST are the ReadySoft games, such as Dragon's Lair.

However, if you still want to lock/unlock a track, then there is a solution: you should query the current random seed used and restore it.
Here is how:
Each track has an individual random seed.
Each time you lock a track use CapsTrackInfoT2. Save the wseed variable returned. Do not change it (as it may weaken the simple random generator).
Next time you lock the same track specify DI_LOCK_SETWSEED and set wseed in the structure to the previously obtained value.
The first time you ever lock a track wseed is always reset; you can observe this by checking the value returned.

Therefore, you should lock the track twice instead of once with your approach.
The first lock will decode and initialize the track with normal flags, the second time you should specify DI_LOCK_SETWSEED to continue the track randomization from the state where it was before you unlocked it. You should save the returned wseed after this call as well, since that is the updated state.
If you do this correctly wseed values before your call and after the call are different for any track that has weak bits.

It is much simpler with a preloaded image; all of the track are already locked, so you'd lock the track only once.


Social Media

     

Return to “Kryoflux”

Who is online

Users browsing this forum: No registered users and 1 guest