PASTI/STX File Format

In this forum you'll find more information about the Pasti & VAPI Tools and the Preservation Project built around these tools. Come on in to find out more about it and discuss these projects.

Moderators: Mug UK, ijor, Moderator Team

User avatar
npomarede
Atari God
Atari God
Posts: 1178
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: PASTI/STX File Format

Postby npomarede » Wed Jan 22, 2014 6:07 pm

IFW wrote:Just wondering... why don't you ask the author to open-source the imager...?
Then you can replicate the process exactly via emulating the WD drive.

Many people did, but Ijor never had the time to either document the format or do a linux/OSX binary or open source the code :(

Nicolas

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Wed Jan 22, 2014 7:52 pm

IFW wrote:Just wondering... why don't you ask the author to open-source the imager...?
Then you can replicate the process exactly via emulating the WD drive.

We did but ... he has no time to document the format and does not want to share the code ...
Only thing I could get was some answer to questions with an sloooow email turnaround time

User avatar
npomarede
Atari God
Atari God
Posts: 1178
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: PASTI/STX File Format

Postby npomarede » Thu Jan 23, 2014 11:13 am

DrCoolZic wrote:IMPORTANT MODIFICATION
Currently Bit 5 of TrackFlags is marked as not used. But in fact it is used!
Sarnau describe it as: 5 - with PASTI 0.4b always set, if Bit 0 is set (= track protected)
Files that I have generated with Aufit did not have this bit set and the resulting file was not working.

I have looked at many stx files and here what I believe is the meaning of bit 5:

- If bit 5 is set (bit 5=1) then this is the new format that implies that the readTime from Sector Descriptor HAS to be used (i.e. this time must not be equal to zero) unless the sector has the RNF flag set in which case readTime=0
For example this is the case in Maupiti Island where you have
Track 00.1 6293 bytes 1 sect FuzBytes=No Flag=61 TImage SDesc RecSize=6328 (11102-11117)
Sector T=7 H=0 SN=7 S=183 CRC=6607 bitPos=440 Time=0 Flags=18 Off=0 (11118-11133)

- if bit 5 is not set (bit 5=0) this is old format and in that case he readTime behave as explained in the doc: value of 0 means normal sector, value !=0 means you have to use the provided value

This need to be confirmed before I modify the Pasti Doc
Any feedback is welcome

Hello
I haven't checked any file, so I can't confirm or not, but in that case, I don't see what information bit 5 is adding to the decoder library ?
Because in all cases, whether bit 5 is 0 or 1, you will always do :
if sector has RNF flag, stops and readtime is 0
else if readtime is != 0, use it directly
else if readtime == 0, use the standard delay for a sector

I don't see where bit5 is needed ; but maybe the meaning of bit5 is what you describe, it's just that from what I see so far in STX format, there's no extra bit added when information can be deduced from others values.

I will have a look at PASTI.PRG from PastiImgKit.zip to see how images are created, this will be the best way to get the answer (assuming images with bit5=1 were produced with v0.4 beta).

Nicolas

Nicolas

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Thu Jan 23, 2014 12:32 pm

npomarede wrote:Hello
I haven't checked any file, so I can't confirm or not, but in that case, I don't see what information bit 5 is adding to the decoder library ?
Because in all cases, whether bit 5 is 0 or 1, you will always do :
if sector has RNF flag, stops and readtime is 0
else if readtime is != 0, use it directly
else if readtime == 0, use the standard delay for a sector

I don't see where bit5 is needed ; but maybe the meaning of bit5 is what you describe, it's just that from what I see so far in STX format, there's no extra bit added when information can be deduced from others values.

I will have a look at PASTI.PRG from PastiImgKit.zip to see how images are created, this will be the best way to get the answer (assuming images with bit5=1 were produced with v0.4 beta).

Nicolas

Nicolas

From a reading point of view you apparently should not care about bit 5 and do exactly what you say. But what is strange is that the Pasti DLL is using this bit because in some cases if not set then the file is not emulated correctly.

I believe the the early Past files where more optimized to generate smaller files and may be to image faster.
Pasti file generated with latest version of the imager seems to always have all information possible available: Sector Descriptor with timing length are always present as well as track image and no optimization to reuse track data to output sector.
Without any optimization a pasti file is about 1.8MB. As file size is not really an issue anymore it is more easy and safe to always output as much information as possible without going into complex algorithm to find out what is really necessary.

For Pasti writer it is important to also understand how some of the flag are used. In case of bit 5 this is definitively used by the PAsti DLL because when I was writing file without it did not work on some cases. And the cases where it failed where the cases with bit width variation (Rob Northen) so this seems to be related to timing information.

If you disassemble the writer I would be of course interested by anything you find out :mrgreen:

User avatar
npomarede
Atari God
Atari God
Posts: 1178
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: PASTI/STX File Format

Postby npomarede » Thu Jan 23, 2014 1:18 pm

DrCoolZic wrote:For Pasti writer it is important to also understand how some of the flag are used. In case of bit 5 this is definitively used by the PAsti DLL because when I was writing file without it did not work on some cases. And the cases where it failed where the cases with bit width variation (Rob Northen) so this seems to be related to timing information.

In that case, doesn't bit5 means "some timing data for each block of 16 bytes are available at the end of the track record" ?
In the image you tested, is bit5 always set when some timing blocks are at the end of the track record ?

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Thu Jan 23, 2014 3:26 pm

No Rob Northen uses one "long sector" 17000++ so no need for timing record in that case :(

User avatar
npomarede
Atari God
Atari God
Posts: 1178
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: PASTI/STX File Format

Postby npomarede » Thu Jan 23, 2014 4:43 pm

Here's the result of disassembling Pasti 0.4, I added my own comments ; from what I see, this is very similar to what Sarnau wrote in his doc.

Code: Select all

;
; Build STX Image
;
L010A:LEA       L039E(PC),A0            ; Image buffer
      MOVE.L    #$52535900,(A0)+        ; $00.l : Header 'RSY\0'
      MOVE.W    #$300,(A0)+             ; $04.w : Version 3
      MOVEQ     #1,D0
      BSR       L01E6
      MOVE.W    D0,(A0)+                ; $06.w : Tool 0x01 (Pasti) or 0xcc (Discovery Cartridge)
      CLR.W     (A0)+                   ; $08.w : Reserved_1
      CLR.B     (A0)+                   ; $0A.b : TrackCount
      MOVE.B    #2,(A0)+                ; $0B.b : Revision 2
      CLR.L     (A0)                    ; $0C.l : Reserved_2
      LEA       L039E(PC),A0
      MOVE.W    L037E(PC),D0            ; track per side
      TST.B     L034E                   ; double sided ?
      BEQ.S     L010B
      ADD.W     D0,D0                   ; TrackCount x 2
L010B:MOVE.B    D0,$A(A0)               ; $0A.b : TrackCount
      RTS


;
; Build Track header
;
L010C:MOVEM.L   D6-D7,-(A7)
      BSR       L01D5
      MOVEA.L   A0,A2                   ; header buffer
      MOVE.W    #$F,D1
L010D:CLR.B     (A0)+                   ; clear buffer
      DBF       D1,L010D
      MOVE.W    L0387(PC),D0            ; track number
      TST.W     L037D                   ; track side
      BEQ.S     L010E
      ORI.B     #-$80,D0
L010E:MOVE.B    D0,$E(A2)               ; $0E.b : TrackNumber
      TST.B     L035F                   ; sector headers present ?
      BEQ.S     L010F                   ; yes, add some sector headers

; Simple track block, with 512 byte sectors only and no sector header
      MOVE.W    L0391(PC),D0            ; sectors per track
      BSR       L01E6
      MOVE.W    D0,8(A2)                ; $08.b : SectorCount
      CLR.L     L0396
      MOVE.W    L0391(PC),D0
      MULU      #$200,D0
      MOVE.L    D0,L0397                ; size of all the sectors
      ADDI.W    #$10,D0                 ; size of the track block
      BRA       L0114

; Complex track with sector headers
L010F:MOVE.W    L038B(PC),D0
      BSR       L01E6
      MOVE.W    D0,8(A2)                ; $08.b : SectorCount
      MOVE.W    #1,D0                   ; TrackFlag bit 0 : has sector blocks
      ORI.W     #$20,D0                 ; TrackFlag bit 5 : always 1 ?
      TST.B     L035E                   ; track image ?
      BEQ.S     L0110
      ORI.W     #$40,D0                 ; TrackFlag bit 6 : has track image
L0110:BSR       L01E6
      MOVE.W    D0,$A(A2)               ; $0A.w : TrackFLags
      MOVE.W    L0388(PC),D0
      BSR       L01E6
      MOVE.W    D0,$C(A2)               ; $0C.w : TrackLength in bytes
      MOVEQ     #$10,D6                 ; size of the track block
      MOVE.L    L0394(PC),D0
      MOVE.L    D0,L0396
      ADD.L     D0,D6                   ; update total size
      BSR       L01E7
      MOVE.L    D0,4(A2)                ; $04.l : size of the fuzzy block
      BSR       L01D6

; Add sectors headers
      MOVEA.L   A0,A3
      MOVE.W    L038B(PC),D7            ; number of sectors
      MOVE.L    L036E(PC),D2            ; data offset for the 1st sector
      BRA.S     L0113
L0111:ADDI.W    #$10,D6
      BTST      #4,$E(A3)               ; $0E.b : SectorFlag bit 4 : RNF
      BNE.S     L0112
      MOVE.L    D2,D0
      BSR       L01E7
      MOVE.L    D0,(A3)                 ; $00.l : DataOffset
      BSR       L01AA                   ; get sector size in bytes
      EXT.L     D0
      ADD.L     D0,D2
L0112:LEA       $10(A3),A3
L0113:DBF       D7,L0111
      MOVE.L    D2,L0397
      ADD.L     D2,D6
      MOVE.L    D6,D0

L0114:MOVE.L    D0,L0389
      BSR       L01E7
      MOVE.L    D0,(A2)                 ; $00.l : track block size
      MOVEM.L   (A7)+,D6-D7
      RTS


This is just a part of the code, but we can see the values for file header and track header.
Version is always "3", tool is always "1", revision is always "2", reserved_1 and reserved_2 are always "0".

For the track header :
- if the track only contains the sector data (similar to a .ST image), then all fields in the track header are set to "0", except TrackBlockSize ($00.l), SectorCount ($08.b) and TrackNumber ($0E.b)
- else, the track will contain some sectors header + optional data. In that case TrackFlag ($0A.b) has : bit0=1, bit5=1, bit6=1 if MFM track image is included, bit7=1 if track image has a 2 words header.

So, bit5 is always 1 ; if your conversion tool creates STX images with these values, I think it should be compatible with images created by Pasti 0.4

Nicolas

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Sat Jan 25, 2014 1:03 pm

Thanks for this information.
Unfortunately this does gives information about the usage of TrackFlags bit 5 by Pasti DLL? We know that latest versions of imager set ti to 1 but the problem is should you care when reading stx file?

Hippy Dave
Atari Super Hero
Atari Super Hero
Posts: 515
Joined: Sat Jan 10, 2009 5:40 am

Re: PASTI/STX File Format

Postby Hippy Dave » Sat Jan 25, 2014 6:55 pm

DrCoolZic wrote:Thanks for this information.
Unfortunately this does gives information about the usage of TrackFlags bit 5 by Pasti DLL? We know that latest versions of imager set ti to 1 but the problem is should you care when reading stx file?

npomarede wrote:
DrCoolZic wrote:For Pasti writer it is important to also understand how some of the flag are used. In case of bit 5 this is definitively used by the PAsti DLL because when I was writing file without it did not work on some cases. And the cases where it failed where the cases with bit width variation (Rob Northen) so this seems to be related to timing information.

In that case, doesn't bit5 means "some timing data for each block of 16 bytes are available at the end of the track record" ?
In the image you tested, is bit5 always set when some timing blocks are at the end of the track record ?

My understanding of this is:
- bit5 means some timing data for each block of 16 bytes are available at the end of the track record.
- bit5 is always set.

User avatar
npomarede
Atari God
Atari God
Posts: 1178
Joined: Sat Dec 01, 2007 7:38 pm
Location: France

Re: PASTI/STX File Format

Postby npomarede » Mon Jan 27, 2014 12:22 am

Hippy Dave wrote:My understanding of this is:
- bit5 means some timing data for each block of 16 bytes are available at the end of the track record.
- bit5 is always set.

Not really, most STX files made with recent pasti have bit 5=1, but no specific timing data for each block of 16 bytes.
My guess is more that bit5 has a different meaning in STX version < 3, now that version 3 made with pasti 0.4 seem to always put more detailed info (instead of leaving sometimes defaut values), maybe this bit is not needed anymore (bit5 could mean "precise dump mode" for example, and this mode would now be the default one in pasti 0.4 / STX v3)

Nicolas

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Mon Jan 27, 2014 8:06 am

Correct. But what is strange is that if you put all the detailed information in the file but you do not set bit 5 to 1 then Pasti.DLL does not interpret the information correctly.

IFW
Captain Atari
Captain Atari
Posts: 181
Joined: Fri Jul 22, 2011 4:53 pm

Re: PASTI/STX File Format

Postby IFW » Mon Jan 27, 2014 7:17 pm

Just a wild guess here: it is being used as quick check (possibly the OR result of several conditions) that makes the DLL treat the track using different processing.
IPF files have a similar shortcut as well, in order to save on scanning a lot more data to find out whether a track has e.g. variable density or changes content with each revolution.

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Fri Feb 28, 2014 5:04 pm

For those interested ...

Pasti DLL used by Steem perform the following operations on reading Pasti file
[*] If program try to do a read track on image that do not contains a track data record Pasti gives a warning message (can be disabled) and returns a completely empty track if no sector is present. If sectors are found then the track is filled with appropriate gap information. For example 0x4E followed by 0x00 before sync etc. Before the first sync these value may be shifted as explained below.
[*] Read track on an imaged track returns for the bytes located before the first sync: either the bytes recorded in the file or bytes that corresponds to decoded MFM equivalent shifted right by one bit. For example 4E=9524 can be changed to 90=492A. This behavior can be useful when reading certain track like in Barbarian where the read track can return data 00 or FF or AA or 55 before the first sync. However for images done by Pasti or new version of Aufit (0.4b) this may not be required (see below)

Image content creation:
When doing a read track the value of the bytes located before the first sync are somewhat unpredictable. For example the character 0x4E present in GAP can take the following values:
4E=9254 21=24A9 9C=4952 42=92A4 39=2549 84=4A92 72=9524 09=2A49 E4=5492 12=A924 C9=5349 24=A492 93=4925 48=924A 27=2495 90=492A 4E
In normal case this is not relevant. However in some cases the decoded values are tested as part of a protection and therefore the way it is sampled and stored in stx file becomes important (for example Barbarian or Falcon).
IN barbarian or Falcon a train of 8µs flux transition is used that results in a MFM string like this ...10001000100010001000100010001...
Depending on where the sampling starts this can result in MFM 1111 2222 4444 8888 that decodes into 55 00 AA 00 respectively.
This kind of protection usually test for presence of 00 or FF in bytes before the first sync. Pasti imager and new Aufit 0.4b+ make sure this kind of sequence is stored in the stx file as 00 and therefore the protection check should succeed even if bytes before sync are not shifted as described above

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Tue Nov 11, 2014 11:47 am

Anybody interested in Pasti.dll 0.2i ???

Here is message from Ijor when delivered
Attached is a Pasti DLL version that is newer than the one available to the public. I don t remember for sure exactly what is new and changed. I would need to check, but I didn t want to delay giving you this update. I do know that some things were fixed, and then some (or all) of the images that didn t work should work now. But, considering that this wasn t widely tested, it is also possible that some things were broken and things that worked in the past didn t work now.


I have been using it for the last three years without problem.
But I have also not found any improvements?

User avatar
Maartau
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2592
Joined: Thu Dec 15, 2005 2:15 am
Location: France

Re: PASTI/STX File Format

Postby Maartau » Tue Nov 11, 2014 3:01 pm

DrCoolZic wrote:Anybody interested in Pasti.dll 0.2i ???


Yes, sir :D !
Member of :
- aTaRi LeGeNd ,
- eLiTe ! ,
- NoExTrA .

Don't hesitate to visit http://www.atarimania.com/ & http://www.atarilegend.com/ :D

-> Slowed due to serious health troubles <-

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Tue Nov 11, 2014 3:08 pm

Yes stupid question :)
You do not have the required permissions to view the files attached to this post.

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PASTI/STX File Format

Postby DrCoolZic » Thu Nov 13, 2014 5:16 pm

New version 1.3 of my documentation on protection see viewtopic.php?f=95&t=21952#p261784
There is a lot about fuzzy track/sector and Pasti limitation


Social Media

     

Return to “Pasti & VAPI”

Who is online

Users browsing this forum: No registered users and 2 guests