Profiler --parse

A forum about the Hatari ST/STE/Falcon emulator - the current version is v2.3.0

Moderators: simonsunnyboy, thothy, Moderator Team

Post Reply
User avatar
Cyprian
10 GOTO 10
10 GOTO 10
Posts: 2564
Joined: Fri Oct 04, 2002 11:23 am
Location: Warsaw, Poland

Profiler --parse

Post by Cyprian »

Hi Eero,

I'm just trying to learn basic functionality of Hatari's debugger based on DML's ST Doom.
I have listed below script which works as expected, when I type by hand those commands into debbuger's console:

Code: Select all

b pc = TEXT :once
c
symbols stdf.sym TEXT
b PC = _raycast_world :once
c
profile on
b VBL = "VBL+200" :once
c
profile save myprofile.txt
q
But, when I put them into my file profiling.ini, and execute ("hatari --parse profiling.ini /testfile.prg"), those commands are executed one after other immediately, without waiting for breakpoints triggers.
I was also trying your example of automatic "chaining" of debugger from http://hg.tuxfamily.org/mercurialroot/h ... reakpoints but without success.

Could you advise me how to "automatize" my script?

thanks
Mega ST 1 / 7800 / Portfolio / Lynx II / Jaguar / TT030 / Mega STe / 800 XL / 1040 STe / Falcon030 / 65 XE / 520 STm / SM124 / SC1435
DDD HDD / AT Speed C16 / TF536 / SDrive / PAK68/3 / Lynx Multi Card / LDW Super 2000 / XCA12 / SkunkBoard / CosmosEx / SatanDisk / UltraSatan / USB Floppy Drive Emulator / Eiffel / SIO2PC / Crazy Dots / PAM Net
Hatari / Steem SSE / Aranym / Saint
http://260ste.atari.org
jok
Atari freak
Atari freak
Posts: 72
Joined: Wed Dec 19, 2012 3:06 pm

Re: Profiler --parse

Post by jok »

Start Hatari with Xbios extensions on. Then you can inject your debugger commands from your ST program via Xbios #254 call (and therefore your ST program triggers the profiling)
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2533
Joined: Sun Jul 31, 2011 1:11 pm

Re: Profiler --parse

Post by Eero Tamminen »

Cyprian wrote:Could you advise me how to "automatize" my script?
You cannot continue emulation from the script (debugger doesn't run emulation, it's emulation that calls debugger and "continue" command just returns control back to emulation). Scripts just execute debugger commands from within given debugger session, not across multiple debugger sessions.

To chain/automate actions over multiple debugger sessions (triggered by breakpoints), you need to put commands that should be executed on a given breakpoint to a separate file, and give its name to breakpoint with the ":file" option.

Some things you can do without external script (see "lock" command and breakpoint ":lock" option), but setting additional breakpoints isn't one of those things.

Note: Using separate scripts is more flexible. You can have multiple breakpoints in effect at the same time with each of them triggering different actions!


As an example, here are scripts I used to profile Douglas' Quake1 TT port; its CPU usage, XBios calls and GEMDOS allocations:

Code: Select all

$ head *.ini
==> tos-start.ini <==
# when TOS does first Fopen(), it's safe to set breakpoint for TEXT
b GemdosOpcode = 0x3D :once :trace :file program-start.ini

==> program-start.ini <==
# start allocation tracking etc when program starts
b pc = text :once :trace :file alloc-tracking.ini

==> alloc-tracking.ini <==
# load symbols
symbols ../source/bin/quake.sym text data bss

# quit emulator if program terminates to Pterm()
b  GemdosOpcode = 0x4C  :trace :quiet :file terminate.ini

# show Maxalloc, Malloc, Mfree, Mshrink callers
b  GemdosOpcode = 0x44  :noinit :quiet :file showcallers.ini
b  GemdosOpcode = 0x48  :noinit :quiet :file showcallers.ini
b  GemdosOpcode = 0x49  :noinit :quiet :file showcallers.ini

# all XBios calls except Gettime()
b  XbiosOpcode ! 0xFFFF && XbiosOpcode ! 0x17  :noinit :quiet :file showcallers.ini

# save startup profile until demo is started
b  pc = _CL_PlayDemo_f  :trace :file profile-start.ini

# trace stuff & profile
setopt --bios-intercept --conout 2
trace gemdos,xbios
profile on

==> profile-start.ini <==
# save profile until this moment
profile save quake-start.txt

==> showcallers.ini <==
# show (profile) callstack for breakpoint on a OS call.
# on next instruction it has returned so one can check return value
profile stack
b pc = "NextPC"  :noinit :quiet :once :file showretval.ini

==> showretval.ini <==
# show return value for OS call
e d0

==> terminate.ini <==
# program terminated, save profile & quit
profile save quake-end.txt
profile off
quit
Attached is script I use to initiate that profiling and post-process the results when Quake terminates.

Btw. Allocation tracking was because 14MB isn't enough for the port, XBios calls are tracked to find out issues in things like resolution setting and OS call tracing is to know a bit better where in the program (startup) execution these thing happen.

Similar stuff is used in Douglas' Doom port to find out where disk accesses happen, but instead of having manually edited debugger scripts, the scripts are written out by a shell script starting BadMood for profiling. This is because shell script has several different tracing options and those affect what happens in debugger scripts.
You do not have the required permissions to view the files attached to this post.
User avatar
Cyprian
10 GOTO 10
10 GOTO 10
Posts: 2564
Joined: Fri Oct 04, 2002 11:23 am
Location: Warsaw, Poland

Re: Profiler --parse

Post by Cyprian »

thanks Eero, based on your explanation and Douglas' file I prepared working version of chain.
It seems that, my main issue was in that string " :once :trace". Based on section "Chaining breakpoints " in Hatari's manual I was trying with only ":trace", and it doesn't work properly. After adding ":once" everything works as expected.

Profiler_App_Load:

Code: Select all

b GemdosOpcode = 0x3D :once :trace :file Profiler_App_Start.ini
Profiler_App_Start:

Code: Select all

b pc = TEXT :once :trace :file Profiler_Wait.ini
Profiler_Wait:

Code: Select all

symbols stdf.sym text data bss
b PC = _raycast_world :once :trace :file Profiler_Start.ini
Profiler_Start:

Code: Select all

profile on
b VBL = "VBL+200" :trace :file Profiler_End.ini
Profiler_End:

Code: Select all

trace none
b all
profile save myprofile.txt
q
Mega ST 1 / 7800 / Portfolio / Lynx II / Jaguar / TT030 / Mega STe / 800 XL / 1040 STe / Falcon030 / 65 XE / 520 STm / SM124 / SC1435
DDD HDD / AT Speed C16 / TF536 / SDrive / PAK68/3 / Lynx Multi Card / LDW Super 2000 / XCA12 / SkunkBoard / CosmosEx / SatanDisk / UltraSatan / USB Floppy Drive Emulator / Eiffel / SIO2PC / Crazy Dots / PAM Net
Hatari / Steem SSE / Aranym / Saint
http://260ste.atari.org
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2533
Joined: Sun Jul 31, 2011 1:11 pm

Re: Profiler --parse

Post by Eero Tamminen »

Cyprian wrote:thanks Eero, based on your explanation and Douglas' file I prepared working version of chain.
It seems that, my main issue was in that string " :once :trace". Based on section "Chaining breakpoints " in Hatari's manual I was trying with only ":trace", and it doesn't work properly. After adding ":once" everything works as expected.
In the example I knew that the breakpoints won't be hit before they're removed at the end of the chain, so I hadn't used ":once" option in it.

However, it can hit in cases where people apply the examples, so it's best to ":once" option also in examples:
http://hg.tuxfamily.org/mercurialroot/h ... ba54c1dbd5

Thanks for pointing that out! :-)
User avatar
Cyprian
10 GOTO 10
10 GOTO 10
Posts: 2564
Joined: Fri Oct 04, 2002 11:23 am
Location: Warsaw, Poland

Re: Profiler --parse

Post by Cyprian »

Eero Tamminen wrote:Thanks for pointing that out! :-)
welcome :) and thanks for the support
Mega ST 1 / 7800 / Portfolio / Lynx II / Jaguar / TT030 / Mega STe / 800 XL / 1040 STe / Falcon030 / 65 XE / 520 STm / SM124 / SC1435
DDD HDD / AT Speed C16 / TF536 / SDrive / PAK68/3 / Lynx Multi Card / LDW Super 2000 / XCA12 / SkunkBoard / CosmosEx / SatanDisk / UltraSatan / USB Floppy Drive Emulator / Eiffel / SIO2PC / Crazy Dots / PAM Net
Hatari / Steem SSE / Aranym / Saint
http://260ste.atari.org
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2009
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: Profiler --parse

Post by TheNameOfTheGame »

I am trying to get the chaining set up so I don't have to enter manually the debugger commands each time.

My pexec.ini has this:

Code: Select all

# continue to "program.ini" on Pexec(0, ....)
b GemdosOpcode = 0x4B && OsCallParam = 0x0 :trace :once :file program.ini
The program.ini is:

Code: Select all

b  pc = TEXT :once
setopt hex
symbols symbols.sym text data bss
The file is loaded when lauching Hatari with:

Code: Select all

/usr/local/bin/hatari --disasm ext --natfeats on --parse pexec.ini
However, when the emulator starts, it breaks on the first AUTO folder program "ARROWFIX.PRG" (which I understand is normal for the command I set)

My wish is to boot to the Desktop, then the first program I start from the Desktop is the one I am debugging. That is when I want the breakpoint to trigger. How would I do this?

Also, is there a way to set the default config when Hatari starts? It seems it want to load hatari.cfg by default so I have to manually go to the config screen and load my debug.cfg each time I start up Hatari. Thanks.
User avatar
troed
Atari God
Atari God
Posts: 1577
Joined: Mon Apr 30, 2012 6:20 pm
Location: Sweden

Re: Profiler --parse

Post by troed »

Code: Select all

--configfile <file>   Read (additional) configuration values from <file>
    or -c <file>
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2009
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: Profiler --parse

Post by TheNameOfTheGame »

troed wrote: Tue Apr 26, 2022 1:18 pm

Code: Select all

--configfile <file>   Read (additional) configuration values from <file>
    or -c <file>
Thanks, that worked. Note that the linux version references things (symbol files, cfg, memdumps, etc) from the home directory, but the config files are in .config/hatari (which from within Hatari it loads cfgs from there).

But so for the command line "--configfile .config/hatari/debug.cfg" picks up the file to load.
User avatar
troed
Atari God
Atari God
Posts: 1577
Joined: Mon Apr 30, 2012 6:20 pm
Location: Sweden

Re: Profiler --parse

Post by troed »

I'm on Linux and I usually load config files from my current working directory. No need to stuff them under ~/.config
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2009
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: Profiler --parse

Post by TheNameOfTheGame »

TheNameOfTheGame wrote: Tue Apr 26, 2022 12:41 pm I am trying to get the chaining set up so I don't have to enter manually the debugger commands each time.

My pexec.ini has this:

Code: Select all

# continue to "program.ini" on Pexec(0, ....)
b GemdosOpcode = 0x4B && OsCallParam = 0x0 :trace :once :file program.ini
The program.ini is:

Code: Select all

b  pc = TEXT :once
setopt hex
symbols symbols.sym text data bss
The file is loaded when lauching Hatari with:

Code: Select all

/usr/local/bin/hatari --disasm ext --natfeats on --parse pexec.ini
However, when the emulator starts, it breaks on the first AUTO folder program "ARROWFIX.PRG" (which I understand is normal for the command I set)

My wish is to boot to the Desktop, then the first program I start from the Desktop is the one I am debugging. That is when I want the breakpoint to trigger. How would I do this?

Also, is there a way to set the default config when Hatari starts? It seems it want to load hatari.cfg by default so I have to manually go to the config screen and load my debug.cfg each time I start up Hatari. Thanks.
*bump* Still trying to find a solution for this. Currently I get to the desktop, run my app and manually enter the debugger to type in the symbols line there and then continue. I am trying to automate this as it gets tedious to do this every time manually.
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2533
Joined: Sun Jul 31, 2011 1:11 pm

Re: Profiler --parse

Post by Eero Tamminen »

TheNameOfTheGame wrote: Tue Apr 26, 2022 12:41 pm ...
However, when the emulator starts, it breaks on the first AUTO folder program "ARROWFIX.PRG" (which I understand is normal for the command I set)

My wish is to boot to the Desktop, then the first program I start from the Desktop is the one I am debugging. That is when I want the breakpoint to trigger. How would I do this?
There are few alternatives:
  • Set first breakpoint to something that is not triggered by AUTO folder programs, and chain rest after that. Use e.g. one of the AES or VDI calls done by the desktop, before it (auto-)starts your program. You can use "--trace vdi,aes" to find out some suitable call
  • If the number of Pexec calls done before your own program start is always <N>, just tell breakpoint to trigger only on <N>th call using ":<N>" breakpoint option
But if you need this only to load debug symbols, easiest is just to compile the debug symbols into your binary. They will then be automatically loaded when debugger is first invoked.

(Only restriction is that program need to be started from GEMDOS HD, so that Hatari knows from which host file to load the debug symbols.)

You need to use separate symbol file only:
  • With things that are not programs (e.g. EmuTOS ROM, or additional code files loaded by programs),
  • If you want filter out symbols e.g. to speed up for profiling, or
  • Program cannot be loaded from GEMDOS HD (it e.g. works only from floppy, or under MiNT).
In last case, you just need to use "symbols" command to tell where (a non-stripped copy of) the program is.

You can load debugger settings, breakpoint definitions etc also later on, using "parse" debugger command, you don't need to chain everything at start.

PS. You do not need to start your program from Atari desktop, just give your program as argument to Hatari, and Hatari tells TOS to auto-start it (using program directory as C: drive). If arguments need to be passed to your program, Hatari provides helper script for that: https://git.tuxfamily.org/hatari/hatari ... rg-args.sh
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2009
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: Profiler --parse

Post by TheNameOfTheGame »

Eero Tamminen wrote: Fri Apr 29, 2022 10:11 pm
TheNameOfTheGame wrote: Tue Apr 26, 2022 12:41 pm ...
However, when the emulator starts, it breaks on the first AUTO folder program "ARROWFIX.PRG" (which I understand is normal for the command I set)

My wish is to boot to the Desktop, then the first program I start from the Desktop is the one I am debugging. That is when I want the breakpoint to trigger. How would I do this?
[*] If the number of Pexec calls done before your own program start is always <N>, just tell breakpoint to trigger only on <N>th call using ":<N>" breakpoint option
Thanks, this solution worked! Finally can stop typing in the symbols command each time. :lol:
But if you need this only to load debug symbols, easiest is just to compile the debug symbols into your binary. They will then be automatically loaded when debugger is first invoked.
The problem is that Pure-C only supports DRI symbols which are max 8 characters. So I need to use the external symbol table since most of the symbols are longer. I see in the source that Hatari supports symbols up to 32 characters.
PS. You do not need to start your program from Atari desktop, just give your program as argument to Hatari, and Hatari tells TOS to auto-start it (using program directory as C: drive). If arguments need to be passed to your program, Hatari provides helper script for that: https://git.tuxfamily.org/hatari/hatari ... rg-args.sh
Nice to know. I will try this out. Thanks again for the help.
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2533
Joined: Sun Jul 31, 2011 1:11 pm

Re: Profiler --parse

Post by Eero Tamminen »

TheNameOfTheGame wrote: Fri Apr 29, 2022 10:34 pm
Eero Tamminen wrote: Fri Apr 29, 2022 10:11 pm But if you need this only to load debug symbols, easiest is just to compile the debug symbols into your binary. They will then be automatically loaded when debugger is first invoked.
The problem is that Pure-C only supports DRI symbols which are max 8 characters. So I need to use the external symbol table since most of the symbols are longer. I see in the source that Hatari supports symbols up to 32 characters.
What kind of external symbol table? What generates it, and how it matches Pure-C generated binary?
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2009
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: Profiler --parse

Post by TheNameOfTheGame »

The external table was generated by Thorsten Otto using his own tools and probably some by-hand work.

We were working on fixing some bugs in ACSpro 3.0.0 2005 beta (which btw are fixed). Hatari debugger was very helpful in identifying where the problems were and Thorsten kindly implemented the fixes in his C reconstruction.

Btw, I never could get the callgraphs working. I was able to generate the the dot and then the SVG files but everything I tried to view them with just displayed a blank window.

What ended up being the feature I used the most was tracing. I set the xterm window buffer to unlimited scrollback buffer and traced between breakpoints and saved and cleared the scrollback buffer each time.

With those output files, I wrote a script to pull out only the symbol names which gave me a good view of the program flow. WinMerge helped compared similar code paths with the traced symbols to narrow down where the code path was diverging. Then it just became looking at the code itself within the debugger.
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2533
Joined: Sun Jul 31, 2011 1:11 pm

Re: Profiler --parse

Post by Eero Tamminen »

Callgraphs for larger programs are often too large to be converted into static SVG or PNG picture before you've used options discarding most (like >90%) of the data. E.g. browsers do not allow them to be scaled up enough for things in them to be readable.

It's best to view the DOT files interactively using something like Xdot (on Debian/Ubuntu as "apt install xdot"): https://github.com/jrfonseca/xdot.py
Post Reply

Return to “Hatari”