LZMA - best packing method ?

GFA, ASM, STOS, ...

Moderators: exxos, simonsunnyboy, Mug UK, Zorro 2, Moderator Team

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2809
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

LZMA - best packing method ?

Postby AtariZoll » Sat Apr 08, 2017 10:43 am

It's used in some latest archivers, so in UPX too - they prefer now it. Possible to make TOS SFX with it, but only if it is given in command line. With reason - depacking is extremely slow. I disassembled depacker - it is very long (2.3 KB code), and I don't think that there is place to make is significantly faster. Yes, there is source code available, but it's not exactly same with what I see in SFX file made with UPX 3.93, plus is not good for Devpac.
I packed 1 screen data with LZMA - using freeware xz packer, ZIP, WinRAR and JEK 2.01.
LZMA: 4686 bytes
ZIP - 5595
WinRAR with best packing - 5250
JEK - 6844
Needs about 5 secs to depack 1 screen data (32KB), what is actually not worse than some depackers used in Atari SW, but with less efficient packing.
Packing on Atari is out of my consideration - I tried UPX.TTP, and it crashed as always. Will not bother to solve, because it would be too slow anyway.
Negative feedback has usually positive effect.

mlynn1974
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 124
Joined: Mon Mar 03, 2008 10:33 pm
Contact:

Re: LZMA - best packing method ?

Postby mlynn1974 » Mon Jun 19, 2017 11:59 pm

I haven't seen the LZMA depacker for the Atari ST. It's interesting that there is one, but I would expect it to be slow.

There are a number of considerations for depackers on systems with limited memory:
1. Memory required by the depacker itself including stack size,
2. Can the decompressed file be written to an absolute or relative address,
3. Dictionary sizes and maximum string match lengths,
4. Decompresssion time.
If the compressor is written in C, depending the optimiser, it might not be as efficient as one written in native assembly.

How does PackFire compare with your test picture?
http://www.pouet.net/prod.php?which=54840

From memory there were lots of neat features in Pack-Ice 2.4 making it better than the older packers at the time.
Still got, still working: Atari 4Mb STe, 520STFM, 2.5Mb STF.
Hardware: Cumana CSA 354, Ultimate Ripper, Blitz Turbo, Synchro Express II (US and UK Versions).

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2809
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: LZMA - best packing method ?

Postby AtariZoll » Tue Jun 20, 2017 8:39 am

I tried briefly PackFire. In large model it made little shorter files than xz packer. Looking into output file, and depacker src. I'm pretty sure that PackFire uses LZMA method too :D It is good to have src for Atari depacker. I will deal with it little - obviously it will be shorter than 2KB . Will see depacking speed - will be not fast - there are multiplications for instance, as there are in depacker I used.

1. It needs 16KB stack space, where dictionary is placed too
2. You serve dest address in one address register of CPU before calling depacker, so it is absolute. Don't see for what relative addressing would be good.
Especially as it must deal with larger files than 64KB.
3. I don't know details - see 1. It worked well with over 50 files I packed for 23 games on 1 floppy project. Longest was over 300 KB,
4. Instead time the speed: about 4-6 KB/sec .

I will test PackFire depacking speeds.
I don't think that we even should mention ICE in same thread with LZMA :mrgreen: It is slow, and ratio is nothing impressive.

Did depacking test. Large model depacking code len. is 700 bytes, and is PC-relative - and that's is what is useful. Speed seems some 10-20% better than UPX depacker speed. So, definitely better implementation of LZMA for Atari, 68000 :D Thanx for pointing on it.
Negative feedback has usually positive effect.

User avatar
leonard
Moderator
Moderator
Posts: 640
Joined: Thu May 23, 2002 10:48 pm
Contact:

Re: LZMA - best packing method ?

Postby leonard » Thu Jul 13, 2017 11:03 pm

For depacking speed of various depacker on various atari machine, you can refer to numbers here: http://www.atari-forum.com/viewtopic.php?f=16&t=31438&p=316491#p316491 (it includes depacker routine size, and packfire tiny and large model .pft and .pfl)

Packfire has really good compression ratio, thanks to arithmetic coding algorithm (one of the key of LZMA). The drawback is the need of arithmetic operation, ie MUL in our case. Depacking is really slow compared to huffman entropy decoder. If you accept to lose just a little packing ratio but still want to use modern arithmetic class decoder, you could use shrinkler! the depacker is two times faster than Packfire. Really impressive. ( .shk in my benchmark )
Leonard/OXYGENE.

mikro
Atari God
Atari God
Posts: 1288
Joined: Sat Sep 10, 2005 11:11 am
Location: Brisbane, Queensland, Australia
Contact:

Re: LZMA - best packing method ?

Postby mikro » Fri Jul 14, 2017 12:08 am

I don't know what your problem with UPX is, I use it for packing all my releases and obviously it works.

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2809
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: LZMA - best packing method ?

Postby AtariZoll » Fri Jul 14, 2017 7:13 am

mikro wrote:I don't know what your problem with UPX is, I use it for packing all my releases and obviously it works.

I don't have that to add - Atari TTP works not . And I don't care actually.
How much time takes to pack some 300 KB executable on 8 MHz Atari ? Do you pack only executables, or some binary files too ?
Negative feedback has usually positive effect.

mikro
Atari God
Atari God
Posts: 1288
Joined: Sat Sep 10, 2005 11:11 am
Location: Brisbane, Queensland, Australia
Contact:

Re: LZMA - best packing method ?

Postby mikro » Fri Jul 14, 2017 7:37 am

AtariZoll wrote:I don't have that to add - Atari TTP works not . And I don't care actually.
How much time takes to pack some 300 KB executable on 8 MHz Atari ? Do you pack only executables, or some binary files too ?

Of course I pack it on PC. :) Executables only, depacking time is not really that bad, hardly noticeable. But bear in mind my software usually targets Falcons/CT60/FireBee, i.e. high end machines.

User avatar
troed
Atari God
Atari God
Posts: 1198
Joined: Mon Apr 30, 2012 6:20 pm
Location: Sweden

Re: LZMA - best packing method ?

Postby troed » Fri Jul 14, 2017 7:48 am

I use UPX for everything, executables and binaries. Requires some manual stripping and modifications by my build system.

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2809
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: LZMA - best packing method ?

Postby AtariZoll » Fri Jul 14, 2017 7:50 am

mikro wrote:Of course I pack it on PC. :) Executables only, depacking time is not really that bad, hardly noticeable. But bear in mind my software usually targets Falcons/CT60/FireBee, i.e. high end machines.

Depacking time is excellent with default NRV packing method - well default only in Atari mode. Otherwise LZMA is default now in PC mode - told me one of UPX guys.
It is easy when you deal with executables only. Packing binary files with NRV is troublesome- because used NRV packer is closed source.
Negative feedback has usually positive effect.

rockyone
Captain Atari
Captain Atari
Posts: 335
Joined: Thu Jan 20, 2011 8:47 pm
Location: France
Contact:

Re: LZMA - best packing method ?

Postby rockyone » Fri Jul 14, 2017 12:28 pm

I have failed to run PackFire.exe with Windows 7 32 and 64-bit

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2809
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: LZMA - best packing method ?

Postby AtariZoll » Fri Jul 14, 2017 12:47 pm

rockyone wrote:I have failed to run PackFire.exe with Windows 7 32 and 64-bit

That's command line program - run it in console.
Negative feedback has usually positive effect.

User avatar
leonard
Moderator
Moderator
Posts: 640
Joined: Thu May 23, 2002 10:48 pm
Contact:

Re: LZMA - best packing method ?

Postby leonard » Tue Jul 18, 2017 6:42 pm

UPX depacks faster than ARJBETA -m7 but has lower compression ratio. ( for instance, We Were @ demo won't fit on two floppies if I use UPX as a packer (I did the test))
If you really want best compression ratio and acceptable depacking speed, use ARJBeta.
Leonard/OXYGENE.


Social Media

     

Return to “Coding”

Who is online

Users browsing this forum: No registered users and 2 guests