PureC VeryVery Slow stdio implementation ?

C and PASCAL (or any other high-level languages) in here please

Moderators: exxos, simonsunnyboy, Mug UK, Zorro 2, Moderator Team

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

PureC VeryVery Slow stdio implementation ?

Postby DrCoolZic » Wed Dec 16, 2009 10:28 am

I was using standard stdio call for copying files: fopen(), fread(), fwrite()
Here is the the bare bone loop with no error checking to copy a file (where BUFSIZE is 100K):

Code: Select all

      /* copy the file */
      start_time = timer();
      elapsed();
      while(!feof(src)) {
         count = fread(buffer1, 1, BUFSIZE, src);
         hp->info.src_time += elapsed();
         fwrite(buffer1, 1, count, dst);
         hp->info.dst_time += elapsed();
      }
      printf("%lu msec. (R+W)\n", timer() - start_time);

And here is the output for a test binary files of size 100, 1000, 10000, 100000, 1000000 bytes
SNAPIT00.jpg

As you can see above 10K the timing seems VERY VERY VERY unreasonably long :evil: :(

I therefore decided to replace the stdio calls by GEMDOS calls:

Code: Select all

      /* copy the file */
      start_time = timer();
      elapsed();
      do {
         scount = Fread(src, BUFSIZE, buffer1);
         hp->info.src_time += elapsed();
         dcount = Fwrite(dst, scount, buffer1);
         hp->info.dst_time += elapsed();
      } while ((scount == BUFSIZE) && (scount >= 0) && (dcount >= 0));
      
      if((scount >= 0) && (dcount >= 0))
         printf("%lu msec. (R+W)\n", timer() - start_time);
      else
         printf("%lu msec. Error Processing File\n", timer() - start_time);

I have rerun the exact same test on the exact same partitions:
SNAPIT01.jpg

Speed execution for large file is now 10-20 times faster !!!! :D

How is it possible that PureC stdio is so inneficient ??????

I'd like to hear your experience on PureC lib :wink:
You do not have the required permissions to view the files attached to this post.

joska
Hardware Guru
Hardware Guru
Posts: 3692
Joined: Tue Oct 30, 2007 2:55 pm
Location: Florø, Norway
Contact:

Re: PureC VeryVery Slow stdio implementation ?

Postby joska » Wed Dec 16, 2009 10:40 am

DrCoolZic wrote:How is it possible that PureC stdio is so inneficient ??????

I'd like to hear your experience on PureC lib :wink:


I have never done any tests like this, but it doesn't surprise me that there's problem with a library that has not been updated/bugfixed since 1990 ;-) You might want to take a look at the stdlib in the ACHH distribution. You should be able to use it with PureC straight out of the box.
Jo Even

Firebee - Falcon060 - Milan060 - Falcon040 - MIST - Mega ST - STM - STE - Amiga 600 - Sharp MZ700 - MSX - Amstrad CPC - C64

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PureC VeryVery Slow stdio implementation ?

Postby DrCoolZic » Wed Dec 16, 2009 10:58 am

joska wrote:You might want to take a look at the stdlib in the ACHH distribution. You should be able to use it with PureC straight out of the box.

What is theACHH distribution and where can I find it?
Thanks
Jean

User avatar
Nyh
Atari God
Atari God
Posts: 1496
Joined: Tue Oct 12, 2004 2:25 pm
Location: Netherlands

Re: PureC VeryVery Slow stdio implementation ?

Postby Nyh » Wed Dec 16, 2009 11:02 am

DrCoolZic wrote:I was using standard stdio call for copying files: fopen(), fread(), fwrite()
Here is the the bare bone loop with no error checking to copy a file (where BUFSIZE is 100K):

Code: Select all

      /* copy the file */
      start_time = timer();
      elapsed();
      while(!feof(src)) {
         count = fread(buffer1, 1, BUFSIZE, src);
         hp->info.src_time += elapsed();
         fwrite(buffer1, 1, count, dst);
         hp->info.dst_time += elapsed();
      }
      printf("%lu msec. (R+W)\n", timer() - start_time);

And here is the output for a test binary files of size 100, 1000, 10000, 100000, 1000000 bytes
As you can see above 10K the timing seems VERY VERY VERY unreasonably long :evil: :(

I have rerun the exact same test on the exact same partitions:
Speed execution for large file is now 10-20 times faster !!!! :D

How is it possible that PureC stdio is so inneficient ??????

I'd like to hear your experience on PureC lib :wink:

Just use:
fread(buffer1, BUFSIZE, 1, src);
instead of
fread(buffer1, 1, BUFSIZE, src);
You are reading BUFSIZE elements of size 1. Reading one element of BUFSIZE is a lot faster.

Hans Wessels

joska
Hardware Guru
Hardware Guru
Posts: 3692
Joined: Tue Oct 30, 2007 2:55 pm
Location: Florø, Norway
Contact:

Re: PureC VeryVery Slow stdio implementation ?

Postby joska » Wed Dec 16, 2009 11:06 am

DrCoolZic wrote:
joska wrote:You might want to take a look at the stdlib in the ACHH distribution. You should be able to use it with PureC straight out of the box.

What is theACHH distribution and where can I find it?


http://members.chello.nl/h.robbers/
Jo Even

Firebee - Falcon060 - Milan060 - Falcon040 - MIST - Mega ST - STM - STE - Amiga 600 - Sharp MZ700 - MSX - Amstrad CPC - C64

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PureC VeryVery Slow stdio implementation ?

Postby DrCoolZic » Wed Dec 16, 2009 12:18 pm

joska wrote:http://members.chello.nl/h.robbers/

Very interesting. I need now to look at it in more detail
Thanks Jo

User avatar
wongck
Ultimate Atarian
Ultimate Atarian
Posts: 12004
Joined: Sat May 03, 2008 2:09 pm
Location: Far East
Contact:

Re: PureC VeryVery Slow stdio implementation ?

Postby wongck » Wed Dec 16, 2009 12:22 pm

Nyh wrote:Just use:
fread(buffer1, BUFSIZE, 1, src);
instead of
fread(buffer1, 1, BUFSIZE, src);
You are reading BUFSIZE elements of size 1. Reading one element of BUFSIZE is a lot faster.
Hans Wessels


Yes, I suggest you try Hans's suggestion.
Instead of going to the I/O to fetch a byte for X number of times, it should be faster to fetach X bytes once.
Of course, it's not going to the real I/O but via the buffers.
My Stuff: FB/Falcon CT63+CTPCI_ATI_RTL8139 14+512MB 30GB HDD CF HxC_SD/ TT030 68882 4+32MB 520MB Nova/ 520STFM 4MB Tos206 SCSI
Shared SCSI Bus:ScsiLink ethernet, 9GB HDD,SD-reader @ http://phsw.atari.org
My Atari stuff for sale - click here for list

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PureC VeryVery Slow stdio implementation ?

Postby DrCoolZic » Wed Dec 16, 2009 12:33 pm

Nyh wrote:Just use:
fread(buffer1, BUFSIZE, 1, src);
instead of
fread(buffer1, 1, BUFSIZE, src);
You are reading BUFSIZE elements of size 1. Reading one element of BUFSIZE is a lot faster.
Hans Wessels

Humm. Rather unusual usage! I am trying to read bin char and therefore size is 1. Calling the other way around will return 1 or 0 (less than BUFSIZE char remaining) and do not get you the actual count of read elements when returning 0.

In term of performance it should really not matter as any decent implementation would anyway read (count * size_elem)
Common ANSI definition for the function is to call getc() but of course actual implementation differs

Actually following the pointer given by Jo I looked at the implementation (download source & look at stdio.c) in AHCC:

Code: Select all

size_t fread(void *data, size_t size, size_t count, FILE *fp)
{
   size_t n, m, lsiz;
   short f, c;

   f    = fp->_flag &= ~_IORW;
   lsiz = size;
   n    = count * lsiz;

   if (f & _IODEV)                     /* device i/o */
   {
      for(m = 0; m < n; ++m)
      {
         if ((c = fgetc(fp)) == EOF)
            break;
         *((char *)data)++ = c;
      }
   othw                           /* file i/o */
      fflush(fp);                     /* re-sync file pointers */
      m = Fread(fp->_file, n, data);
   }

   return m > 0 ? m / lsiz : (errno = m);
}

and as expected it does a call to Fread with n = (size * count) and therefore in term of performance calling it either way is the same.
This seems the ovious way to implement on Atari and I really wander how it was implemented in PureC

User avatar
wongck
Ultimate Atarian
Ultimate Atarian
Posts: 12004
Joined: Sat May 03, 2008 2:09 pm
Location: Far East
Contact:

Re: PureC VeryVery Slow stdio implementation ?

Postby wongck » Wed Dec 16, 2009 1:30 pm

DrCoolZic wrote:Humm. Rather unusual usage! I am trying to read bin char and therefore size is 1. Calling the other way around will return 1 or 0 (less than BUFSIZE char remaining) and do not get you the actual count of read elements when returning 0.

yes, if you are using it to count the number of elements read, then you are forced to use only 1 way and not the other.

DrCoolZic wrote:In term of performance it should really not matter as any decent implementation would anyway read (count * size_elem)
Common ANSI definition for the function is to call getc() but of course actual implementation differs

Ok I admit it, the only time I actually timed the fread/fwrite was on a DOS machine 8O , fetching/putting data from/to the network and HDD.
If I buffer my read/write from/to the network, i.e. writting/reading from/to the HDD in a large chunk I get performance increase vs writting/reading many times in small chunks.

Even I have used it many times in Atari, back when i was programming, I did not worry about the performance as it was a hobby. :wink:
My Stuff: FB/Falcon CT63+CTPCI_ATI_RTL8139 14+512MB 30GB HDD CF HxC_SD/ TT030 68882 4+32MB 520MB Nova/ 520STFM 4MB Tos206 SCSI
Shared SCSI Bus:ScsiLink ethernet, 9GB HDD,SD-reader @ http://phsw.atari.org
My Atari stuff for sale - click here for list

User avatar
Nyh
Atari God
Atari God
Posts: 1496
Joined: Tue Oct 12, 2004 2:25 pm
Location: Netherlands

Re: PureC VeryVery Slow stdio implementation ?

Postby Nyh » Sat Dec 19, 2009 1:56 pm

DrCoolZic wrote:This seems the ovious way to implement on Atari and I really wander how it was implemented in PureC

I had a short peek with the debugger and it seems Pure C just uses a loop to fetch every element.

Maybe we should patch the Pure C library on this point?

Hans Wessels

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: PureC VeryVery Slow stdio implementation ?

Postby DrCoolZic » Mon Dec 21, 2009 1:30 pm

Nyh wrote:
DrCoolZic wrote:This seems the ovious way to implement on Atari and I really wander how it was implemented in PureC

I had a short peek with the debugger and it seems Pure C just uses a loop to fetch every element.

Maybe we should patch the Pure C library on this point?

Hans Wessels

Thats what I thought. I looked for infromation about the fread implementation. Seems like ANSI define it as loop of getc() command/macro. But of course most implementation do better.
Would be nice to have a patched version. However using GEMDOS call fix the problem. Only you cant mix GEMDOS and stdlib calls;

User avatar
Nyh
Atari God
Atari God
Posts: 1496
Joined: Tue Oct 12, 2004 2:25 pm
Location: Netherlands

Re: PureC VeryVery Slow stdio implementation ?

Postby Nyh » Thu Dec 24, 2009 4:15 pm

DrCoolZic wrote:Thats what I thought. I looked for infromation about the fread implementation. Seems like ANSI define it as loop of getc() command/macro. But of course most implementation do better.
Would be nice to have a patched version. However using GEMDOS call fix the problem. Only you cant mix GEMDOS and stdlib calls;

I disassembled the fread() from Pure C and it follows the ANSI suggestion. It is indeed a loop of getc() commands. So the element size and count have a very low impact on fread() speed.

Before I change implementation O have to carefully study the effect of the various file modi. Read only and write only are simple, but RW might be tricky. I have to make sure the FILE handle of Pure C will be OK after my optimized fread() call.

Hans Wessels


Social Media

     

Return to “C / PASCAL etc.”

Who is online

Users browsing this forum: No registered users and 1 guest