Undefined behaviour C compilers (was: Pointer type for ST video addresses?)

C and PASCAL (or any other high-level languages) in here please

Moderators: exxos, simonsunnyboy, Mug UK, Zorro 2, Moderator Team

User avatar
Mikefulton
Captain Atari
Captain Atari
Posts: 169
Joined: Sun Nov 29, 2015 10:27 am

Undefined behaviour C compilers (was: Pointer type for ST video addresses?)

Postby Mikefulton » Thu Feb 11, 2016 9:51 pm

mfro wrote:Strictly speaking, the union approach is violating the C standard (although it usually works as intended).

The standard says that you are only allowed to read from one of the accessors if you've written through the same before. Writing through the void * pointer followed by a read through the "bytes" accessor is explicitely stated undefined behaviour (so your compiler is free to reboot your machine or release an H-bomb somewhere if it wants to).


I always thought that bit about unions was kinda missing the point of having them in the first place... it's like, "Hey, you know that thing you wanna do with a union that is realistically the biggest reason you'd ever want to be using a union in the first place? Yeah, don't do that."

User avatar
mfro
Atari Super Hero
Atari Super Hero
Posts: 681
Joined: Thu Aug 02, 2012 10:33 am
Location: SW Germany

Re: Pointer type for ST video addresses?

Postby mfro » Sun Feb 14, 2016 6:51 am

Mikefulton wrote:
mfro wrote:Strictly speaking, the union approach is violating the C standard (although it usually works as intended).

The standard says that you are only allowed to read from one of the accessors if you've written through the same before. Writing through the void * pointer followed by a read through the "bytes" accessor is explicitely stated undefined behaviour (so your compiler is free to reboot your machine or release an H-bomb somewhere if it wants to).


I always thought that bit about unions was kinda missing the point of having them in the first place... it's like, "Hey, you know that thing you wanna do with a union that is realistically the biggest reason you'd ever want to be using a union in the first place? Yeah, don't do that."


Yes and no.

Found that annoying as well when I first saw it (especially since original K&R C proposed this as valid usage, IIRC, and you still see a lot of code using it eg. as a convenient way for endianess swaps).

But in the end, many of the more recent (and more restrictive) ANSI C specs have been done to allow compilers for more aggressive code optimizations. If that's really the case here (I don't know), I'd consider that a small loss for a bigger gain.

User avatar
Mikefulton
Captain Atari
Captain Atari
Posts: 169
Joined: Sun Nov 29, 2015 10:27 am

Re: Pointer type for ST video addresses?

Postby Mikefulton » Sun Feb 14, 2016 10:07 am

Given that enforcing the spec would require runtime code that wouldn't be needed otherwise, it seems unlikely that facilitating better optimization was part of the motivation.

User avatar
Eero Tamminen
Atari God
Atari God
Posts: 1558
Joined: Sun Jul 31, 2011 1:11 pm

Re: Pointer type for ST video addresses?

Postby Eero Tamminen » Sun Feb 14, 2016 12:14 pm

Mikefulton wrote:Given that enforcing the spec would require runtime code that wouldn't be needed otherwise, it seems unlikely that facilitating better optimization was part of the motivation.


That's not how it works. Compiler doesn't "enforce" the spec, it expects code to be valid according to spec and does optimizations based on that. If code doesn't confirm to spec, it's undefined and that's also what you get as result, at least on higher optimization levels.

More information here:
* https://en.wikipedia.org/wiki/Undefined_behavior
* http://blog.llvm.org/2011/05/what-every ... -know.html
* http://blog.llvm.org/2011/05/what-every ... ow_14.html
* http://blog.llvm.org/2011/05/what-every ... ow_21.html

The points in LLVM blog are valid for all newer & better optimizing compilers (Clang, GCC from 3.x upwards...).

GCC offers options with which you can disable some of the expectations on spec correctness, the compiler has for the code it's compiling. I think most commonly used disabling option is "-fno-strict-aliasing". One should use it if one doesn't want to fix code doing pointer aliasing with different types, and one is using -O2 or higher (GCC) optimization level.

There are also options for getting warnings about source code issues with these expectations. When checking for warnings, use highest optimization level because that determines how throughly the compiler analyzes the code for issues.

User avatar
shoggoth
Nature
Nature
Posts: 854
Joined: Tue Aug 01, 2006 9:21 am
Location: Halmstad, Sweden
Contact:

Re: Pointer type for ST video addresses?

Postby shoggoth » Sun Feb 14, 2016 12:56 pm

You say tomato, I say tomato.

Use common sense; if this is Atari-specific code, the alternative (since we're usually short on performance) is often to use some inline assembly etc. I'm much more careful if I write things for other platforms, or things that need to be bomb proof on multiple architectures/compilers.
Ain't no space like PeP-space.

User avatar
Mikefulton
Captain Atari
Captain Atari
Posts: 169
Joined: Sun Nov 29, 2015 10:27 am

Re: Pointer type for ST video addresses?

Postby Mikefulton » Mon Feb 15, 2016 12:23 am

Eero Tamminen wrote:
Mikefulton wrote:Given that enforcing the spec would require runtime code that wouldn't be needed otherwise, it seems unlikely that facilitating better optimization was part of the motivation.


That's not how it works. Compiler doesn't "enforce" the spec, it expects code to be valid according to spec and does optimizations based on that. If code doesn't confirm to spec, it's undefined and that's also what you get as result, at least on higher optimization levels.

More information here:
* https://en.wikipedia.org/wiki/Undefined_behavior
* http://blog.llvm.org/2011/05/what-every ... -know.html
* http://blog.llvm.org/2011/05/what-every ... ow_14.html
* http://blog.llvm.org/2011/05/what-every ... ow_21.html

The points in LLVM blog are valid for all newer & better optimizing compilers (Clang, GCC from 3.x upwards...).

GCC offers options with which you can disable some of the expectations on spec correctness, the compiler has for the code it's compiling. I think most commonly used disabling option is "-fno-strict-aliasing". One should use it if one doesn't want to fix code doing pointer aliasing with different types, and one is using -O2 or higher (GCC) optimization level.

There are also options for getting warnings about source code issues with these expectations. When checking for warnings, use highest optimization level because that determines how throughly the compiler analyzes the code for issues.


First, it's simply not true that compilers don't have the option of enforcing the spec. I would agree that not checking is probably the default behavior in most cases, but it's also true that some compilers have options to enable runtime checking of certain things that otherwise don't get checked, sort of like having compiler-generated ASSERT statements.

Second, because it says "the result is undefined" that means the compiler is free to do whatever it wants regarding optimization anyway. Therefore, the only way in which optimization could possibly be affected would be if there was runtime enforcement.

User avatar
mfro
Atari Super Hero
Atari Super Hero
Posts: 681
Joined: Thu Aug 02, 2012 10:33 am
Location: SW Germany

Re: Pointer type for ST video addresses?

Postby mfro » Mon Feb 15, 2016 7:37 am

Mikefulton wrote:First, it's simply not true that compilers don't have the option of enforcing the spec. I would agree that not checking is probably the default behavior in most cases, but it's also true that some compilers have options to enable runtime checking of certain things that otherwise don't get checked, sort of like having compiler-generated ASSERT statements.

Second, because it says "the result is undefined" that means the compiler is free to do whatever it wants regarding optimization anyway. Therefore, the only way in which optimization could possibly be affected would be if there was runtime enforcement.


Honestly, I don't get your point. Maybe its because I'm kind of "gcc-biased" (since this is the compiler I use most) but there is nothing to enforce any standard. At higher optimization levels, the compiler just acts to these two rules:

  • "if the result of something is undefined, we can optimize away the whole something since this results in the same thing (something undefined)".
  • "its safe to assume the coder does not rely on undefined behaviour"
There is a nice real-world example (that led to a Linux kernel vulnerability) in one of the articles Eero linked that goes like this (I do not use our union example here since I just found out that gcc apparently explicitly allows this as an extension):

Code: Select all

void assign_if_not_null(int *ip)
{
    int value = *ip;
   
    if (ip == NULL)
    {
        do_something(&ip);  /* ensure ip points to something meaningful */
    }
    *ip = 4;
}


The first (innocent looking) line in the function was left there by accident.

The standard says that dereferencing a NULL pointer is undefined behaviour. Gcc will not enforce any standard here, it will just assume that since you already dereferenced the pointer, it can never be NULL and optimise away the following call resulting in this:

Code: Select all

void assign_if_not_null(int *ip)
{
    *ip = 4;
}


which is most likely not what one intended. The code will work fine without optimisation, but will break otherwise.

User avatar
Mikefulton
Captain Atari
Captain Atari
Posts: 169
Joined: Sun Nov 29, 2015 10:27 am

Re: Undefined behaviour C compilers

Postby Mikefulton » Mon Feb 15, 2016 9:16 am

mfro wrote:
Mikefulton wrote:First, it's simply not true that compilers don't have the option of enforcing the spec. I would agree that not checking is probably the default behavior in most cases, but it's also true that some compilers have options to enable runtime checking of certain things that otherwise don't get checked, sort of like having compiler-generated ASSERT statements.

Second, because it says "the result is undefined" that means the compiler is free to do whatever it wants regarding optimization anyway. Therefore, the only way in which optimization could possibly be affected would be if there was runtime enforcement.


Honestly, I don't get your point. Maybe its because I'm kind of "gcc-biased" (since this is the compiler I use most) but there is nothing to enforce any standard.


OK, I'll try this again...

First, to be clear, I never said unions weren't a potential source of huge weird problems.

Second, I didn't mean to suggest that either GCC or other compilers besides GCC commonly implemented any sort of runtime enforcement for union member access rules.

Third, you suggested that the bit about unions was in the spec to "allow compilers for more aggressive code optimizations". I disagree, because I can't see any way in which the code generation for accessing a union member could be impacted one way or the other by this, if there's no runtime enforcement. Accessing a union member is basically the same thing as accessing a structure member and it's going to end up being one or two instructions of machine code at the end of the compilation. Not really much room for more optimization there.

Fourth, I tried to make the point that if there was runtime enforcement, even as a special compiler option, the extra code involved would mean that optimization was a bigger concern in many respects. The point here being that this is the only condition in which the idea that the bit in the language spec was motivated by optimization concerns would make any sense.

Fifth, many compilers have options to enable additional runtime checking of various conditions, as a means of testing your code and trying to shake out weird runtime problems that otherwise might be hard to figure out. I don't actually know of any specific compiler that could check this particular thing about union member access, but there are a lot of compilers out there and most of them aren't GCC.

Sixth, just as an observation, it doesn't sound like you're GCC-biased so much as maybe just not experienced with many other compilers? There's actually a fair number of them out there, with different strengths and weaknesses. (The fact that they're not free being the biggest weakness for many GCC fans.)

User avatar
Eero Tamminen
Atari God
Atari God
Posts: 1558
Joined: Sun Jul 31, 2011 1:11 pm

Re: Undefined behaviour C compilers

Postby Eero Tamminen » Mon Feb 15, 2016 11:11 pm

Mikefulton wrote:Third, you suggested that the bit about unions was in the spec to "allow compilers for more aggressive code optimizations". I disagree, because I can't see any way in which the code generation for accessing a union member could be impacted one way or the other by this, if there's no runtime enforcement.


Unions are there so that you can tell the compiler that variable may be aliased to a different type. I.e. compiler then knowns that it can *NOT* legally do certain optimizations for them, which it might do without the union. Not the other way round.

Mikefulton wrote:Accessing a union member is basically the same thing as accessing a structure member and it's going to end up being one or two instructions of machine code at the end of the compilation. Not really much room for more optimization there.


Optimization can be more extensive, e.g. just skipping the access and code surrounding it completely. The kind of line-by-line conversion of C-code of which you seem to be thinking of, happens only when most compiler optimizations are disabled... At least if we're talking about still relevant compilers from this century.

Mikefulton wrote:Fifth, many compilers have options to enable additional runtime checking of various conditions, as a means of testing your code and trying to shake out weird runtime problems that otherwise might be hard to figure out. I don't actually know of any specific compiler that could check this particular thing about union member access, but there are a lot of compilers out there and most of them aren't GCC.


Both GCC and LLVM have nowadays run-time instrumentation / checking support for certain undefined behavior issues, in addition to compile time static analysis checks:
* http://clang.llvm.org/docs/UndefinedBeh ... tizer.html
* https://gcc.gnu.org/onlinedocs/gcc/Inst ... tions.html

But as far as I can see, they don't catch the issue we're talking about.

In typed languages like C, type (cast) checks are best done statically, at compile time. I don't think doing type checks at run-time is really practical in languages that don't store type information with the variables. If you want type info and run-time checking of type matches for pointer accesses, you should be using some other language.

Mikefulton wrote:Sixth, just as an observation, it doesn't sound like you're GCC-biased so much as maybe just not experienced with many other compilers? There's actually a fair number of them out there, with different strengths and weaknesses. (The fact that they're not free being the biggest weakness for many GCC fans.)


Which compiler(s) you're referring to, something not on this list:
https://en.wikipedia.org/wiki/List_of_c ... _compilers
?

User avatar
Mikefulton
Captain Atari
Captain Atari
Posts: 169
Joined: Sun Nov 29, 2015 10:27 am

Re: Undefined behaviour C compilers

Postby Mikefulton » Tue Feb 16, 2016 2:04 am

Eero Tamminen wrote:Unions are there so that you can tell the compiler that variable may be aliased to a different type. I.e. compiler then knowns that it can *NOT* legally do certain optimizations for them, which it might do without the union. Not the other way round.


Sigh... of course, the compiler takes unions into account when optimizing code. I never suggested otherwise.The point here is the suggestion that the part of the spec that talks about accessing union members was written that way specifically to facilitate particular optimizations. That's the part I disagree with. That's the only thing in this entire thread I've ever disagreed with, but people seem to be skipping right past that idea

User avatar
mfro
Atari Super Hero
Atari Super Hero
Posts: 681
Joined: Thu Aug 02, 2012 10:33 am
Location: SW Germany

Re: Undefined behaviour C compilers

Postby mfro » Tue Feb 16, 2016 8:26 am

Mikefulton wrote:... The point here is the suggestion that the part of the spec that talks about accessing union members was written that way specifically to facilitate particular optimizations...


It doesn't make sense to argue about that. The comitee members might have had optimization in mind, it's also very well possible they were unable to agree on something and it was just high time for dinner...

We'll probably never know for sure, so this is a dispute about nothing.

In the end, the whole concept of undefined behaviour in C is not (at least not solely) a weak, incomplete language definition, it _is_ about performance optimization. It allows compilers to just ignore dubious corner cases that would otherwise hurt performance.

I definitely agree that the standards definition appears a little "taken away" with that.

In the gcc case (yes, I do in fact work with gcc most of the time, even at work, where we decided to go that way not because it's free, but merely inspite of it), this discussion is definitely pointless. gcc appears to allow alternating union member access as an extension to the standard, so it seems to be safe to use it for Atari code.

User avatar
Mikefulton
Captain Atari
Captain Atari
Posts: 169
Joined: Sun Nov 29, 2015 10:27 am

Re: Undefined behaviour C compilers

Postby Mikefulton » Tue Feb 16, 2016 10:02 pm

mfro wrote:
Mikefulton wrote:... The point here is the suggestion that the part of the spec that talks about accessing union members was written that way specifically to facilitate particular optimizations...


It doesn't make sense to argue about that. The comitee members might have had optimization in mind, it's also very well possible they were unable to agree on something and it was just high time for dinner...

We'll probably never know for sure, so this is a dispute about nothing.


Nobody is more amazed than I am about what was spawned by what was intended as a little throw-away comment.


Social Media

     

Return to “C / PASCAL etc.”

Who is online

Users browsing this forum: No registered users and 2 guests