Such a recognisable name, greetz after many years :D You might remember me from uni as lycium, a few lifetimes ago. Going to check the rest of your blog now...
`_t` should not be used for custom types since it's reserved for future standard types (and/or types declared in a header you might include someday). This does cause real-world problems (`key_t` anyone?).
Gratuitous allocations are gratuitous.
The whole "prevent double free" claim is completely bogus. Setting a variable to `NULL` only works for cases where there is one, obvious, owner, which is not the circumstance under which double free is prone to happening in the first place. Actually preventing double free requires determining ownership of every object, and C sucks at that.
> `_t` should not be used for custom types since it's reserved for future standard types (and/or types declared in a header you might include someday).
That old thing again...
The _t postfix is only reserved in the POSIX standard, but not in the C standard (and C and POSIX are entirely different things - outside the UNIX bubble at least).
It's unlikely that POSIX changes anymore, but if you get a name collision in a new POSIX version it's still just a simple name collision, and it's up to the user code to fix that.
And it's not like symbol collision problems are limited to POSIX, the world won't end because some piece of C code collides with a symbol used in a dependency, there's always ways to isolate its usage.
Also, it's good practice in the C world to use a namespace prefix for libraries, and such a prefix will also make sure that any _t postfix will not collide with POSIX symbols (the question is of course why POSIX couldn't play nice with the rest of the world and use a posix_ prefix - historical reasons I guess, but then just going a ahead and squatting on the _t postfix for all eternity is a bit rich).
> A potentially reserved identifier becomes a reserved identifier when an implementation begins using it or a future
standard reserves it, but is otherwise available for use by the programmer.
Which, in practice, does mean using _t is likely to cause you problems, as it may become a reserved identifier, when an implementation like POSIX begins using it.
Still debatable since the C standard doesn't reserve the _t postfix (it does reserve a single leading underscore followed by a capital letter, e.g. _Bool, and IIRC it also reserves two leading underscores).
What POSIX reserves or doesn't reserve doesn't affect code that follows only the C standard but doesn't care about POSIX compatibility, and especially _t is so widely used in C libraries that POSIX's opinion obviously doesn't matter all that much in the real world.
Whilst you do point to 6.4.3 for where it does reserve "All identifiers that begin with a double underscore (__) or begin with an underscore (_) followed by an uppercase letter"... That section also has the lovely:
> Other identifiers may be reserved.
If an implementation of C uses it... Just... Don't. The standard won't save you here, because it's happy for an implementation to do whatever they feel like.
For the _t suffix it is deeply unfortunate that on the one hand C's standard library gives people this idea but then the standard says they mustn't use it.
I understand exactly why it was necessary, but to my mind that highlighted an urgent need to provide actual namespacing so that we don't need to rope off whole categories of identifiers for exclusive use by the stdlib, with the implication that every single library will need to do the same. This should have been addressed last century IMO.
> The whole "prevent double free" claim is completely bogus.
"Completely" means "for all". Are you seriously claiming that "for all instances of double-free, setting the pointer to NULL after freeing it would not help"?
Not in the case of bogosity. Completely bogus things might occasionally work under some very particular circumstances, but unless those particular circumstances just happen to be the circumstances you actually care about, complete bogosity can still obtain.
> setting the pointer to NULL
There is no such thing as setting a pointer to null. You can set the value of a variable (whose current value is a pointer) to null, but you cannot guarantee that there isn't a copy of the pointer stored somewhere else except in a few very particular circumstances. This is what the GP meant by "setting a variable to `NULL` only works for cases where there is one, obvious, owner". And, as the GP also pointed out, this "is not the circumstance under which double free is prone to happening in the first place." Hence: complete bogosity.
I've been shouting this from "the rooftops" ever since I eventually independently had come to the same conclusion through sufficient volume of experience and empiry. I consider it a small tragedy, however, that my words tend to fall on deaf ears as I try to explain to e.g. co-workers why they should, in fact, parse their input and use the language's type system to their advantage, vs. all the `validateThis` and `validateThat` they seem to be copying from e.g. Stack Overflow. I see them stumbling and breaking their nose time and again passing URIs around as strings -- even in languages other than C (so this isn't obviously a problem C created and needs to solve alone) -- where it's never enforced that the URI is e.g. "absolute" or "relative" (nevermind the fact these people often don't seem to care for the difference). They do the same error with date and time (how many decades of advice do we need to accumulate and impart on our younger peers for them to irrevocably understand why e.g. date and time should never be passed around as numbers or strings, with very clearly documented and justified exceptions?). But the tragedy is never more evident when the person looks at you like you're suddenly talking Klingon to them and how bored they are with the notion and how it cannot be so important as to disturb their very important "coding project" (we don't write Knuth's "literal" programs any more -- we _code_).
The trouble I have with this approach (which, conceptually, I agree with) is that it's damned hard to do anything with the parse results. Want to print that email_t? Then you're right back to char*, unless you somehow write your own I/O system that knows about your opaque conventions.
So you say, okay, I'll make an `email_to_string` function. Does it return a copy or a reference? Who frees it? etc, etc, and you're back to square one again. The idea is to keep char* and friends at "the edge", but I've never found a way to really achieve that.
Could just be my limitations as a C programmer, in which case I'd be thrilled to learn better.
In the past I've taken inspiration from strncpy: the caller needs to allocate the memory. For the email example, you'd probably also want a function to tell you the length of the emailstring, but for other types there are clear size limits. This puts the caller in control of memory allocation, so they may be able to statically allocate, allocate in an arena, or use other methods which promote performance. The static approach is really nice when it works, because there's nothing to free.
Firstly, `parsing` is just a way to say "serialise from a string". The reverse operation can be done for every type you are creating. If the reverse operation (serialise to a string) does not exist in the interface then adding it gives you a single place to catch all the bugs.
I'm thinking of that recent git bug that occurred because the round-trip of
`string -> type -> string` had an error (stripping out the CR character). Using a specific type for a value that is being round-tripped means that a bugfix needs to only be made in the parser function. Storing the value as simple strings would result in needing to put your fix everywhere.
> The trouble I have with this approach (which, conceptually, I agree with) is that it's damned hard to do anything with the parse results.
You're right - it is damn hard, but that is on purpose; if you're doing something with the email that boils down to "treat it like a `char *`" then the potential for error is large.
If you're forced to add in a new use-case to the `email_t` interface then you have reduced the space of potential errors.
For example:
> Want to print that email_t? Then you're right back to char, unless you somehow write your own I/O system that knows about your opaque conventions.
is a bug waiting to surface, because it's an email, not a string, and if you decide to print an email* that was read as a `char *` you might not get what you expect.
It's all a trade-off - if you want more flexibility with the value stored in a variable, then sure, you can have it but it comes at a cost: some code somewhere almost certainly will eventually use that flexibility to mismatch the type!
If you want to prevent type mismatches, then a lot of flexibility goes out the window.
The main benefit for me with this approach is that the boundries are not transparent anymore. That content printing is such a boundry. Your data is about to exit through there and you're summoned to handle that. The inconvenience that comes with it is as any other when security enters the play. The same with the data management responsibilities - who handles what, for how long, and with whom. Without data type distinctions everything is (more or less) common, with vague or broadly defined ownership.
email_t doesn't have to be opaque; if it's just a visible wrapper around char* then you can still do everything with it as a char* (that is, everything you do with strings).
The benefit is to avoid treating char*s as email_t, not avoiding treating email_t as char*.
Python’s NewType is, confusingly, a very different thing: it’s a compile-time-only subtype of the original, rather than a Haskell-style newtype (which is an entirely separate type from its source).
In the example code they explicitly put the struct in the c file so the char* is not available.
If you're suggesting getting around this by casting an email_t* to char* then I wish you good luck on your adventures. There's some times you gotta do stuff like that but this ain't it.
You could probably get away with the typecast if you satisfy the "common struct prefix" requirement, that's nowhere near necessary.
While the article does hide the internal char*, that's not strictly necessary to get the benefit of "parse, don't validate". Hide implementation details sure, but not everything is an implementation detail.
> Here, I’ll build on that by showing how this technique can be used outside of niche academic languages by demonstrating it in a language that is as practical as it is dangerous - C.
The "practical" part really bugged me because the entire post is trying to explain exactly why it is not.
The only way to make C reasonably safe is to encode information via newtype pattern. Wrap `char *` inside a struct that have proper names and include the size in there as well.
Basically, there should be ZERO pointers except at creation and consumption by outside libraries (open, write, etc)
They write the non-pseudo variant later. There, the return value is a pointer and the check is against NULL. Which is fairly standard for C code, albeit not always desirable.
This pseudocode is "Validate" for at least 3 reasons:
Forgetting to check:
this check is fragile: it’s extremely easy to forget. Because its return value is unused, it can always be omitted, and the code that needs it would still typecheck.
Repeatable/redundant checks:
First, it’s just annoying. We already checked that the list is non-empty, why do we have to clutter our code with another redundant check?
Second, it has a potential performance cost. Although the cost of the redundant check is trivial in this particular example, one could imagine a more complex scenario where the redundant checks could add up, such as if they were happening in a tight loop.
Not using the type system:
Use a data structure that makes illegal states unrepresentable. Model your data using the most precise data structure you reasonably can. If ruling out a particular possibility is too hard using the encoding you are currently using, consider alternate encodings that can express the property you care about more easily. Don’t be afraid to refactor.
> How do you know PARSE_ERROR is not email_t
It has to be for it to compile, right? Which means that email_t is the type which represents both valid and invalid emails. How do you know if it's valid? You remember to write a check for it. Why not just save yourself some keystrokes and use char* instead.
This is validate, not parse.
> It has to be for it to compile, right? Which means that email_t is the type which represents both valid and invalid emails. How do you know if it's valid? You remember to write a check for it. Why not just save yourself some keystrokes and use char* instead. This is validate, not parse.
I feel this kind of fundamentalism is letting the perfect be the enemy of the good.
Every C programmer is already doing it the 'good' way (validation), so this article doesn't really add anything.
The only fundamentalism involved in PdV is: if you have an email, it's actually an email. It's not arbitrary data that may or may not an email.
Maybe you want your emailing methods to accept both emails and not-emails in your code base. Then it's up to each method to validate it before working on it. That is precisely what PdV warns against.
Such a recognisable name, greetz after many years :D You might remember me from uni as lycium, a few lifetimes ago. Going to check the rest of your blog now...
`_t` should not be used for custom types since it's reserved for future standard types (and/or types declared in a header you might include someday). This does cause real-world problems (`key_t` anyone?).
Gratuitous allocations are gratuitous.
The whole "prevent double free" claim is completely bogus. Setting a variable to `NULL` only works for cases where there is one, obvious, owner, which is not the circumstance under which double free is prone to happening in the first place. Actually preventing double free requires determining ownership of every object, and C sucks at that.
> `_t` should not be used for custom types since it's reserved for future standard types (and/or types declared in a header you might include someday).
That old thing again...
The _t postfix is only reserved in the POSIX standard, but not in the C standard (and C and POSIX are entirely different things - outside the UNIX bubble at least).
It's unlikely that POSIX changes anymore, but if you get a name collision in a new POSIX version it's still just a simple name collision, and it's up to the user code to fix that.
And it's not like symbol collision problems are limited to POSIX, the world won't end because some piece of C code collides with a symbol used in a dependency, there's always ways to isolate its usage.
Also, it's good practice in the C world to use a namespace prefix for libraries, and such a prefix will also make sure that any _t postfix will not collide with POSIX symbols (the question is of course why POSIX couldn't play nice with the rest of the world and use a posix_ prefix - historical reasons I guess, but then just going a ahead and squatting on the _t postfix for all eternity is a bit rich).
The C23 spec also says:
> A potentially reserved identifier becomes a reserved identifier when an implementation begins using it or a future standard reserves it, but is otherwise available for use by the programmer.
Which, in practice, does mean using _t is likely to cause you problems, as it may become a reserved identifier, when an implementation like POSIX begins using it.
Still debatable since the C standard doesn't reserve the _t postfix (it does reserve a single leading underscore followed by a capital letter, e.g. _Bool, and IIRC it also reserves two leading underscores).
What POSIX reserves or doesn't reserve doesn't affect code that follows only the C standard but doesn't care about POSIX compatibility, and especially _t is so widely used in C libraries that POSIX's opinion obviously doesn't matter all that much in the real world.
Whilst you do point to 6.4.3 for where it does reserve "All identifiers that begin with a double underscore (__) or begin with an underscore (_) followed by an uppercase letter"... That section also has the lovely:
> Other identifiers may be reserved.
If an implementation of C uses it... Just... Don't. The standard won't save you here, because it's happy for an implementation to do whatever they feel like.
For the _t suffix it is deeply unfortunate that on the one hand C's standard library gives people this idea but then the standard says they mustn't use it.
I understand exactly why it was necessary, but to my mind that highlighted an urgent need to provide actual namespacing so that we don't need to rope off whole categories of identifiers for exclusive use by the stdlib, with the implication that every single library will need to do the same. This should have been addressed last century IMO.
> an urgent need to provide actual namespacing
Some newer parts of the standard library use a stdc_ prefix now (https://en.cppreference.com/w/c/numeric/bit_manip.html).
> The whole "prevent double free" claim is completely bogus.
"Completely" means "for all". Are you seriously claiming that "for all instances of double-free, setting the pointer to NULL after freeing it would not help"?
> "Completely" means "for all".
Not in the case of bogosity. Completely bogus things might occasionally work under some very particular circumstances, but unless those particular circumstances just happen to be the circumstances you actually care about, complete bogosity can still obtain.
> setting the pointer to NULL
There is no such thing as setting a pointer to null. You can set the value of a variable (whose current value is a pointer) to null, but you cannot guarantee that there isn't a copy of the pointer stored somewhere else except in a few very particular circumstances. This is what the GP meant by "setting a variable to `NULL` only works for cases where there is one, obvious, owner". And, as the GP also pointed out, this "is not the circumstance under which double free is prone to happening in the first place." Hence: complete bogosity.
I've been shouting this from "the rooftops" ever since I eventually independently had come to the same conclusion through sufficient volume of experience and empiry. I consider it a small tragedy, however, that my words tend to fall on deaf ears as I try to explain to e.g. co-workers why they should, in fact, parse their input and use the language's type system to their advantage, vs. all the `validateThis` and `validateThat` they seem to be copying from e.g. Stack Overflow. I see them stumbling and breaking their nose time and again passing URIs around as strings -- even in languages other than C (so this isn't obviously a problem C created and needs to solve alone) -- where it's never enforced that the URI is e.g. "absolute" or "relative" (nevermind the fact these people often don't seem to care for the difference). They do the same error with date and time (how many decades of advice do we need to accumulate and impart on our younger peers for them to irrevocably understand why e.g. date and time should never be passed around as numbers or strings, with very clearly documented and justified exceptions?). But the tragedy is never more evident when the person looks at you like you're suddenly talking Klingon to them and how bored they are with the notion and how it cannot be so important as to disturb their very important "coding project" (we don't write Knuth's "literal" programs any more -- we _code_).
The trouble I have with this approach (which, conceptually, I agree with) is that it's damned hard to do anything with the parse results. Want to print that email_t? Then you're right back to char*, unless you somehow write your own I/O system that knows about your opaque conventions.
So you say, okay, I'll make an `email_to_string` function. Does it return a copy or a reference? Who frees it? etc, etc, and you're back to square one again. The idea is to keep char* and friends at "the edge", but I've never found a way to really achieve that.
Could just be my limitations as a C programmer, in which case I'd be thrilled to learn better.
In the past I've taken inspiration from strncpy: the caller needs to allocate the memory. For the email example, you'd probably also want a function to tell you the length of the emailstring, but for other types there are clear size limits. This puts the caller in control of memory allocation, so they may be able to statically allocate, allocate in an arena, or use other methods which promote performance. The static approach is really nice when it works, because there's nothing to free.
Firstly, `parsing` is just a way to say "serialise from a string". The reverse operation can be done for every type you are creating. If the reverse operation (serialise to a string) does not exist in the interface then adding it gives you a single place to catch all the bugs.
I'm thinking of that recent git bug that occurred because the round-trip of `string -> type -> string` had an error (stripping out the CR character). Using a specific type for a value that is being round-tripped means that a bugfix needs to only be made in the parser function. Storing the value as simple strings would result in needing to put your fix everywhere.
> The trouble I have with this approach (which, conceptually, I agree with) is that it's damned hard to do anything with the parse results.
You're right - it is damn hard, but that is on purpose; if you're doing something with the email that boils down to "treat it like a `char *`" then the potential for error is large.
If you're forced to add in a new use-case to the `email_t` interface then you have reduced the space of potential errors.
For example:
> Want to print that email_t? Then you're right back to char, unless you somehow write your own I/O system that knows about your opaque conventions.
is a bug waiting to surface, because it's an email, not a string, and if you decide to print an email* that was read as a `char *` you might not get what you expect.
It's all a trade-off - if you want more flexibility with the value stored in a variable, then sure, you can have it but it comes at a cost: some code somewhere almost certainly will eventually use that flexibility to mismatch the type!
If you want to prevent type mismatches, then a lot of flexibility goes out the window.
The main benefit for me with this approach is that the boundries are not transparent anymore. That content printing is such a boundry. Your data is about to exit through there and you're summoned to handle that. The inconvenience that comes with it is as any other when security enters the play. The same with the data management responsibilities - who handles what, for how long, and with whom. Without data type distinctions everything is (more or less) common, with vague or broadly defined ownership.
email_t doesn't have to be opaque; if it's just a visible wrapper around char* then you can still do everything with it as a char* (that is, everything you do with strings).
The benefit is to avoid treating char*s as email_t, not avoiding treating email_t as char*.
(Using a thin wrapper like this to add safety is called the newtype pattern, if anyone wants to know.)
I was curious how this would look in C, and I found this article[1] how this could look in C, apparently with very little overhead.
And as I just saw, Python 3.10 also introduced a NewType[2] wrapper. I'll have to see how that feels to handle.
1: https://blog.nelhage.com/2010/10/using-haskells-newtype-in-c...
2: https://typing.python.org/en/latest/spec/aliases.html#newtyp...
Python’s NewType is, confusingly, a very different thing: it’s a compile-time-only subtype of the original, rather than a Haskell-style newtype (which is an entirely separate type from its source).
In the example code they explicitly put the struct in the c file so the char* is not available.
If you're suggesting getting around this by casting an email_t* to char* then I wish you good luck on your adventures. There's some times you gotta do stuff like that but this ain't it.
You could probably get away with the typecast if you satisfy the "common struct prefix" requirement, that's nowhere near necessary.
While the article does hide the internal char*, that's not strictly necessary to get the benefit of "parse, don't validate". Hide implementation details sure, but not everything is an implementation detail.
> Here, I’ll build on that by showing how this technique can be used outside of niche academic languages by demonstrating it in a language that is as practical as it is dangerous - C.
The "practical" part really bugged me because the entire post is trying to explain exactly why it is not.
The only way to make C reasonably safe is to encode information via newtype pattern. Wrap `char *` inside a struct that have proper names and include the size in there as well.
Basically, there should be ZERO pointers except at creation and consumption by outside libraries (open, write, etc)
validating twice is safe. it will not error the second time.
it is against the rules to call someone dumb on this server.
This stuck out:
An email_t is not a parse error, and a parse error is not one of the emails, so this shouldn't compile (and I don't take 'pseudocode' as an excuse).> and I don't take 'pseudocode' as an excuse
They write the non-pseudo variant later. There, the return value is a pointer and the check is against NULL. Which is fairly standard for C code, albeit not always desirable.
Correct, it is fairly standard C code. It is not Parse, Don't Validate.
> and I don't take 'pseudocode' as an excuse
Weird hill to die on, since neither email_t nor PARSE_ERROR were defined in the sample snippets. How do you know PARSE_ERROR is not email_t?
It's the parse-versus-validate hill in this case.
This pseudocode is "Validate" for at least 3 reasons:
Forgetting to check:
Repeatable/redundant checks: Not using the type system: > How do you know PARSE_ERROR is not email_tIt has to be for it to compile, right? Which means that email_t is the type which represents both valid and invalid emails. How do you know if it's valid? You remember to write a check for it. Why not just save yourself some keystrokes and use char* instead. This is validate, not parse.
> It has to be for it to compile, right? Which means that email_t is the type which represents both valid and invalid emails. How do you know if it's valid? You remember to write a check for it. Why not just save yourself some keystrokes and use char* instead. This is validate, not parse.
I feel this kind of fundamentalism is letting the perfect be the enemy of the good.
Every C programmer is already doing it the 'good' way (validation), so this article doesn't really add anything.
The only fundamentalism involved in PdV is: if you have an email, it's actually an email. It's not arbitrary data that may or may not an email.
Maybe you want your emailing methods to accept both emails and not-emails in your code base. Then it's up to each method to validate it before working on it. That is precisely what PdV warns against.
You don't think there's a degree of difference between (valid email_t or null) and (valid char pointer or invalid char pointer)?
Because an error is not an email?
By that logic, a float couldn't store NaN.
Correct. You'll never see a raw float on the LHS of a PDV expression.
Who puts their personal data vault on the left hand side of a raw float?
[dead]
I'm with you, don't do crap like that. Always return a valid object.
This is validate.
You made an email-or-error type and named it email_t and then manually checked it.
PDV returns an non-error-email type from the check method.