Nullable class members in tl schema?

[dancojocaru2000]

How to determine from td_api.tl which members in a class are nullable?

Take this example:

//@description Contains a list of messages @total_count Approximate total count of messages found @messages List of messages; messages may be null
messages total_count:int32 messages:vector<message> = Messages;

Here, the documentation specifies that "messages may be null". But how can this be determined in a concrete manner? Languages like Rust, Kotlin, Swift insist in null safety, so anything that might be nullable must be declared accordingly. Unless there is a way to determine which members are nullable, I would have to mark everything as nullable, which will be quite a bad UX.

[levlam]

In incoming requests to TDLib any object field are allowed to be null. This is up to the application and may form a correct or an incorrect request depending on other parameters.

In updates and responses from TDLib for some object fields it is guaranteed that they are never null. Such fields are not marked as "may be null" in the documentation. So the correct way to add Nullable annotations is by parsing the schema and searching for "may be null" substring in the field documentation. For example, this is how it is done for our native Java bindings:

td/td/generate/TlDocumentationGenerator.php

Line 245 in d26cf8f

$may_be_null = stripos($info[$name], 'may be null') !== false;

The Nullable annotations are added along-side with the documentation in the generated code.

[dancojocaru2000]

So, using the JSON interface, for the above object, an user might send to tdlib:

{
  "@type": "messages",
  "total_count" : 0
}

But tdlib will always send to the user:

{
  "@type": "messages",
  "total_count" : 0,
  "messages": []
}

Therefore not requiring any special handling of null types since all the fields will be there with the correct type.

Did I get this right?

[levlam]

Any parameter can be omited when sending a request to TDLib. In that case null/false/0/'' will be used as parameter's value.

In responses and updates TDLib omits only null object fields. In the messages field only underlying objects may be null, not the field as the whole. So it could be only something like

{
  "@type": "messages",
  "total_count" : 2,
  "messages": [ null, null ]
}

Or for updateChatChatList it could be:

{
  "@type": "updateChatChatList",
  "chat_id" : 12345
}

[dancojocaru2000]

In the messages field only underlying objects may be null, not the field as the whole.

The documentation is quite ambiguous on this one. Since the field is called messages, "messages may be null" to me sounds like the field can be null.

[levlam]

"may be null" can refer only to an object and for them is written in the form "; may be null". But suggestions on documentation improvements are always welcome.

[dancojocaru2000]

So the search shall actually be for the documentation for a field ending in "; may be null".

Personally, I think the nullable annotation should have been part of the tl schema type (vector<message?> for example). Or in the case of messages:vector<message>, something like "; may contain null" would have been way more clear.

Checking for String.endsWith("; may be null") is fine by me, but without this issue it wasn't really clearly specified anywhere in, for example, this page or this page.

[levlam]

The search for "may be null" is on purpose, because sometimes arrays also can be annotated as allowed to have null objects.

Unfortunately, TL doesn't allow to mark nullable objects as such in an easy and short way, so we use the documentation comments instead for that.

This isn't mentioned anywhere, because automatic parsing of the TL-schema is far beyond the scope of the Getting started guide and the JSON client usage documentation. If you need a reference implementation of the td_api TL-scheme parsing, you can use

td/td/generate/TlDocumentationGenerator.php

Line 100 in d26cf8f

public function generate($tl_scheme_file, $source_file)

.

[dancojocaru2000]

The search for "may be null" is on purpose, because sometimes arrays also can be annotated as allowed to have null objects.

Makes sense. So "may be null" for object means the object itself may be null and for vector means the contents may be null. Would there be any other meanings of "may be null"?
I will look through the PHP if needed but if you know and could help that would be great.

This isn't mentioned anywhere, because automatic parsing of the TL-schema is far beyond the scope of the Getting started guide and the JSON client usage documentation.

Understanding where something is null and where something isn't seems to me like it should be a part of the getting started, but then again, that might be just me.

[dancojocaru2000]

Also, I think this is basically solved at this point so I'll close the issue.

[levlam]

Makes sense. So "may be null" for object means the object itself may be null and for vector means the contents may be null. Would there be any other meanings of "may be null"?

This is absolutely correct.

Understanding where something is null and where something isn't seems to me like it should be a part of the getting started, but then again, that might be just me.

This is a part of the documentation along-side other restrictions like The first name of the user; 1-64 characters. Fields nullability is not beyond the basic documentation, but that the notation is parser-friendly is knowledge needed by a very few developers, to it is not documented.