Backward Compatible Refactoring

Nirum is designed to marshal well-typed data for networking. That means types defined in Nirum IDL are shared with other programs over the network. “Other programs” mean they usually aren’t deployed together. Even if some of your types defined in Nirum IDL was evolved and the changes were deployed to some programs that use them, some other programs still works as the types were not changed yet until the changes are deployed to all involved programs together.

To help types defined in Nirum IDL to be possible to be evolved with backward compatibility, Nirum types provide several properties.

Behind names for old names

The most of identifiers in Nirum IDL have a facial name and a behind name. By default, these two names are the same. For example, the following two declarations are equivalent:

record money (decimal amount, currency currency);

record money/money (decimal amount/amount, currency currency/currency);

A facial name is a descriptive identifier for humans. Object codes generated by Nirum compiler have identifiers named after facial names.

On the other hand, a behind name is an arbitrary identifier for programs. JSON payloads produced by serializers (that are generated by Nirum compiler) have identifiers named after behind names.

The following declaration shows a record and its fields having behind names different from their facial names:

record point2d/point (
    float64 left/x,
    float64 top/y,
);

The above IDL becomes to the following Python code for example (although it approximates details):

class Point2d:
    def __init__(self, left: float, top: float) -> None:
        self.left = left
        self.top = top

For example, Point2d(left=1.23, top=4.56) value is serialized to the following JSON payload:

{
    "_type": "point",
    "x": 1.23,
    "y": 4.56
}

Note that the above JSON payload names its type and fields after the behind names like point, x and y, whereas the above Python code names its class and attributes after the facial names like Point2d, left and top.

Separating a facial name from a behind name is useful when to rename a misleading name (or an outdated name) to a more clear name (or an up-to-date name).

Type alias

Type aliases are not distinct. While it’s being compiled all type aliases are simply replaced by types they refer to. So every type alias doesn’t break backward compatibility. Feel free to define type aliases!

Unboxed type

Unboxed types can be used for semi-refactoring: you can make people to say things by a specific name in program codes, while leave programs to communicate each other in the way they’ve done.

The key difference between unboxed types and single-field records is that unboxed types don’t make additional structure on its inner type whereas records always adds object structure to wrap its fields.

For example, the following two types are seemingly equivalent to program codes and programmers:

unboxed meter (bigint);

record meter (bigint value);

However, they make different JSON payloads:

"123"

{"_type": "meter", "value": "123"}

This means unboxed types are indistinguishable from their inner types in JSON payloads, while in program codes they are distinct individual types.

Suppose we’ve represented distance bigint with assuming 1 means 1 meter.

service map-service (
    bigint find-distance(coord a, coord b),
);

As such scale/unit assumptions are error-prone, we want to encode them as distinct type without breaking backward compatibility. It’s the when unboxed types are useful:

unboxed meter (bigint);

service map-service (
    meter find-distance (coord a, coord b),
);

The above change doesn’t affect to JSON payloads the find-distance method returns, but in program codes we become able to deal with distance using meter type rather than primitive bigint type.

Removing a field

Any fields in payload that are unlisted in an interface definition are ignored by a deserializer. If you are going to remove an existing field you should deploy the newer version to a payload consumer first, and then a payload provider last.

Interchangeability of enum type and text

Enum types are represented as JSON strings as like text. If a field had been represented as text but there are only the limited number of values, it’s okay to be refactored to enum type.

Suppose the following record had represented gender as freeform text:

record person (
    text name,
    text gender,  // represented as texts e.g. "male", "female"
);

Although it had become text due to a design mistake, we still have a chance to make up for it. Because there are only few cases:

{"_type": "person", "name": "Jane Doe", "gender": "male"}

{"_type": "person", "name": "John Doe", "gender": "female"}

The following change is still mostly compatible with the previous revision:

enum gender = male | female | unknown;

record person (
    text name,
    gender gender,
);

Making field optional

An option type modifier (?) indicates a field can be null in JSON payloads. Fortunately making a field which had disallowed null to allow null does not require any structural changes of JSON payloads. In other words, every valid JSON payload that a field disallowing null accepts is also accepted by the same field but of null allowance.

On the other hand, when it comes to reversed, optional fields cannot be required without any compatible breakage. Also, if two programs communicate each other, one deserializing and receiving payloads has to be deployed before other one serializing and sending payloads.

Interchangeability of set and list

A set contains zero or more unique values of the same type, without order.

A list contains zero or more values of the same type, in an order, and allows duplicated values.

However, both set and list types are serialized to equally an array in JSON payloads. So a set field and a list type are interchangeable each other unless their element types are different.

(More accurately, a set and a list still might be interchangeable even if their element types are different since two element types may be interchangeable. For example, {[a]} and [{a}] are interchangeable since [a] and {a} are interchangeable, and a and a are interchangeable again.)

When a JSON array serialized from a set field is deserialized to a list, its order becomes arbitrary. A program accepting payloads have to deal with its lack of order if necessary.

When a JSON array serialized from a list field is deserialized to a set, the same values shown more than once are collapsed to unique values. Also, its order is not preserved.

Interchangeability of union type and record type

Sometimes we need to evolve an existing record type to be extended.

But when we change a record type to a union type, it breaks backward compatibility. Suppose we have a record type named name that looks like:

record name (text fullname);

An example of JSON serialized one would look like:

{
    "_type": "name",
    "fullname": "John Doe"
}

What if we need to be more sensible to culture-specific names? Now we decide to change it to a union:

union name
    = wastern-name (text first-name, text? middle-name, text last-name)
    | east-asian-name (text family-name, text given-name)
    | culture-agnostice-name (text fullname)
    ;

Since union types requires "_tag" field besides "_type" field when they are deserialized, data sent from the older programs becomes to break compatibility.

In order to make union types possible to deserialize existing record data (which lacks "_tag" field), we need to choose default tag for data lacking "_tag" field:

union name
    = wastern-name (text first-name, text? middle-name, text last-name)
    | east-asian-name (text family-name, text given-name)
    | default culture-agnostice-name (text fullname)
    ;

With a default tag, union types become possible to deserialize data lacking "_tag" field, and they are treated as an instance of the default tag. For example, where we have a payload data like:

{
    "_type": "name",
    "fullname": "John Doe"
}

It’s treated as equivalent to the following one:

{
    "_type": "name",
    "_tag": "culture_agnostic_name",
    "fullname": "John Doe"
}