debuginfo: How to (ideally) represent reference and pointer types in DWARF

[michaelwoerister]

Currently, we represent thin references and pointers with DW_TAG_pointer_type DIEs and fat pointers (slices and trait objects) as DW_TAG_struct DIEs with fields representing payload and metadata pointers. This is not ideal and with debuggers knowing about Rust, we can do better. The question is, what exactly do we want the representation for these kinds of types to look like.

Some things seem pretty straightforward to me:

• Rust references should be DW_TAG_reference_type DIEs.
• Rust raw pointers should be DW_TAG_pointer_type DIEs.

But beyond that, there are some decisions to be made:

(1) How do we represent mutability?

The C++ version of DWARF represents a const pointer like const char * with three separate type entries:

0:
DW_TAG_base_type
    DW_AT_name "char"
    ... 

1:
DW_TAG_const_type
    DW_AT_type: ref to <0>

2:
DW_TAG_pointer_type
    DW_AT_type: ref to <1>

I think this is a bit verbose and I'm not sure it is entirely appropriate for Rust. Do we really have const and mut types? That is, does Rust have the concept of a mut i32 at the type level, for example? I mean there are mutable and immutable slots/memory locations and we have "mutable" and "shared" references, but those two things seem kind of different to me.

As an alternative to using DW_TAG_const_type for representing mutability, we could re-use the DW_AT_mutable attribute that is already defined in DWARF. In C++ DWARF it is used for mutable fields. We could use it for reference type and local variable DIEs:

0: // char
DW_TAG_base_type
    DW_AT_name "char"
    ... 

1: // &mut char
DW_TAG_reference_type
    DW_AT_type: ref to <0>
    DW_AT_mutable: true

2: // &char
DW_TAG_reference_type
    DW_AT_type: ref to <0>
    DW_AT_mutable: false       // or just leave it off

3: 
DW_TAG_variable
    DW_AT_name: "foo"
    DW_AT_type: ref to <0>
    DW_AT_mutable: true
    ...

(2) How to represent fat pointers?

The pointer types in C/C++ DWARF don't have DW_TAG_member sub-DIEs, since they are always just values. Fat pointers in Rust are different: they have one field that is a pointer to the data, and another field that holds additional information, either the size of a slice or the pointer to a vtable. These need to be described somehow.
I see a few options:

1. A fat-pointer type is described by a DW_TAG_pointer_type or DW_TAG_reference_type DIE with two fields that are described by DW_TAG_member sub-DIEs, both having the DW_AT_artificial attribute. @tromey once suggested for slices that the field entries have no name and the debugger determines which is which by the type (the size is always an integer type, the data is always a pointer type). This could also be extended for trait objects, since the data pointer will always be a pointer to a trait and the vtable-pointer will always be something else.
2. Treat trait objects and slices differently. Have a new DW_TAG_slice_type DIE that follows the encoding above and borrow some other attributes for trait objects: a DW_AT_vtable_elem_location attribute holds the offset of the vtable field within the fat-pointer value, and a DW_AT_object_pointer attribute does the same for the data pointer. This is distinctly not how these attributes are used in a C++ context but it would be a nice fit, I think.
3. Mix of the above with DW_AT_object_pointer indicating data pointer field

Another questions is: Should fat-pointers (and thin pointers too, maybe) have a DW_AT_byte_size attribute that specifies their size explicitly?

cc @tromey, @Manishearth
See also #33073

[Manishearth]

That is, does Rust have the concept of a mut i32 at the type level, for example?

No. mut is part of reference types, no more. I like your DW_AT_mutable solution, except we would only include that attribute for DW_TAG_reference_types (and possibly only when it is on). Otherwise we have the possiblility of mut i32 being a type.

How to represent fat pointers?

I like both solution (1) and (2), no clear preference.

Beware that traits and slices aren't the only things that can be DSTd, you also have struct MyDST(u8, u8, u8, [u8]). This is similar to a slice DST but with different offsets. In such cases you also want to generate field names and other debuginfo for the DST itself (unlike slices and traits where we interpret them specially). I think we can handle this somehow by doing whatever C does for trailing T[] fields and generating debuginfo structs for those.

[tromey]

As an alternative to using DW_TAG_const_type for representing mutability, we could re-use the DW_AT_mutable attribute that is already defined in DWARF.

This seems reasonable to me.

[Manishearth]

I think we can handle this somehow by doing whatever C does for trailing T[] fields and generating debuginfo structs for those.

C seems to use a TAG_array_type with a TAG_subrange_type that doesn't have an AT_count. Should work for us.

[tromey]

@tromey once suggested for slices that the field entries have no name and the debugger determines which is which by the type (the size is always an integer type, the data is always a pointer type).

For fat pointers I was thinking DW_AT_vtable_elem_location would be required. I'd rather the decoding be based on DWARF attributes. (If this concept can be extended to slices, so much the better; I was being inconsistent here.)

On the whole I'd rather the output be much more explicit. That is, instead of determining whether something is a slice by checking its name, introduce DW_TAG_slice_type. Similarly, have a separate tag to represent a trait object -- don't try to reuse whatever is being done for slices. I think it's fine if existing DWARF tags are repurposed in a rust-specific way; but I find it a bit ugly if the tags have their meaning overloaded and require the debugger to encode "excess" knowledge of rust to do its job.

Another questions is: Should fat-pointers (and thin pointers too, maybe) have a DW_AT_byte_size attribute that specifies their size explicitly?

Thin pointers, probably not. Fat pointers, sure. Ideally it should be possible for gdb to create a trait object; though of course we're still a ways away from that.

[tromey]

Beware that traits and slices aren't the only things that can be DSTd

What does "DSTd" mean?

[michaelwoerister]

What does "DSTd" mean?

I assume "turned into a dynamically sized type".

[tromey]

I assume "turned into a dynamically sized type".

Aha, thanks. For dynamically sized types, we should just use the standard DWARF stuff. If the length is known then DW_AT_count can be a location expression (there are other similar ways as well).

[nagisa]

What are semantics of DW_TAG_reference_type and how do they differ from the pointer_type?

[tromey]

What are semantics of DW_TAG_reference_type and how do they differ from the pointer_type?

DWARF doesn't go into great detail here, but basically the answer is "just like C++". However, in DWARF it is also normal to reuse tags for different things depending on the CU's language; and for Rust I think the obvious answer is that a DW_TAG_pointer_type should be used for raw pointer types and DW_TAG_reference_type for ordinary (safe) references. So, a pointer might be null or invalid; but a reference will not be (with the usual caveat that debuggers can sometimes see uninitialized objects, memory can be trashed, etc).

[michaelwoerister]

Beware that traits and slices aren't the only things that can be DSTd, you also have struct MyDST(u8, u8, u8, [u8]).

That's a good point. In a way a regular slice is just a special case of a struct with a trailing [T]. Also (as opposed to C?) in Rust we always know the length of the trailing [T] because of the fat-pointer field.

So, we have this:

kind	fat
&T (regular reference)	no
&[T] (regular slice)	yes
&Struct /w trailing [T]	yes
&Trait (regular trait object)	yes
&Struct /w trailing trait	yes

I think it would be nice if &[T] and &Struct /w trailing [T] would be represented the same way, and correspondingly also &Trait and &Struct /w trailing trait, since in both cases the former is just a special case of the latter (kind of).

Also note there are is also a pointer variant for each of the above. If we have different tags for each of those, we would get quite a few tags:

DW_TAG_reference_type        // &T (regular reference)
DW_TAG_slice_type            // &[T] (regular slice)
DW_TAG_dst_reference_type    // &Struct /w trailing [T]
DW_TAG_trait_object          // &Trait (regular trait object)
DW_TAG_trait_object2         // &Struct /w trailing trait

DW_TAG_pointer_type          // *T (regular pointer)
DW_TAG_ptr_slice_type        // *[T] (regular slice)
DW_TAG_dst_pointer_type      // *Struct /w trailing [T]
DW_TAG_trait_object_ptr      // *Trait (regular trait object)
DW_TAG_trait_object_ptr2     // *Struct /w trailing trait

This seems a bit excessive to me.

We could also just have a DW_AT_RUST_PTR_KIND attribute with the possible values of thin, slice, trait and attach those to either a DW_TAG_reference_type or a DW_TAG_pointer_type. The rest of the DIE's fields depends on the kind. That would still be rather explicit, we'd just need one additional attribute.

[michaelwoerister]

To give an example of what the DW_AT_rust_ptr_kind variant could look like:

// &T (regular reference)
DW_TAG_reference_type
    DW_AT_rust_pointer_kind  thin
    DW_AT_mutable            true or false // defaults to false if not present
    DW_AT_type               <ref to type>

// &[T] (regular slice)
DW_TAG_reference_type
    DW_AT_rust_pointer_kind  slice
    DW_AT_mutable            true or false // defaults to false if not present
    DW_AT_type               <ref to type>
    DW_AT_object_pointer     <expr that yields address of first element>
    DW_AT_count              <expr that computes count>

// &Struct /w trailing [T]
DW_TAG_reference_type
    DW_AT_rust_pointer_kind  slice
    DW_AT_mutable            true or false // defaults to false if not present
    DW_AT_type               <ref to type>
    DW_AT_object_pointer     <expr that yields address of struct>
    DW_AT_count              <expr that computes count>

// &Trait (regular trait object)
DW_TAG_reference_type
    DW_AT_rust_pointer_kind     trait
    DW_AT_mutable               true or false // defaults to false if not present
    DW_AT_type                  <ref to type>
    DW_AT_object_pointer        <expr that yields address of object>
    DW_AT_vtable_elem_location  <expr that computes address of vtable>

// &Struct /w trailing trait
DW_TAG_reference_type
    DW_AT_rust_pointer_kind     trait
    DW_AT_mutable               true or false // defaults to false if not present
    DW_AT_type                  <ref to type>
    DW_AT_object_pointer        <expr that yields address of object>
    DW_AT_vtable_elem_location  <expr that computes address of vtable>

In this particular form, we would still have to look at the target type to find out if we have a regular slice or a struct/enum with a trailing [T] (the same goes for trait objects).

[cuviper]

Keep in mind that every new syntax you invent means new things you have to teach tools. GDB and LLDB aren't the only debuginfo consumers out there. So I would strongly lean towards semantics that a C++ tool would understand, even if seems less ideal to your own taste. At least think about how an uninformed tool might interpret your proposed scheme, compared to the status quo.

[michaelwoerister]

@cuviper Do you have examples of tools that use type information (as opposed to just line-tables)?

[cuviper]

Sure, SystemTap and Dyninst are two that I'm very familiar with.