Structs and reflection

[EthernautParticipant]

Is there any chance structs will be supported by the reflection system in the future? (i.e. no technical reason why it can’t happen?)

I’m working on a basic general purpose serialization that uses reflection to load and save all of an object’s “settable” properties, but I got stuck figuring out the type of properties that contain structs.

Monkey

#Import "<std>" #Import "<mojo>" #Import "<reflection>" Using std.. Using mojo.. Function Main() Local test01 := "Test!" Local test02 := 1000.0 Local test03 := New Bool[]( True, False, True ) Local test04 := New Image( 128, 128 ) Local test05 := New Stack<Int> Local test06 := New Vec3f( 0.5 ) Local test07 := New Color( 0.1, 0.5, 1.0, 1.0 ) Print Typeof( test01 ) Print Typeof( test02 ) Print Typeof( test03 ) Print Typeof( test04 ) Print Typeof( test05 ) Print Typeof( test06 ) Print Typeof( test07 ) End

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

#Import "<std>" #Import "<mojo>" #Import "<reflection>"   Using std.. Using mojo..   Function Main() Local test01 := "Test!" Local test02 := 1000.0 Local test03 := New Bool[]( True, False, True ) Local test04 := New Image( 128, 128 ) Local test05 := New Stack<Int> Local test06 := New Vec3f( 0.5 ) Local test07 := New Color( 0.1, 0.5, 1.0, 1.0 ) Print Typeof( test01 ) Print Typeof( test02 ) Print Typeof( test03 ) Print Typeof( test04 ) Print Typeof( test05 ) Print Typeof( test06 ) Print Typeof( test07 ) End

I don’t need this right now, so I can put this aside until there’s a solution, but I need to know if it’s not really doable so I can try to find a different solution on my own side.

Thanks!

[Mark SiblyKeymaster]

Just had a look at this, and the big problem is that structs are always passed ‘by value’, which means its impossible to modify them ‘by reference’, eg:

Monkey

Struct C Field x:Int End Function SetX( c:C,i:Int ) c.x=i End Function Main() Local c:=New C SetX( c,100 ) Print c.x 'alas wont print 100! End

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Struct C    Field x:Int End   Function SetX( c:C,i:Int )    c.x=i End   Function Main()    Local c:=New C    SetX( c,100 )    Print c.x   'alas wont print 100! End

This fails to SetX because SetX() receives a copy of ‘c’, not a reference to the same ‘c’.

Reflection has a similar problem – when you pass the ‘instance’ object to ‘Set’ or ‘Invoke’, you’re passing a copy of the struct value, so any modifications made by Set or Invoke are being made to a copy that will just be thrown away.

So I’ll keep looking, but right now the answer is no.

[Mark SiblyKeymaster]

I’ve actually had some success with this and it may yet happen!

The trick is to treat the Variant as a ‘box’ for struct values. The struct value still gets copied when you set or get it, but the Set() and Invoke() methods can directly access the value in the variant/box.

So as long as you keep passing the same variant to decl.Set() etc, you’ll be updating the same struct value.

[nerobotParticipant]

Very nice!

[EthernautParticipant]

So close! This code almost works with the latest dev branch. The goal is to print out all settable properties, recursively.
Only when we uncomment the “BrandNewName” property, which returns a struct, it fails. Can’t see why, since the Color property also returns a struct, yet that one works perfectly.

Monkey

#Import "<std>" #Import "<mojo>" #Import "<reflection>" Using std.. Using mojo.. Class JsonObject Extension Method Serialize( key:String, v:Variant ) Select v.Type.Name Case "Double" SetNumber( key, Cast<Double>( v ) ) Case "Float" SetNumber( key, Cast<Float>( v ) ) Case "String" SetString( key, Cast<String>( v ) ) Case "Bool" SetBool( key, Cast<Bool>( v ) ) Default Local newObj:= New JsonObject newObj.SetString( "Class", v.Type.Name ) For Local decl:DeclInfo = Eachin v.Type.GetDecls() If decl.Kind = "Property" And decl.Settable newObj.Serialize( decl.Name, decl.Get( v ) ) End Next SetObject( key, newObj.ToObject() ) End End End Function Main() Local json := New JsonObject() json.Serialize( "text","Just a little text" ) json.Serialize( "number", 100.0 ) json.Serialize( "isItTrue?", False ) json.Serialize( "Knight1", New KnightWhoSaysNi ) json.Serialize( "Knight2", New KnightWhoSaysNi ) json.Serialize( "KnightStruct", New NonsenseStruct ) Print json.ToJson() End '***************** Test classes and structs ***************** Class KnightWhoSaysNi Private Field _name := "Ni!" Field _value:Float = 1000.0 Field _schroob:= New Schruberry Field _nonsense:= New NonsenseStruct Field _color := New Color( 0, 0, 1, 1 ) Public Property Value:Float() Return _value Setter( v:Float ) _value = v End Property Name:String() Return _name Setter( n:String ) _name = n End Property Request:Schruberry() Return _schroob Setter( s:Schruberry ) _schroob = s End 'Uncomment these lines to get a scary error! ' Property BrandNewName:NonsenseStruct() ' Return _nonsense ' Setter( n:String ) ' _nonsense.Words = n ' End Property Color:Color() Return _color Setter( c:Color ) _color = c End End Class Schruberry Property Random:Float() Return Rnd() Setter( bogus:Float ) 'One that looks nice. And not too expensive. End End Struct NonsenseStruct Private Field _words:= "Ekke Ekke Ekke Ekke Ptang Zoo Boing!" Public Property Words:String() Return _words Setter( w:String ) _words = w End End

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113

#Import "<std>" #Import "<mojo>" #Import "<reflection>"   Using std.. Using mojo..   Class JsonObject Extension   Method Serialize( key:String, v:Variant ) Select v.Type.Name Case "Double" SetNumber( key, Cast<Double>( v ) ) Case "Float" SetNumber( key, Cast<Float>( v ) ) Case "String" SetString( key, Cast<String>( v ) ) Case "Bool" SetBool( key, Cast<Bool>( v ) ) Default Local newObj:= New JsonObject newObj.SetString( "Class", v.Type.Name ) For Local decl:DeclInfo = Eachin v.Type.GetDecls() If decl.Kind = "Property" And decl.Settable newObj.Serialize( decl.Name, decl.Get( v ) ) End Next SetObject( key, newObj.ToObject() ) End End   End     Function Main() Local json := New JsonObject() json.Serialize( "text","Just a little text" ) json.Serialize( "number", 100.0 ) json.Serialize( "isItTrue?", False ) json.Serialize( "Knight1", New KnightWhoSaysNi ) json.Serialize( "Knight2", New KnightWhoSaysNi ) json.Serialize( "KnightStruct", New NonsenseStruct ) Print json.ToJson() End   '***************** Test classes and structs *****************     Class KnightWhoSaysNi Private Field _name := "Ni!" Field _value:Float = 1000.0 Field _schroob:= New Schruberry Field _nonsense:= New NonsenseStruct Field _color := New Color( 0, 0, 1, 1 ) Public Property Value:Float() Return _value Setter( v:Float ) _value = v End Property Name:String() Return _name Setter( n:String ) _name = n End Property Request:Schruberry() Return _schroob Setter( s:Schruberry ) _schroob = s End 'Uncomment these lines to get a scary error! ' Property BrandNewName:NonsenseStruct() ' Return _nonsense ' Setter( n:String ) ' _nonsense.Words = n ' End Property Color:Color() Return _color Setter( c:Color ) _color = c End End     Class Schruberry Property Random:Float() Return Rnd() Setter( bogus:Float ) 'One that looks nice. And not too expensive. End End     Struct NonsenseStruct Private Field _words:= "Ekke Ekke Ekke Ekke Ptang Zoo Boing!" Public Property Words:String() Return _words Setter( w:String ) _words = w End End

This will spit out the following string:

Monkey

{ "Knight1":{ "Class":"default.KnightWhoSaysNi", "Color":{ "A":1, "B":1, "Class":"std.graphics.Color", "G":0, "R":0 }, "Name":"Ni!", "Request":{ "Class":"default.Schruberry", "Random":0.41015625 }, "Value":1000 }, "Knight2":{ "Class":"default.KnightWhoSaysNi", "Color":{ "A":1, "B":1, "Class":"std.graphics.Color", "G":0, "R":0 }, "Name":"Ni!", "Request":{ "Class":"default.Schruberry", "Random":0.59253692626953125 }, "Value":1000 }, "KnightStruct":{ "Class":"default.NonsenseStruct", "Words":"Ekke Ekke Ekke Ekke Ptang Zoo Boing!" }, "isItTrue?":true, "number":100, "text":"Just a little text" }

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

{ "Knight1":{ "Class":"default.KnightWhoSaysNi", "Color":{ "A":1, "B":1, "Class":"std.graphics.Color", "G":0, "R":0 }, "Name":"Ni!", "Request":{ "Class":"default.Schruberry", "Random":0.41015625 }, "Value":1000 }, "Knight2":{ "Class":"default.KnightWhoSaysNi", "Color":{ "A":1, "B":1, "Class":"std.graphics.Color", "G":0, "R":0 }, "Name":"Ni!", "Request":{ "Class":"default.Schruberry", "Random":0.59253692626953125 }, "Value":1000 }, "KnightStruct":{ "Class":"default.NonsenseStruct", "Words":"Ekke Ekke Ekke Ekke Ptang Zoo Boing!" }, "isItTrue?":true, "number":100, "text":"Just a little text" }

Sorry for all the Monty Python references.

[EthernautParticipant]

Found it. The property that wasn’t working returns a struct “NonsenseStruct”, but it sets a String property in that struct, instead of setting the struct itself.
Changing it to this works:

Monkey

Property BrandNewName:NonsenseStruct() Return _nonsense Setter( n:NonsenseStruct ) _nonsense = n End

1 2 3 4 5 6 7

Property BrandNewName:NonsenseStruct() Return _nonsense Setter( n:NonsenseStruct ) _nonsense = n End

[Mark SiblyKeymaster]

Nice work, very bleeding edge! Will fix the property type bug ASAP.

I’ve successfully had generic struct instances reflected here (eg: Vec3f, Mat4f etc) but it needs a bit more work and I think a REFLECTION_FILTER equivalent also needs to be added before any of this is truly useful.

[EthernautParticipant]

Thanks Mark!

Things are working nicely, but I’ve ran into a problem that may be more related to how inheritance works than to reflection… in the following code, I can’t serialize the objects coming from a stack with the correct extended type. They end up all being serialized as “BaseClass”, even though one of them is “ExtendedClass”.

Is there a way to cast them as i iterate through the stack?

Monkey

#Import "<std>" #Import "<reflection>" Using std.. '************************ json extension '************************ Class JsonObject Extension Method Serialize( key:String, v:Variant ) Select v.Type.Name Case "Double" SetNumber( key, Cast<Double>( v ) ) Case "Float" SetNumber( key, Cast<Float>( v ) ) Case "String" SetString( key, Cast<String>( v ) ) Case "Bool" SetBool( key, Cast<Bool>( v ) ) Default Local newObj:= New JsonObject newObj.SetString( "Class", v.Type.Name ) For Local decl:DeclInfo = Eachin v.Type.GetDecls() If ( decl.Kind = "Property" And decl.Settable )' Or ( decl.Kind = "Field" ) newObj.Serialize( decl.Name, decl.Get( v ) ) End Next SetObject( key, newObj.ToObject() ) End End End '************************ main function '************************ Function Main() Local test1 := New BaseClass( "test1" ) 'This object will be reported as "BaseClass", instead of "ExtendedClass" Local test2 := New ExtendedClass( "test2", "Ni!" ) Local json := New JsonObject() For Local t := Eachin BaseClass.All() json.Serialize( t.Name, t ) Next 'However this one, not coming from the stack, works! json.Serialize( "test3", New ExtendedClass( "test3", "NiNiNi!" ) ) Print json.ToJson() End '************************ test classes '************************ Class BaseClass Property Name:String() Return _name End Method New( name:String ) _name = name _words= "Ekke Ekke Ekke Ekke Ptang Zoo Boing!" _all.Push( Self ) End Function All:BaseClass[]() Return _all.ToArray() End Protected Field _name:String Field _words:String Global _all:= New Stack<BaseClass> End Class ExtendedClass Extends BaseClass Property Words:String() Return _words Setter( w:String ) _words = w End Method New( name:String, words:String ) Super.New( name ) _words = words End End

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93

#Import "<std>" #Import "<reflection>"   Using std..   '************************ json extension '************************   Class JsonObject Extension Method Serialize( key:String, v:Variant ) Select v.Type.Name Case "Double" SetNumber( key, Cast<Double>( v ) ) Case "Float" SetNumber( key, Cast<Float>( v ) ) Case "String" SetString( key, Cast<String>( v ) ) Case "Bool" SetBool( key, Cast<Bool>( v ) ) Default Local newObj:= New JsonObject newObj.SetString( "Class", v.Type.Name ) For Local decl:DeclInfo = Eachin v.Type.GetDecls() If ( decl.Kind = "Property" And decl.Settable )' Or ( decl.Kind = "Field" ) newObj.Serialize( decl.Name, decl.Get( v ) ) End Next SetObject( key, newObj.ToObject() ) End End End   '************************ main function '************************   Function Main() Local test1 := New BaseClass( "test1" ) 'This object will be reported as "BaseClass", instead of "ExtendedClass" Local test2 := New ExtendedClass( "test2", "Ni!" ) Local json := New JsonObject() For Local t := Eachin BaseClass.All() json.Serialize( t.Name, t ) Next 'However this one, not coming from the stack, works! json.Serialize( "test3", New ExtendedClass( "test3", "NiNiNi!" ) ) Print json.ToJson() End     '************************ test classes '************************     Class BaseClass Property Name:String() Return _name End Method New( name:String ) _name = name _words= "Ekke Ekke Ekke Ekke Ptang Zoo Boing!" _all.Push( Self ) End Function All:BaseClass[]() Return _all.ToArray() End Protected Field _name:String Field _words:String Global _all:= New Stack<BaseClass> End     Class ExtendedClass Extends BaseClass   Property Words:String() Return _words Setter( w:String ) _words = w End Method New( name:String, words:String ) Super.New( name ) _words = words End   End

[Mark SiblyKeymaster]

This is because Variant.Type only returns the static type of the variant. You’ll need to use InstanceType on the object contained in the variant instead of v.Type, eg:

Local type:=Cast<Object>( v ).InstanceType

…safer…

Monkey

Function DynamicType:Type( v:Variant ) If v.Type.Kind="Class" or v.Type.Kind="Interface" Local inst:=Cast<Object>( v ) If inst Return inst.InstanceType Endif Return v.Type End

1 2 3 4 5 6 7 8

Function DynamicType:Type( v:Variant )    If v.Type.Kind="Class" or v.Type.Kind="Interface"       Local inst:=Cast<Object>( v )       If inst Return inst.InstanceType    Endif    Return v.Type End

I now think InstanceType should probably be renamed to DynamicType, and possibly the above DynamicType code added to Variant too which can do it more efficiently.

Anyway, quick fix for the above: change the Default block to:

Monkey

Local newObj:= New JsonObject Local type:=Cast<Object>( v ).InstanceType newObj.SetString( "Class", type.Name ) For Local decl:DeclInfo = Eachin type.GetDecls() If ( decl.Kind = "Property" And decl.Settable )' Or ( decl.Kind = "Field" ) newObj.Serialize( decl.Name, decl.Get( v ) ) End Next SetObject( key, newObj.ToObject() )

1 2 3 4 5 6 7 8 9 10

Local newObj:= New JsonObject Local type:=Cast<Object>( v ).InstanceType newObj.SetString( "Class", type.Name ) For Local decl:DeclInfo = Eachin type.GetDecls() If ( decl.Kind = "Property" And decl.Settable )' Or ( decl.Kind = "Field" ) newObj.Serialize( decl.Name, decl.Get( v ) ) End Next SetObject( key, newObj.ToObject() )

[EthernautParticipant]

Sweet! Works now.

I remember something about serialization not recognizing generics, and I definitely see it not working.It also fails on Enum types, apparently. The way I’m gonna work around it is having properties that return arrays or other values, instead of generic classes and structs, so that instead of:

Monkey

Property Position:Vec3<Float>() Return _position Setter( p:Vec3<Float> ) _position = p End

1 2 3 4 5

Property Position:Vec3<Float>() Return _position Setter( p:Vec3<Float> ) _position = p End

I’ll do:

Monkey

Property Position:Float[]() Return New Float[]( _position.X, _position.Y, _position.Z ) Setter( p:Float[] ) _position = New Vec3f( p[0], p[1], p[2] ) End

1 2 3 4 5

Property Position:Float[]() Return New Float[]( _position.X, _position.Y, _position.Z ) Setter( p:Float[] ) _position = New Vec3f( p[0], p[1], p[2] ) End

Or I can create ToArray() and FromArray() method extensions to classes and structs I know I’ll use anyway, like Vec3f, etc.

Gonna work on loading now. Will post Serialize/Deserialize code to archives once it works well (I already have it handling arrays, yay!).

Monkey2 is a lot of fun. Cheers!

[nerobotParticipant]

and I think a REFLECTION_FILTER equivalent also needs to be added before any of this is truly useful.

I’m waiting for such filter!:)

I want to have minimal reflection where each class/interface would have its own instance for Typeof – to have comparison of them, and type should have at least Name property.

[Mark SiblyKeymaster]

I’m waiting for such filter!:)

Soon – hopefully by v1.1.09!

[Author]

Posts