[sicilicaParticipant]

The fact that you have two AppInstance’s isn’t doing anything for you here, unless I’m crazy. When you call App.Run(), it’s executing on pAppinstance2, which starts an SDL event loop. Both windows work because they aren’t tied directly to the AppInstance; both pAppinstance1 and pAppinstance2 should have access to both windows when they call Window.VisibleWindows().

I don’t know what you’re trying to do here, but I have a lot of trouble believing that that could possibly help anything, and it certainly doesn’t give you any extra ability to close windows individually or to exit that SDL event loop and re-enter a new one (via App.Terminate or whatever).

I’m reasonably confident that using SDL commands directly is the only solution right now.

[sicilicaParticipant]

The biggest argument not to use structs is the 1kb rule – 90% of heap allocations are garbage by the time another 1kb of data has been allocated, or something.

Even if computers are fast, I don’t think “never use structs” is a good answer though. While I don’t care about how memory is configured most of the time, I care a lot for performance-critical code, and structs give you a lot more control over how your memory is laid out (so do custom allocators, but ew). For doing 3D, for example, you can use structs to make sure that all of your mesh data is contiguous in memory. If there are tens of thousands of entities in your game, you might care about being able to have a fast way to index all of them (to iterate over some subset, or just get a list of all enemies, or whatever). The addition of structs was by far the most compelling reason for me to want to switch to monkey2 instead of staying on the otherwise-fantastic monkey1.

But yeah, unless you *know* that there’s a reason you’re supposed to use a struct, always use classes! You’ll get yourself into a lot less trouble.

[sicilicaParticipant]

Monkey

Function LoadImageWithFlags:Image(path:String, flags:TextureFlags, shader:Shader=Null) Local pixmap:= Pixmap.Load(path, Null, True) If Not pixmap Return Null If Not shader shader = mojo.graphics.Shader.GetShader("sprite") Local image:= New Image(pixmap, flags, shader) image.OnDiscarded += Lambda() pixmap.Discard() End Return image End Local myImage:= LoadImageWithFlags("myImage.png", TextureFlags.WrapST)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Function LoadImageWithFlags:Image(path:String, flags:TextureFlags, shader:Shader=Null) Local pixmap:= Pixmap.Load(path, Null, True) If Not pixmap Return Null   If Not shader shader = mojo.graphics.Shader.GetShader("sprite")   Local image:= New Image(pixmap, flags, shader)   image.OnDiscarded += Lambda() pixmap.Discard() End   Return image End   Local myImage:= LoadImageWithFlags("myImage.png", TextureFlags.WrapST)

You should be able to do something like this (slightly modified from Image.Load). Dunno why it doesn’t accept flags for wrap in the first place. Most of the other possible flags aren’t useful in the case of just dealing with the high-level Image class instead of writing your own rendering code, I suppose.

Anyway, if you make an image like that, you should be able to use it with DrawPrimitives. I didn’t test any of that though, so sorry if it throws any complex errors.

EDIT: I guess to clarify – an Image is a wrapper around an internal Texture, which is normally created when you create the Image. The flags that would make your texture wrap need to be passed in at the time when you create that Texture. Since you’re probably loading your Image from file, you unfortunately don’t have a way to pass flags like that in normally. This code handles that for you.

[sicilicaParticipant]

Running in console mode – wouldn’t that not get you access to SDL, though? Still would like to be using SDL for input and rendering and everything. Maybe I’m wrong here, but getting the WinAPI handle of a manually created window seems like the wrong approach.

SDL_HideWindow and SDL_DestroyWindow aren’t quite the same, no.

It looks like the sdl2 module exports SDL_SetWindowBordered and SDL_SetWindowFullscreen, among others, so maybe the solution is just to run SDL commands manually. In that case I would have to hack mojo though – mojo.app.Window._sdlWindow is private. While I’ve posted quite a bit in the past about wanting more direct access to OpenGL/SDL directly, in this case for simple 2D I do plan on actually using mojo, is all. Oh, there is actually a getter for the SDL window, nevermind. I should’ve read your code more carefully.

Is there a reason that more of this functionality isn’t already exposed by the Window class? This was all so easy by the time I started using monkey1, and I don’t think I was super late to the party. Maybe the worst part is that 90% of the methods on these classes are hidden. I get that Mark wants to wait to expose stuff that isn’t finalized, but it means I have to spend a lot of time looking through the actual source for all these modules in order to be able to actually do anything productive with them at all. It was all I could do to figure out how to get an actual timestamp for SeedRnd:

Monkey

SeedRnd(libc.tolong(libc.time(Null)))

1	SeedRnd(libc.tolong(libc.time(Null)))

All of the time_t and tm_t structs, and all of libc, aren’t even documented – but from the looks of things it would’ve been really hard to get an actual timestamp from std.time.Time even after calling std.time.Time.Now to get the system time in the first place.

[sicilicaParticipant]

@taumel

“Toolchain” here basically just means “a C compiler with bindings for your hardware.” In any case, yeah, I’m completely not talking about mojo and how it’s implemented here, so you’re probably right and that just makes me a black sheep.

[sicilicaParticipant]

@taumel
The problem with supporting the best APIs for each platform you want to run on is that it’s extremely costly, especially for a small studio – in practice, the way you handle memory and the way you interact with the graphics pipeline need to completely change based on the target, and that’s probably more true for consoles than for different desktop OSes. Worse, most of this code is so integral to the engine that it touches almost all of the game, making it hard to completely abstract out from gameplay code. Even a professional studio carefully chooses which platforms to support and has to devote a lot of resources to porting code, so you really only get releases on Windows and PS/Xbox, since each “version” of the game needs to be carefully maintained.

In any case, all of these API concerns have little to do with language – your language choice should have little impact on any of this so long as you have an available toolchain. I’m only considering C++ targets with existing libraries for the target system; this conversation is essentially just about the technical details of connecting those libraries to Monkey. Whether you want to use D3D, or OpenGL, or Vulkan – whether you want SDL, or anything else – the choice of which is the “best” choice has little to do with the fact that any of those options should be possible.

 

Anyway, Mark publishes a gles20 module, and I’d like to use the GLES C API without forcing the driver to compile GLSL scripts in compatibility mode, ideally also without hacking Mark’s code too overly much. Of course I could link in a different OpenGL lib for the C bindings, since it’s apparent that I’m not concerned with supporting mobile and such anyway, but it’d be nice to not have to worry about it. If anyone – in particular Mark, who should know how things are laid out better than the rest of us – can give me pointers on what part of the module is giving me grief (probably the Angle stuff, if I had to guess), it’d be appreciated.

 

Edit:

@cocon, that actually looks like a really interesting approach, which I guess is what the Rust community came up with? That could be really convenient for general use, but I doubt it has any longevity, since by adding that additional layer of abstraction between the application and the driver (at best you have to remap function calls, at worst you have to emulate some things) you will always lose performance. Vulkan is interesting precisely because it has momentum as a more universal way to work with GPUs and is MORE efficient than most of the currently used methods by being a better representation of the underlying hardware and drivers (same with DX12, except for the M$ label).

[sicilicaParticipant]

Here’s a much better example.

Monkey

Function Mesh_Alloc(mesh:Mesh Ptr, numVertices:Int, numTriangles:Int) mesh->numVertices = numVertices mesh->numTriangles = numTriangles Local mem:Byte Ptr = Cast<Byte Ptr>(malloc(numVertices*Vertex.SIZE + numTriangles*Triangle.SIZE)) mesh->vertices = Cast<Vertex Ptr>(mem) mesh->triangles = Cast<Triangle Ptr>(Varptr mem[numVertices*Vertex.SIZE]) End Function Mesh_Free(mesh:Mesh Ptr) free(mesh->vertices) mesh->numVertices = 0 mesh->numTriangles = 0 mesh->vertices = Null mesh->triangles = Null End Struct Mesh Field numVertices:Int Field vertices:Vertex Ptr Field numTriangles:Int Field triangles:Triangle Ptr End Struct Vertex Field texCoords:Vec2f Field position:Vec3f Const SIZE:= 2*SIZE_OF_FLOAT + 3*SIZE_OF_FLOAT End Struct Triangle Field vertexIndex0:Short Field vertexIndex1:Short Field vertexIndex2:Short Const SIZE:= 3*SIZE_OF_SHORT End Const SIZE_OF_SHORT:= 2 Const SIZE_OF_FLOAT:= 4

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

Function Mesh_Alloc(mesh:Mesh Ptr, numVertices:Int, numTriangles:Int) mesh->numVertices = numVertices mesh->numTriangles = numTriangles   Local mem:Byte Ptr = Cast<Byte Ptr>(malloc(numVertices*Vertex.SIZE + numTriangles*Triangle.SIZE)) mesh->vertices = Cast<Vertex Ptr>(mem) mesh->triangles = Cast<Triangle Ptr>(Varptr mem[numVertices*Vertex.SIZE]) End Function Mesh_Free(mesh:Mesh Ptr) free(mesh->vertices)   mesh->numVertices = 0 mesh->numTriangles = 0 mesh->vertices = Null mesh->triangles = Null End   Struct Mesh   Field numVertices:Int Field vertices:Vertex Ptr Field numTriangles:Int Field triangles:Triangle Ptr   End       Struct Vertex   Field texCoords:Vec2f Field position:Vec3f   Const SIZE:= 2*SIZE_OF_FLOAT + 3*SIZE_OF_FLOAT   End Struct Triangle   Field vertexIndex0:Short Field vertexIndex1:Short Field vertexIndex2:Short   Const SIZE:= 3*SIZE_OF_SHORT   End   Const SIZE_OF_SHORT:= 2 Const SIZE_OF_FLOAT:= 4

[sicilicaParticipant]

In case anyone searches into this thread later with a similar situation, a solution like this should work:

Monkey

Import "<libc>" Using libc.. Struct Vec3     Property X:Float()         Return data[0]     Setter(val:Float)         data[0] = val     End     Property Y:Float()         Return data[1]     Setter(val:Float)         data[1] = val     End     Property Z:Float()         Return data[2]     Setter(val:Float)         data[2] = val     End     Field data:Float Ptr     Const SIZE = 12    ' 3 floats, 4 bytes each End Function Vec3_Alloc(out:Vec3 Ptr, count:Int = 1)     Local mem:Byte Ptr = malloc(count * Vec3.SIZE)     ' this assignment work depends on your specific struct     ' probably need to do array syntax into mem, since pointer arithmetic isn't allowed?     For Local i = 0 Until count         out[i].data = Varptr mem[i * Vec3.SIZE]     Next End Function Vec3_Free(val:Vec3 Ptr)     ' this depends on your specific struct     ' free whatever you had at mem[0], if you have multiple fields in the struct     free(val.data) End

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

Import "<libc>"   Using libc..   Struct Vec3       Property X:Float()         Return data[0]     Setter(val:Float)         data[0] = val     End     Property Y:Float()         Return data[1]     Setter(val:Float)         data[1] = val     End     Property Z:Float()         Return data[2]     Setter(val:Float)         data[2] = val     End       Field data:Float Ptr       Const SIZE = 12    ' 3 floats, 4 bytes each   End   Function Vec3_Alloc(out:Vec3 Ptr, count:Int = 1)     Local mem:Byte Ptr = malloc(count * Vec3.SIZE)       ' this assignment work depends on your specific struct     ' probably need to do array syntax into mem, since pointer arithmetic isn't allowed?     For Local i = 0 Until count         out[i].data = Varptr mem[i * Vec3.SIZE]     Next End   Function Vec3_Free(val:Vec3 Ptr)     ' this depends on your specific struct     ' free whatever you had at mem[0], if you have multiple fields in the struct     free(val.data) End

Vec3_Alloc() can allocate arrays of contiguous memory for storing a large number structs.

Your struct should only contain pointers if you do this (even, say, a single float needs to be a pointer; classes are fine, since they are always be pointers, but ofc they shouldn’t be allocated from your malloc memory).

If you allocate an array of structs, they have to all be free’d at once, pass a pointer to the first one to the Vec3_Free() function (duh).

You still need to store your ACTUAL structs somewhere on the heap, so they are technically separate from the memory, eg:

Monkey

Global triangle:= New Vec3[3] ' this is telling the GC where the actual structs themselves are, use pointers everywhere except here Vec3_Allocate(Varptr triangle, 3)

1 2	Global triangle:= New Vec3[3] ' this is telling the GC where the actual structs themselves are, use pointers everywhere except here Vec3_Allocate(Varptr triangle, 3)

And as the last caveat, you’ll fracture your memory a lot worse than even the GC would if you don’t know what you’re doing, so for example, don’t actually create a single triangle this way.

[sicilicaParticipant]

@taumel – Other than mojo being based on a particular graphics library, nothing there is a problem for M2. There’s no reason you can’t use it with a different version of OpenGL, or Vulkan, etc – you would just need to wrap those C APIs and throw them in a module. Also, Vulkan is already in a really good position – basically 100% of cards you care about already support it, and I’ve played with the API and it all seems sensible. It’s not a question of whether it can “solve the issues” or whether vendors support it; the question is whether game developers will actually start using it. If it never gets bigger than something a few indie devs use, then support for it won’t last into the future. That said, for me personally, I’m not going to develop anything commercial with it until I’ve seen some others do so. I’m not the early adopter type.

@mark – Yeah, what you have there is almost exactly what I’m doing, except I’m not using libc for anything. I tested again, though, and it wasn’t the SetAttribute() call that crashes, it was the glGetString() call just afterwards:

Monkey

' sdl init... SDL_GL_SetAttribute(SDL_GL_CONTEXT_PROFILE_MASK, SDL_GL_CONTEXT_PROFILE_CORE) ' make window and context here... Print String.FromCString(glGetString(GL_VERSION)) ' null pointer exception

1 2 3 4 5 6 7

' sdl init...   SDL_GL_SetAttribute(SDL_GL_CONTEXT_PROFILE_MASK, SDL_GL_CONTEXT_PROFILE_CORE)   ' make window and context here...   Print String.FromCString(glGetString(GL_VERSION)) ' null pointer exception

So, my bad on that. That proves that moving the SetAttribute() to before the window creation made it do more than noop, since before it had no impact on anything – but glGetString() should return a ubyte pointer and I tested and showed it giving me null, which is… weird. Any idea?

Oh, and if I skip testing GL_VERSION, then trying to compile a shader leaves GL_COMPILE_STATUS as 0 and glGetShaderInfoLogEx() returns an empty string, so I can only imagine the driver is failing to write anything to that byte array, too.

EDIT: WHOA, libc gives you bindings for malloc and free? And memset, too. Wow. I posted before about wanting to store fixed-size arrays inside a struct, and I concluded that if I really wanted to I’d either have to write the struct in C and import it or make a really complicated RawArray struct for it based on the Array in the core module. But this way I could just store a pointer and do malloc/free in a constructor/destructor (though I probably have to call them manually)! Sweet. And certainly pointers support array syntax, since M2’s “->” just seems to be an alias for “[0].”. Cool.

[sicilicaParticipant]

Hmm, no dice. I tried disabling SDL_VIDEO_OPENGL_ES2 and SDL_VIDEO_OPENGL_EGL as well, but it didn’t seem to make a difference. Also, calling SDL_GL_SetAttribute() before creating a window crashes, presumably because there’s no GL context bound (you can’t create a context without a window unless I’m mistaken?).

I mean, you’re not wrong about compatibility and stuff – I’ve definitely seen people have problems with OpenGL working on their cards/drivers – but the whole thing is just a bunch of crap. Seriously hoping Vulkan manages to clean all of this up….

EDIT: I deleted my build folder between all these to make sure, too.

[sicilicaParticipant]

So there won’t be any way to allocate even fixed-size arrays “inside” a struct? That’s too bad, but it makes sense that arrays need to always be pointers. Guess I’ll have to continue indexing into databuffers and large float arrays.

It sounds like my intuition for everything else with struct pointers was right, then, but can you clarify what you said about how strings are handled? You said strings weren’t GC’d – but certainly they are, since they would always be on the heap, no?

[sicilicaParticipant]

How structs work under the hood – is precisely what I’m asking here. I know very well what a struct is.

So this code, which works in Monkey-X, isn’t valid in M2:

Monkey

Class Whatever Field arr:Float[4] End

1 2 3

Class Whatever     Field arr:Float[4] End

I’m guessing that that was just an alias for “:= New Float[4]” anyway. In any case, for structs to make any sense, they need to be able to actually ‘own’ or ‘contain’ their data – for example, a struct for a vector or matrix needs to contain some array of floats. If the struct just contains a pointer, and the array ITSELF is going through the GC anyway, then the struct serves absolutely no purpose and provides no optimization – and Arrays are always Objects, from what I can tell.

 

Edit:

Actually, I probably found a bug that I should report. Here was the actual struct I was playing with; it throws a runtime error when you init an instance of it, on “data[0] = val” because data.Length is 0 at the time. Maybe it’s trying to initialize the properties and fields, but “data” hasn’t been assigned the pointer to the new array yet?

Monkey

Struct Vector3 Field data:= New Float[3] Property x:Float() Return data[0] Setter(val:Float) data[0] = val End Property y:Float() Return data[1] Setter(val:Float) data[1] = val End Property z:Float() Return data[2] Setter(val:Float) data[2] = val End End

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Struct Vector3   Field data:= New Float[3]   Property x:Float() Return data[0] Setter(val:Float) data[0] = val End Property y:Float() Return data[1] Setter(val:Float) data[1] = val End Property z:Float() Return data[2] Setter(val:Float) data[2] = val End   End

[sicilicaParticipant]

Sorry I’m being kinda nit-picky about this. I guess the thing for me is, this is the first time I’ve ever seen any form of unmanaged memory present in a language that also does managed pointers and automatic garbage collection, and it’s probably the first for a lot of people. With a language like C, I know exactly how all of that’s working; with a language like Java, I know that I have no ability to influence things so it’s okay if the inner workings are a magical black box; but in a situation like this, I’m excited to try to use these tools that could allow huge optimization, but it’s hard/scary to do so without understanding exactly how all of the memory management is performed under the hood.

[sicilicaParticipant]

It has less to do with how much memory you need to use and how often you need to allocate it then it has to do with managing the heap. Any time you “new up” a class in a language with managed pointers, all you really get is a pointer. Even if I store that class in a Local, which would be on the stack, the stack only contains the pointer – the actually data is somewhere in the heap. Even if you know how many “things” you’ll need and make, say, an array of 20 instances of some class – your array, wherever it’s stored, is an array of 20 pointers. The 20 instances you actually create will all be in the heap, and since they are allocated individually, it’s almost certain that they will all be at random places in memory. You want to control your allocations because contiguous memory will result in a lot fewer cache misses, and because polluting memory like that leads to fragmentation as objects get created and destroyed essentially ad hoc. For software that doesn’t need to be highly performant you don’t care, but for a game, you have to manage huge amounts of memory, you have a lot of calculations to make many many times per second, and you need all of this to happen as smoothly as possible so your framerate doesn’t have significant dips and spikes.

 

@mark – Let me try to think of a couple of the specific questions I have, I’m being super vague.

Is there a difference between declaring an array these two ways?

Monkey

Global myArray:MyStruct[20] Global myArray:MyStruct[] = New MyStruct[20] ' or with := ... Global myArray:MyStruct[] ' do stuff, get in a method... myArray = New MyStruct[20]

1 2 3 4 5 6 7

Global myArray:MyStruct[20]   Global myArray:MyStruct[] = New MyStruct[20] ' or with := ...   Global myArray:MyStruct[] ' do stuff, get in a method... myArray = New MyStruct[20]

When you say strings need to be malloc’d / free’d – how or when does that happen? Is the GC still the one that tracks their usage?

And on the GC, I presume that it works by spidering memory to see what malloc’d objects are still accessible, similar to the way I think Java does? The only other algorithm I know of is to count references, which I don’t think we’re doing? Does it rely on reflection to know how to follow pointers at runtime?

In addition to giving us structs, M2 gave us the ability to convert a value into a pointer, like getting a reference in C++ (I think we use varptr or some weird keyword, I don’t know off the top of my head…). As in the above example, lets assume I declare an array of struct values in some global or local scope. I then pass a pointer to one of those structs into another function (because in this case, I didn’t make it an array of values because I wanted to pass by value, what I wanted was to control the allocator – I want the subroutine to be able to write to it still). What happens with the GC here? Now all the sudden there’s a pointer on the stack in my subroutine – will the GC do anything to try to manage it? What if the reference I passed was from a Local, on the stack, rather than something I had allocated in the heap? In C++, that pointer would be an address in the stack, and nothing would stop me from shooting myself in the foot if the subroutine ends up storing it somewhere and trying to use it after the caller’s stack frame goes out of scope. And again, what is the GC going to think / do about this pointer – especially if I do something nasty like store it?

[sicilicaParticipant]

That’s exactly what the difference between passing a value and a reference is. @gcmartijn, I don’t think structs exist in any of the languages you mentioned, since they are all pretty high level. A struct is like if you duct-tape a bunch of variables together: if I pass a struct into a function, for example, it’s the same as if I was taking a parameter for each variable in the struct and passing all of them at the same time. All of the languages you mention follow the “every object is a pointer” mindset, so if you pass an object into a function, you’re not passing its data; instead, you’re passing an integer that represents where in memory the object is.

Anyway, whether you pass values or pointers into a function isn’t really the question I’m getting at here – of course you would never want to copy large structs around, and if you don’t know exactly what you’re doing you might not want to use structs at all. My question is about memory allocation – if I can’t be confident that I have a lot of control over the allocator, then it doesn’t make any sense to mess with what I’m talking about at all.

[Author]

Posts