Component system for Mojo3D

About Monkey 2 › Forums › Monkey 2 Code Library › Component system for Mojo3D

Tagged: , ,

This topic contains 14 replies, has 6 voices, and was last updated by  DruggedBunny 1 year, 6 months ago.

Viewing 15 posts - 1 through 15 (of 15 total)
  • Author
    Posts
  • October 17, 2017 at 5:46 am #11133

    Ethernaut
    Participant

    Hi,
    I thought this was a good moment to share what I’ve been working on.

    https://github.com/DoctorWhoof/game3d

    This is still very, very early, and was a humongous effort for me, considering I’m just learning programming and this needed delving into hairy things like Serialization, Reflection, Generics, etc., things that looked like an arcane alien language to me as little as two or three years ago.

    I’m sure the veterans around here will poke many holes in it, but be gentle. This was probably more than I could chew, so I’m very happy that I managed to actually get it working, and can’t wait to move past the overly technical stuff and get to the more fun stuff (for me, visuals and game design).

    Here’s a typical use of the main SceneView class:

    As you can see… well, you can’t see!!! All the game logic is actually defined in .json files containing the entities, their components and values.

    That way, you can “hot load” scene changes without recompiling anything. It will also make things like a visual editor in the future easier, I hope. That said, I literally just got this feature working, so I’m sure there’s lots of extra work to be done.

    The serialization system will load/save any Fields or Properties that:
    – Are settable
    – Do not have names that start with “_” (usually private fields in Mojo, a convention I adopted)

    Simple example scene (From the ‘_examples’ folder):

    Anyway, I hope it can help someone. Getting the Reflection/Serialization part working was really hard for me, and can still use improvements (can’t serialize object arrays that are not Component[], for instance). Learning things like this is why I adopted Monkey as my learning programming language, and I hope Monkey stays around.

    Cheers.

    October 17, 2017 at 7:36 am #11138

    nerobot
    Participant

    Great work! Subscribed. 🙂

    October 17, 2017 at 8:02 am #11140

    CopperCircle
    Participant

    That’s great, thanks for sharing 🙂

    October 18, 2017 at 8:33 am #11166

    Mark Sibly
    Keymaster

    Very nice!

    Not sure how happy you’ll be about this (hopefully happy!) but I actually started on a component system for mojo3d based on what we were talking about in the extending mojo3d thread, and I was planning to release it in v1.1.08.

    I only did the ‘update’ stuff – Animator, Collider, RigidBody – as I didn’t want to move *everything* to a component system just yet for fear of upsetting the massive throng of existing users. Plus, I really just wanted to play with the general idea first anyway, and Animator and co are still very much experimental.

    The ComponentType and Component classes look like this:

    A few notes:

    • the ‘Singleton’ ComponentType enum flag is for components that an Entity can only have 1 of. If this is not set, you can add more than one of a particular component type. Maybe ‘Unique’ might be a better name for this to prevent confusion? or even ‘Multiple’ with opposite meaning?
    • The ‘Priority’ property is used when adding a component and determines component order within an entity. This allows you to make sure certain component types are processed/updated before/after other component types.
    • I only did a handful of ‘On’ virtual methods in Component. More can easily be added, although I think we should be careful about what we add here, as there’s a danger of overcomplicating things IMO. Currently, entities are visited in ‘tree order’. OnBeginUpdate happens first and is ‘bottom up’, then OnUpdate happens ‘top down’. IME it’s very convient to have both bottom up and top down passes, although having a bottom up pass does imply visiting ALL nodes, whereas a top down pass can potentially ‘early out’, say for culling or something.
    • I didn’t want to use reflection just yet as it’s not clear just how much overhead that will eventually use. I don’t expect it to be massive, but I still like to keep things as lean as possible and I don’t think we need reflection in this case. Hence the ComponentType class.

    Here’s an example of an actual Component, the new RigidBody:

    The const ComponentType ‘Type’ member is a convention – every component type should implement it so that entity methods with <T> params can easily get component type using ‘T.Type’. For example, here’s Entity.NumComponents<T>()

    Another, non-generic way to implement NumComponents could be:

    I haven’t decided which way is the best/right way yet, but I just thought I’d throw in both ways for now.

    Note that the actual component management mechanism in Entity is very simple – it’s just a Stack<Component> – and could theoretically be souped up with Maps or something. But it also could be that simpler is better in this case as the actual component insert/count/remove etc code is very simple and it may not be worth making it more complex. Would definitely want to test before changing it anyway.

    I haven’t done anything to Camera, Light, Renderer, Sprite etc yet, but my plan right now is to implement a CameraComponent (as you have above) and rewrite the Camera entity so it just redirects to CameraComponent. Ditto with Light. This provides a backwards-ish compatiblity layer for blitz3d users and is just plain easier to use for lazy coders like myself.

    Renderable will probably go, and something like MeshRenderer added. A Model subclass can also be added to blitz3d-ify MeshRenderer. Not quite sure what Model.Load will do any more yet though!

    Note that I haven’t pulled out the matrix stuff into a Transform component yet. I may do this later, but can’t really see the point right now except for the fact it’s a bit cleaner conceptually. The downside of  having a Transform component is it’s a bit slower as there’s an extra indirection for each matrix or parent access (OK, not a big deal) and a bit more fiddly code-wise.

    There are probably gonna issues with all this, but so far I like it. I hacked together a quick GameController component that behaves like a virtual joystick, with ‘ButtonDown’ etc methods, and a simple KeyboardController subclass that implemented keyboard input. I then moved the FlyAround code in util.monkey2 to a component that used the current GameController component for input.

    So creating a flyable camera ended up looking like:

    There is of course a cost to all this, but IMO it’s minimal and defintely bearable – probably just the cost of the  Stack<Component> allocation (plus underlying array) per entitiy created.

    I guess you’re not really meant to subclass components, but I have done in several cases such as this (also, Collider is subclassed by BoxCollider, SphereCollider) and don’t see it as a problem just yet – as long as all subclasses have the same ComponentType that is. Otherwise, GetComponent etc may get ugly to implemement.

    Is everyone OK with mojo3d going in this direction? It’s a bit of a major change, but it *does* make mojo3d way more flexible and expandable. I also hope you’re not too bummed out that I’ve gone in this direction too, Ethernaut! I do think your reflection/serialization stuff will be an invaluable addition to this, and eventually a reflection based GUI ‘object editor’ system along with custom editors for certain classes/types that could be implemented as ted2go plugins. A sort of ‘open’ RAD system with no fixed functionality.

    October 18, 2017 at 9:43 am #11168

    Ethernaut
    Participant

    Not sure how happy you’ll be about this (hopefully happy!) but I actually started on a component system for mojo3d

    Very happy, considering everything I make by myself tends to be 178.3% slower and 451% more bug infested, on average.

    It does mean I have to rewrite a bunch of stuff (again, since I had originally started something similar for Monkey-X!), but that’s Ok, I feel like I can probably add extra features on top of this and keep things cleaner that way.

    I’ll definitely want to add serialization at some point. I was playing with bringing 3D models from Houdini today (for my game project, not included in the Github repo), and not recompiling every time I make a tiny change is way more pleasant!

    Whatever you can do to make life easier in this regard is welcome, like always offering an optional New() constructor without arguments. The other thing I had to do on my system to make deserialization possible is being able to refer to objects by name. So Thing like Materials, Textures, GameObjects, Components, etc., can all be referred to by unique names, so you can set an Entity’s parent by name, for instance, or a material can refer to an existing texture by name, etc.

    Or I can keep this stuff “on top” like it is now.
    Cheers!

    October 18, 2017 at 10:55 am #11170

    nerobot
    Participant

    Wow! I like it.

    To construct generic-like system we already have minimal ‘reflection w/o reflection’ via typeName:

    Code for Map’s fans as me 🙂

    I like this generic way because

    • you can’t compile if did mistake
    • avoid to use ComponentType (at least for a first look) (with moving priority + unique properties into component class)

     

    Maybe ‘Unique’ might be a better name for this to prevent confusion?

    +1.

    Also Enabled propertywould be useful.

    October 18, 2017 at 12:31 pm #11176

    DruggedBunny
    Participant

    Can somebody dumb-down “components” for me? I’ve seen reference to component systems done in Blitz before, but never really grasped what they are/do, and I don’t get what they’re actually doing in these examples…

    October 18, 2017 at 1:34 pm #11178

    AdamStrange
    Participant

    Agreed.

    There is loads of talk about adding ‘this’ and ‘that’ to the language and NOTHING about how to actually use what is already there. Fibers anyone?

    There is a real feeling of ‘feature creep’ going on with Monkey2. Why not concentrate on getting it usable.

     

    There is a danger that it becomes the answer to the question that no-one was interested in!

     

    If you are serious about adoption of Monkey2 by anyone. the real task should be about documentation. and spelling out how thing are done and how they can be done. what are the best ways to do things, etc.

    October 18, 2017 at 1:37 pm #11179

    AdamStrange
    Participant

    and… what is the difference between a component and an entity?

    What about plugins and factories, Whilst we are at it I’ve got an old fridge that we could make a module of, stuff it with reverse extends and sell it, etc, etc

    October 18, 2017 at 1:46 pm #11181

    AdamStrange
    Participant

    and another thing…. (I’m on a roll here, possibly a ham one?)

    Am I the only one that this gives the shudders to?

    You are (in essence) abstracting stuff to the point of irrelevance.

    This all looks to be about data hiding. What happened to writing code that did something?

     

    So, I load the demo, look at the source (to understand what is going on, to learn from it), the source gives me nothing, just wrappers around data that is further hidden, that I then need to find the source of and then need to get the modules  of in the hope it might explain what is going on, but that is a wrapper round a secondary function of a class which has been subclassed and extended (cause the original author thought it would be great to use all privates and not expose anything).

    And on it goes.

    The key phrase I was taught by very large companies when coding is ‘CLARITY’ and KISS (Keep It Simple Stupid)

    October 18, 2017 at 6:55 pm #11188

    Mark Sibly
    Keymaster

    Can somebody dumb-down “components” for me?

    Component systems provide a way to create new types of entities by ‘gluing together’ component pieces at runtime.

    In some ways, both blitz3d and mojo3d are already crude component systems. Some of what an entity does is not coded into the actual entity, but coded via an optional object such as an Animator or RigidBody that is ‘attached’ to an entity when needed. This is done behind the scenes in blitz3d, but is more explicit in mojo3d, eg: New RigidBody( entity ) both creates a RigidBody ‘component’ and adds it to an entity in one step. Using a separate RigidBody object here as opposed to just hacking *everything* into Entity helps keep Entity much simpler and lighter.

    But Animator and RigidBody in mojo3d are currently attached to bodies in ‘ad hoc’ ways. Entity has an ‘Animator’ field to hold an optional animator object, but RigidBody uses a wonky ‘dynamic property’ system. This is because I want to keep physics in an optional module, so Entity can’t ‘know’ about physics, so it can’t have a RigidBody field.

    Adding a Component system sort of generalizes all this and provides a consistent way to create components like Animator and RigidBody that can be optionally added to an Entity. Instead of Entity having an ‘Animator’ field etc, it just has an internal Stack<Component> that stores *everything* attached to the entity. When the entity needs to Update, it  calls an OnUpdate Method inside each component in this stack, when it wants to render, ditto etc (although currently component system is not used fo rendering).

    There is some similarity here to adding ‘child’ entites to an entity. But a child entity cannot ‘drive’ a parent the way a parent can ‘drive’ a child (drive=move etc) so things like RigidBody and ‘update scripts’ cannot be done like this. And this is where the simple blitz 3d entity approach falls down really. If I were to stick with a ‘pure’ blitz3d system and make RigidBody just a new type of entity, users would have no choice but to make RigidBody the ‘root’ of any entity hierarchy they wanted to apply physics to. This is doable, but is very inconvenient compared to just being able to attach a rigidbody to any entity.

    There are lots of cool things about all this. For me, it makes my life easier as I now have a consistent way to create things that can optionally be attached to entities. And of course, anyone can write their own components using the same system – which also makes them a cool way to share code. Components can be combined in different ways, eg: the KeyboardContooller component described above can be easily switch to a JoystickController, even at runtime if you want. There is nothing mind bindingly complicated about being able to do this – but it does at the very least you don’t need to ‘roll your own system’ to handle it.

    Note that at the extreme case, it’s possible to create a component system where an Entity has pretty much NO ‘functionality’ (well, except for AddComponent!) and does everything via components – even storing entity matrix and parent in a component. I’m not going to go that far (mostly I suspect because I’m just not that good a programmer) but it can be done.

    I also haven’t converted Camera, Light etc to compnents yet. I would like to do this eventually, but would still like to provide blitz3d stylee Camera, Light entities too so you don’t have to deal with any component stuff if you don’t want to.

    Finally, yes, this *is* a little more complex than blitz3d’s approach, but IMO it’s worth it. I agree with KISS, but not all solutions *can* be simple, ie: ‘make it as simple as possible, but not simpler’. I don’t want be afraid of adding features that are more complex than I’m used to and that make take me outside my comfort zone a bit (and this definitely qualifies), although for sure I do want to come up with the nicest/simplest solutions I can.

    And seriously Adam, aren’t you just gonna write your own 3d engine from scratch anyway no matter what I do?!?

    October 18, 2017 at 7:03 pm #11190

    Mark Sibly
    Keymaster

    …here’s a nice piece on component systems, mostly as it appies to ‘game logic’:

    http://gameprogrammingpatterns.com/component.html

    In fact, that whole ‘book’ is really nice:

    http://gameprogrammingpatterns.com/

    October 18, 2017 at 7:15 pm #11192

    Ethernaut
    Participant

    I apologize, the examples I have are kinda terrible and were made only for my own testing. There are classes within classes that are not necessary. My bad, I blame my own incompetence. Will try to add something more informative when I have time.

    That said, my take on it is to put more emphasis on the data than on the code.

    It may help to know that I want to design both the assets and the scenes themselves from within a 3D software (SideFX Houdini is the best candidate so far, but it can be Blender, Maya, etc.), export everything to FBX and Json files, and use M2 code to load and run the scenes, occasionally creating new components as M2 code and attaching said components to entities from within Houdini.

    if it sounds too complicated, it isn’t: The assets are already being created in Houdini, so instancing them in a separate 3D scene and making a few scripts to export stuff out is the next logical step, and is actually a simpler pipeline (from an artist’s perspective) than doing everything in code.

    As for the other question, “What’s the point of components”?… components are little bits of functionality added to entities. You can create complex behaviors by combining multiple small components (instead of subclassing and inheriting everything, OO style). For instance (not actual code!):

    Entity “Dude”
    | Component: “Transform”
    | Component: “Sprite Render”
    | Component: “Collider”
    | Component: “Platformer Movement”
    | Component: “Character stats”
    | Component: “Player_Input”

    This is your main character. To make an enemy, simply make a new entity and add those components:

    Entity “BadDude”
    | Component: “Transform”
    | Component: “Sprite Render”
    | Component: “Collider”
    | Component: “Platformer Movement”
    | Component: “Character stats”
    | Component: “Cause Damage to Stats on Collision”
    | Component: “AI_Input”

    As you can see, we re-used a few components, but added a couple different ones. The “Platformer Movement” component always looks for an “Input” component, so in one case it is controlled by a human, on the other it is controlled by AI, yet the actual platforming (Jumping, colliding, etc.) is independent from where the input is coming from.

    Which components are used can easily be defined in an external editor, and reloaded from the engine at any time, allowing quick testing and iterations.

    Adam, I understand your stance, and certainly considered it in the past. But this approach feels right to me, and is actually quite popular in game engines these days. I also get to leverage my experience in film animation pipelines, where your data is your gold and all editing is done in 3D editors, which I’m very familiar with.

    Hope that clarifies it for you even if you still don’t agree, which I totally respect 🙂
    Cheers!

    October 19, 2017 at 5:02 am #11196

    AdamStrange
    Participant

    First let me thanks both Mark and Ethernaut for having patience with me.

    I especially liked the references detailing how components work, abstraction, etc. Much clearer now. Thanks Mark.

    And yep (to Mark). I will ‘roll my own’ 3d renderer. But not because I didn’t want to use Mojo3d (I did try). It just has a different approach which placed to many restrictions on me experimenting with things.

    October 19, 2017 at 12:30 pm #11207

    DruggedBunny
    Participant

    Thanks for the in-depth explanations, chaps! Basically get it now, though I suppose proper understanding will just comesdown to how it’s applied in real-world programs.

  • Author
    Posts
Viewing 15 posts - 1 through 15 (of 15 total)

You must be logged in to reply to this topic.