Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dividing Clients and Entities into smaller components #199

Closed
rj00a opened this issue Jan 26, 2023 · 3 comments · Fixed by #294
Closed

Dividing Clients and Entities into smaller components #199

rj00a opened this issue Jan 26, 2023 · 3 comments · Fixed by #294
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@rj00a
Copy link
Member

rj00a commented Jan 26, 2023

Brainstorming some ideas for how the monolithic Client and McEntity components could be split up into a set of smaller components. Here are some reasons you would want to do this:

  • Improve query performance by increasing parallelism and useful data in the CPU cache.
  • More modular design can split code into smaller understandable pieces.
  • Integrate with Bevy's change detection.
  • Share data between client and Minecraft entity when they share the same ECS entity. (Typically the client will have a player entity associated with it).
  • Save memory by spawning entities without unused components.

Some reasons you wouldn't want to do this:

  • Many corner cases and interactions that are not accounted for can lead to bugs.
  • Potentially confusing and unintuitive.
  • Invalid states are very much representable.

Component Name Description Effect When Added Effect When Removed Effect When Modified
Despawned Any ECS Entities with this component are despawned at the end of the tick. This gives Valence an opportunity to perform deinitialization of components that can't be removed explicitly. In particular, McEntity and its other components.
Client The main client component. Contains the TCP connection and packet buffers. Also contains immutable info such as username, UUID, and IP among other things. Joins the client (Should only be done by Valence). Maybe we want to allow attaching Client to existing entities somehow? No effect, but the client is disconnected when dropped. Removing only this component from an McEntity will allow you to take control.
Disconnected Marker component to indicate that a client is disconnected.
Location Contains the instance an Entity or Client is inside. Also contains the instance from the previous tick. Disallowed at least for entities because we need it to clean up the instance. Changes the instance a client or entity is in. Makes clients respawn even if old_instance == new_instance. Can be used to respawn the client after death.
Position The XYZ coordinates (old and new) of a client or entity. Automatically updated by Valence as a client moves around. Disallowed at least for entities because we need it to clean up the instance. Teleports clients and entities when modified
Direction Contains the pitch/yaw pair of an entity or client. Automatically updated by clients as they look around. Disallowed(?) Sets the pitch/yaw of entities and teleports clients to set their pitch/yaw. Can be combined with Position modifications to form a single teleport.
Velocity Contains the velocity vector for clients and entities. Causes clients to be launched. Unknown effect on entities. (Velocity is really weird in Minecraft).
OnGround The on ground boolean for clients and entities. Automatically updated by clients. Changes the on ground state for entities.
Vehicle The vehicle entity a client or entity is riding. TODO TODO TODO
DeathLocation Contains a (DimensionId, BlockPos) pair. Used by respawning clients to set their recovery compass position Can be modified, but won't have an effect until the client is respawned.
GameModeComponent (GameMode was taken) Contains the game mode of a client. Disallowed(?) Sets the gamemode of the client it's attached to.
McEntity The main component for Minecraft entities. Contains protocol entity ID and UUID. Spawns the entity. Must have Location and Position to do so. Sets the protocol ID to the next unique value. Was initialized to zero when the component was created. Definitely disallowed because we need the protocol ID to send despawn packets to clients. Also, we need the position+location to remove its ID from the instance it was in. Otherwise, there would be a memory leak.
EntityKindComponent (EntityKind was taken) Contains the type of Minecraft entity (Blaze, Chicken, Player, etc). Disallowed(?) Respawns the entity as the new type. All metadata is reset. Protocol ID is reused(?)
*Metadata, where * is every entity type. Component and system definitions would be generated. Contains the metadata (TrackedData) of a particular entity type. TrackedData is currently an enum, so this would save a lot of space. Not to mention easier because it's queryable and you wouldn't have to match on any enums. Initializes the metadata. Would be nice if we allowed spawning entities without metadata to save memory. Clears the metadata(?) Updates the entity's metadata. Has no effect if component does not match the entity's type as determined by EntityKindComponent. When PlayerMetadata is modified on a client, the changes are sent to the client's own player entity.
ExpOrbCount The amount of experience in an exp orb entity. Changes its visuals. Only used when spawning exp orb entities. Modifications have no effect afterwards.
FallingBlockState The BlockState of falling block entities Only used when spawning falling block entities. Modifications have no effect afterwards.
Statuses Bitset containing a bit for each entity status. Cleared at the end of the tick. Triggers any set statuses for an entity. Triggers any set statuses for an entity.
Animations Bitset containing a bit for each entity animation. Cleared at the end of the tick. Triggers any set animations for an entity. Triggers any set animations for an entity.
Hitbox Contains the AABB hitbox for an entity. Updated at the end of each tick. No effect, but likely to be overwritten at the end of the tick.

Questions

  • How granular should the components be? Should there be Location/OldLocation and Position/OldPosition, or is that kind of thing too cumbersome?
  • What about miscellaneous data in Client like is_hardcore, is_flat, etc?
  • Should UUID be shared between clients and entities as a component? Should it not be a component because it's immutable?
  • Should the protocol ID of entities be a component? This would be immutable too.
  • Should metadata components be broken up further to match Minecraft's class hierarchy? That seems very challenging.
  • Working with metadata internally when there are a million different component types might be challenging. Probably need to cache metadata initialization bytes in McEntity.
  • What about the inventory stuff for clients?
  • What about poses?
  • Does McEntity need to exist? It's kind of an ugly name.
  • What is the most effective way to disallow removing components? Log an error? Panic in debug mode?
@rj00a rj00a added documentation Improvements or additions to documentation enhancement New feature or request help wanted labels Jan 26, 2023
@rj00a rj00a pinned this issue Jan 26, 2023
@dyc3
Copy link
Collaborator

dyc3 commented Jan 26, 2023

This will be great to have. Can we get an ecs rewrite label on this?

Location seems like a bad name because it can be easily confused with Position.

For block entities: I would like to have a BlockPos component in addition to the Position component. Position would be able to handle sub-block positions (eg. x=3.5) and BlockPos is just for integer coordinates (eg. x=3)

Another thing that would be nice is to have a better way to inject received packets and assert sent packets for unit tests, ideally without having a serialization pass both ways. #198 currently requires a round trip for injecting packets, and a round trip for asserting read packets. This might be possible if we had client packets come in and be processed via an EventReader and out via an EventWriter. Maybe we could have traits PacketStreamSource and PacketStreamSink

Edit: This is no longer relevant, we've made the changes to make unit tests work.

Could look like this?

erDiagram
    PacketStreamSource ||--|| Client : injects
    PacketStreamSink ||--|| Client : extracts
Loading
classDiagram
    class PacketStreamSource {
        +recv()
    }

    class PacketStreamSink {
        +send()
    }

    class MockPacketStreamSource {
        queue
        +recv()
    }

    class RealPacketStreamSource {
        socket
        +recv()
    }

    RealPacketStreamSource --|> PacketStreamSource
    MockPacketStreamSource --|> PacketStreamSource

    class MockPacketStreamSink {
        record
        +send()
    }

    class RealPacketStreamSink {
        socket
        +send()
    }

    RealPacketStreamSink --|> PacketStreamSink
    MockPacketStreamSink --|> PacketStreamSink
Loading

Not a comprehensive diagram by any means, but just brainstorming


Regarding some of your questions:

Should there be Location/OldLocation and Position/OldPosition, or is that kind of thing too cumbersome?

That sounds incredibly jank.

Working with metadata internally when there are a million different component types might be challenging. Probably need to cache metadata initialization bytes in McEntity.

We might be able to do something with traits: https://bevy-cheatbook.github.io/patterns/generic-systems.html#using-traits

What about the inventory stuff for clients?

The Inventory and OpenInventory kinda already implement this design philosophy, so we don't need to worry about that. Regarding the inventory-related properties on Client, maybe have a new component named ClientInventoryMeta or similar?

Does McEntity need to exist? It's kind of an ugly name.

Yes, definitely. It should denote that the Entity it's attached to is directly correlated to a Minecraft entity.

What is the most effective way to disallow removing components? Log an error? Panic in debug mode?

The most effective way would be to put an API layer over all the ecs stuff that doesn't allow that stuff. This seems incredibly impractical, but certainly would be effective.

Logging would probably be the preferred way to go about it. People get angry when their services shit themselves and die. Panics should be reserved for states that are absolutely and totally unrecoverable.

It might not be worth the effort at all. Consider that it may be significantly easier to just make it so that doing these things are undesirable compared to whatever is "best practice".

@rj00a
Copy link
Member Author

rj00a commented Jan 27, 2023

Think I'm going to start working on this in earnest after #184 is merged.

Location seems like a bad name because it can be easily confused with Position.

Agree. Also I believe Minecraft has a concept of "location" which is an XYZ coordinate paired with a dimension (hence DeathLocation), so my use of location would be inconsistent with that. But I'm not sure what else to call it.

For block entities: I would like to have a BlockPos component in addition to the Position component. Position would be able to handle sub-block positions (eg. x=3.5) and BlockPos is just for integer coordinates (eg. x=3)

The position of a block entity would be known by its position in the Chunk data structure, so I don't think a BlockPos component is needed. It's useful if you need to know where a block entity is during a query, but it isn't anything that Valence needs to know about strictly speaking.

Another thing that would be nice is to have a better way to inject received packets and assert sent packets for unit tests, ideally without having a serialization pass both ways. #198 currently requires a round trip for injecting packets, and a round trip for asserting read packets. This might be possible if we had client packets come in and be processed via an EventReader and out via an EventWriter. Maybe we could have traits PacketStreamSource and PacketStreamSink

Originally clients were designed to work exactly like this. Clients had a pair of channels for sending and receiving packets. Encoding and decoding of those packets were handled elsewhere by a tokio task in another thread. Simple as that.

Problem is, this is measurably less efficient than the current approach for a few reasons:

  • Using a channel (Or EventReader/EventWriter) means the packets need to be 'static. For instance, writing a packet that contains a &str requires us to copy the string into a heap allocated String before we can actually encode it. This isn't a problem for received packets because we need to put the data in a client event anyway, but for sent packets it makes a difference.
  • Using channels means we need to buffer packets in a homogeneous list. We need to use an enum of every possible packet. The size of that enum is the size of the largest variant, so that's a lot of wasted space. A stream of bytes is much more compact. We still have packet enums, but those are ephemeral and only used during decoding.
  • Impossible to cache clientbound packets. We always need to send actual packet objects and not just raw bytes that we know are valid packets. Packet caching is what really makes Valence fast.

Here is what I propose: Add a public constructor to Client that takes the byte channels to read from and write to. We could even take this a step further and wrap all of Valence's tokio and networking code in an optional Bevy plugin. Not only is this good for testing, but Valence can then be used as an integrated server for a singleplayer experience. Pretty cool.

We might be able to do something with traits: https://bevy-cheatbook.github.io/patterns/generic-systems.html#using-traits

I guess in theory we could make all of our internal systems that work on entities generic and register hundreds of monomorphized systems, but that's obviously not very practical. Even then I think there would have to be some major algorithm changes.

Combining EntityKindComponent and the metadata components into a single enum (as it exists currently) seems like the way to go. Best to use a marker component if you need to filter entity types while querying.

Yes, definitely. It should denote that the Entity it's attached to is directly correlated to a Minecraft entity.

How about the name ProtocolEntity?

Logging would probably be the preferred way to go about it. People get angry when their services shit themselves and die. Panics should be reserved for states that are absolutely and totally unrecoverable.

It might not be worth the effort at all. Consider that it may be significantly easier to just make it so that doing these things are undesirable compared to whatever is "best practice".

Panics in debug mode with warnings in release mode sound like the way to go. I don't think this would be very difficult to set up actually. All we need is a check_component_invariants system that runs right after the user stage. If Archetype Invariants become a thing we could use that.

@dyc3
Copy link
Collaborator

dyc3 commented Jan 27, 2023

Also I believe Minecraft has a concept of "location" which is an XYZ coordinate paired with a dimension

Ah, I see. Location seems fine then.

How about the name ProtocolEntity?

I think that's less clear.

@rj00a rj00a unpinned this issue Feb 15, 2023
rj00a pushed a commit that referenced this issue Feb 20, 2023
<!-- Please make sure that your PR is aligned with the guidelines in
CONTRIBUTING.md to the best of your ability. -->
<!-- Good PRs have tests! Make sure you have sufficient test coverage.
-->

## Description

<!-- Describe the changes you've made. You may include any justification
you want here. -->
Dropping items is heavily coupled to inventories. Clients also predict
state changes when they try to drop items, so we need to be able to
replicate that change in order to stay in sync.

This will also remove `DropItem` events in favor of just `DropItemStack`
events. Having 2 event streams that basically mean the same thing seems
verbose and error prone.

As of right now, these changes require the event loop to have a
reference to the client's inventory. This seems like something we are
going to need to do a lot more of to complete #199.

## Test Plan

<!-- Explain how you tested your changes, and include any code that you
used to test this. -->
<!-- If there is an example that is sufficient to use in place of a
playground, replace the playground section with a note that indicates
this. -->

<details>

<summary>Playground</summary>

```rust
use tracing::info;
use valence::client::despawn_disconnected_clients;
use valence::client::event::{default_event_handler, DropItemStack};
use valence::prelude::*;

#[allow(unused_imports)]
use crate::extras::*;

const SPAWN_Y: i32 = 64;

pub fn build_app(app: &mut App) {
    app.add_plugin(ServerPlugin::new(()).with_connection_mode(ConnectionMode::Offline))
        .add_system_to_stage(EventLoop, default_event_handler)
        .add_startup_system(setup)
        .add_system(init_clients)
        .add_system(despawn_disconnected_clients)
        .add_system(toggle_gamemode_on_sneak)
        .add_system(drop_items);
}

fn setup(world: &mut World) {
    let mut instance = world
        .resource::<Server>()
        .new_instance(DimensionId::default());

    for z in -5..5 {
        for x in -5..5 {
            instance.insert_chunk([x, z], Chunk::default());
        }
    }

    for z in -25..25 {
        for x in -25..25 {
            instance.set_block([x, SPAWN_Y, z], BlockState::GRASS_BLOCK);
        }
    }

    world.spawn(instance);
}

fn init_clients(
    mut clients: Query<&mut Client, Added<Client>>,
    instances: Query<Entity, With<Instance>>,
) {
    let instance = instances.get_single().unwrap();

    for mut client in &mut clients {
        client.set_position([0.5, SPAWN_Y as f64 + 1.0, 0.5]);
        client.set_instance(instance);
        client.set_game_mode(GameMode::Creative);
    }
}

fn drop_items(clients: Query<&Client>, mut drop_stack_events: EventReader<DropItemStack>) {
    if drop_stack_events.is_empty() {
        return;
    }

    for event in drop_stack_events.iter() {
        info!("drop stack: {:?}", event);
    }
}

```

</details>

<!-- You need to include steps regardless of whether or not you are
using a playground. -->
Steps:
1. `cargo test -p valence --tests`
2. Run playground `cargo run -p playground`
3. Open creative menu
4. Pick an item and click to drop it outside of the creative menu
5. Pick an entire stack of an item, place it in your hotbar
6. Hover over the item, press your drop key to drop an item from the
stack
7. Press shift to switch to survival
8. Select the item stack in your hotbar, press your drop key to drop an
item from the stack
9. Open your inventory, grab the stack, hover outside the window and
click to drop the entire stack
10. Grab another stack from creative, place it in your hotbar
11. switch back to survival, select the stack, and press your control +
drop key to drop the entire stack
12. For each item you dropped, you should see a log message with the
event

#### Related

<!-- Link to any issues that have context for this or that this PR
fixes. -->
This was referenced Mar 3, 2023
rj00a added a commit that referenced this issue Mar 11, 2023
## Description

Divides the `Client` component into a set of smaller components as
described by #199 (with many deviations). `McEntity` will be dealt with
in a future PR.

- Divide `Client` into smaller components (There's a lot to look at).
- Move common components to `component` module.
- Remove `Username` type from `valence_protocol` because the added
complexity wasn't adding much benefit.
- Clean up the inventory module.

I've stopped worrying about the "Effect When Added" and "Effect When
Removed" behavior of components so much, and instead assume that all
components of a particular thing are required unless otherwise stated.

## Test Plan

Steps:
1. Run examples and tests.

A large number of tweaks have been made to the inventory module. I tried
to preserve semantics but I could have made a mistake there.

---------

Co-authored-by: Carson McManus <[email protected]>
Co-authored-by: Carson McManus <[email protected]>
@rj00a rj00a mentioned this issue Mar 12, 2023
56 tasks
@rj00a rj00a mentioned this issue Mar 19, 2023
3 tasks
@rj00a rj00a closed this as completed in 4cf6e1a Mar 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants