* feat(protocol): Implement char serialization and deserialization
This commit introduces support for serializing and deserializing individual characters within the pumpkin-protocol. Characters are handled as big-endian u32 Unicode scalar values.
Key changes:
- Implemented `Serializer::serialize_char` to write a `char` as a `u32`.
- Implemented `Deserializer::deserialize_char` to read a `u32` and convert it to a `char`, including validation for invalid scalar values.
- Added a new test suite `test_char_reserialize` with cases for ASCII, multi-byte Unicode characters (e.g., 'Ω'), and characters requiring 4-byte UTF-8 representation (e.g., '🎃').
- Corrected warnings related to unused mutable variables in the new tests.
Additionally, to support the testing and building of related crates:
- fix(build): Enabled the `io-util` feature for the `tokio` workspace dependency in the root `Cargo.toml`. This resolved compilation errors in the `pumpkin-world` crate by providing necessary async I/O utilities like `AsyncReadExt` and `AsyncWriteExt`.
* feat(protocol): Implement i128/u128 serialization and deserialization
This commit adds support for serializing and deserializing 128-bit integer types (i128 and u128) in the pumpkin-protocol. These values are written and read as 16 bytes in big-endian order.
Key changes:
- Implemented `Serializer::serialize_i128` and `Serializer::serialize_u128` to write i128/u128 values.
- Added `NetworkReadExt::get_i128_be` and `NetworkReadExt::get_u128_be` for reading these types.
- Implemented `Deserializer::deserialize_i128` and `Deserializer::deserialize_u128`.
- Added new test cases `test_i128_reserialize` and `test_u128_reserialize` to ensure correctness for both positive and negative i128 values, and positive u128 values.
* feat(protocol): Implement unit type (de)serialization
This commit introduces serialization and deserialization logic for Rust's unit type `()` and unit structs.
- `Serializer::serialize_unit` now correctly writes no bytes for unit types, as they carry no value.
- `Deserializer::deserialize_unit` and `Deserializer::deserialize_unit_struct` are implemented to
consume no input and correctly reconstruct unit types/structs.
- Added `test_unit_reserialize` to verify that serializing and deserializing unit structs, both standalone and as fields within other structs, behaves as expected (i.e., no bytes are written or read for the unit components).
* feat(protocol): Implement enum (de)serialization for all variant types
This commit introduces comprehensive support for serializing and
deserializing Rust enums, including unit, newtype, tuple, and
struct variants.
Key changes:
- `serializer.rs`:
- Implemented `serialize_struct_variant` to correctly write the
variant index (as a VarInt) and then delegate to the
`SerializeStructVariant` trait for field serialization.
- Implemented `SerializeStructVariant` to serialize individual
fields of a struct variant.
- `deserializer.rs`:
- Implemented `deserialize_enum` to use `visitor.visit_enum`.
- Implemented `EnumAccess` to read the variant index (as a VarInt,
converted to u32) and deserialize the correct variant.
- Implemented `VariantAccess` to handle deserialization of
unit, newtype, tuple, and struct variants.
- `ser/mod.rs`:
- Added `test_enum_reserialize` to thoroughly test (de)serialization
of enums with unit, newtype (e.g., `Enum::B(i32)`), and struct
variants (e.g., `Enum::C { field: T }`).
This ensures that enums with various structures can be reliably
transmitted over the network protocol.
* cargo fmt
* feat(protocol): Implement (de)serialization for tuple structs
This commit adds support for serializing and deserializing tuple
structs within the network protocol.
Key changes:
- `serializer.rs`:
- `serialize_tuple_struct` in the main serializer now delegates
to the `SerializeTupleStruct` trait implementation.
- Implemented `SerializeTupleStruct` to serialize fields of a
tuple struct sequentially.
- Updated `SerializeTupleVariant::serialize_field` to correctly
serialize tuple variant fields.
- `deserializer.rs`:
- Implemented `deserialize_tuple_struct` to reuse the existing
`deserialize_tuple` logic, as their serialization format is
identical.
- `ser/mod.rs`:
- Added `test_tuple_struct_reserialize` to verify the correct
(de)serialization of tuple structs.
These changes ensure that tuple structs are handled correctly by the
protocol.
* feat(protocol): Implement (de)serialization for maps
This commit introduces support for serializing and deserializing
maps (specifically `std::collections::HashMap`) within the network
protocol.
Key changes:
- `serializer.rs`:
- `serialize_map` now requires a known length for the map and
writes this length as a VarInt before serializing key-value
pairs.
- Implemented `ser::SerializeMap` to handle the sequential
serialization of map keys and values.
- `deserializer.rs`:
- `deserialize_map` reads the VarInt length prefix.
- Implemented `de::MapAccess` to deserialize key-value pairs
according to the read length.
- `ser/mod.rs`:
- Added `test_map_reserialize` to verify the correct
(de)serialization of `HashMap<String, i32>`, including
empty maps and maps with multiple entries.
These changes enable generic map types to be transmitted via the
protocol, with their length explicitly prefixed.
* inital chunk features
* add more things
great title i know
* add some features
still no result, but we are getting there...
* make it parse
* add simple BlockPredicate
* add simple random selector
* new things
* change stuff
* base tree impl
* change things
* add more heightmaps
* fix master merge
* steps towards proper random
* fix upper level block placement logic
* Make it compile
* fix block dir serde name
* Added Tree Foliage
* more Tree work
* Implement would_survive check
I don't say its good, but it works :D
* Fixed biome placement modifier (#836)
* Implement all tree foliages
* Fix grass, flowers, seagrass, seapickles...
* Add Ores
* add Bamboo, Vine, Spring feature
Also more fixes
* Corals
* Add Tree decoration & more
Many many fixes
* fix vines
* fixed bamboo
* fix some clippy warns
* fix trees outside of chunk
* make it compile again
upsi
* fix clippy warns
* fix InsideWorldBoundsBlockPredicate
* set dirt under tress trunk
dirt, not grass
* more trees work
---------
Co-authored-by: kralverde <github@email.kralverde.dev>
Co-authored-by: Laurent Stéphenne <laurent@guibi.dev>
* feat(player-data): Support saving and loading player NBT Data
* fix(player_data): ensure player data is save before continuing
* fix(player_data): implement NBTStorage for hunger data
* feat(player-data): allow setting world name in config
* fix(player-data): use correct method to retrieve world_name
* feat(player-data): move PlayerDataStorage to pumpkin-world
* fix(player-data): ci not building
* fix(player-data): ci not building
* feat(player-data): support inventory saving + review fix
* fix(player-data): forgot the format
* some fixes
* fix clippy
---------
Co-authored-by: Alexander Medvedev <lilalexmed@proton.me>
* feat: add linear chunk format with zstd lib dependency
* fixing a typo
* refactor: rename file handling methods for clarity and more lint fixes
* refactor: improve clarity on validation methods
* fix: invert version check in LinearFile header validation
* fix: remove leading dot from file path formatting in LinearChunkFormat
* refactor: improve error handling and method naming in LinearFile operations
* fix: ensure file is flushed after writing in LinearFile operations
* fix: lints and warnings
* fix: more lints
* refactor: streamline LinearFile structure and improve header management
* refactor: change chunks_headers to use Box for improved memory management and use exception to know if file exist
* refactor: update path parameter types to PathBuf for consistency and improved usability
* refactor: update path parameter types to Path avoiding uknown changes and adjust file truncation logic
* fix: avoid using the same path for different tests
* fix: fix some bytes sizes for the header based on the spec
* refactor: enable file truncation when opening for writing and simplify save method
* fix: removing IoSlices for windows issues.
* fix: clippy lints
* refactor: improve incompatible files cheking, some documentation and variable nameing
* feat: add file_format to ChunkConfig and implement format handling in Level
* feat: add error logging for invalid file headers and data size checks in LinearFile and tmp filenames on write
* feat: implement file locking mechanism for concurrent chunk read/write operations
* fix: bug at desserializing the chunks headers.
* feat: (Bulk API) refactor chunk reading and writing to handle multiple chunks at once
* refactor: streamline chunk reading and writing methods, improve chunk management logic
* feat: enhance chunk reading and writing to support bulk and improve concurrency
* feat: add file locking during chunk writing to ensure safe concurrent access
* fix: resolve some issues with the merge
* fix: update chunk header size calculation in linear chunk parsing
* refactor: optimize chunk processing and improve concurrency handling
* feat: implement chunk I/O management with file locking for safe concurrent access
* refactor: simplify public load method and fix clippy issues
* feature: add file cache for files with ongoing IO ops
* refactor: update LoadedData enum variants for consistency and clarity and include docs.
* refactor: restore compression enum and replace ChunkSerializingError with ChunkWritingError for improved error handling
* refactor: enhance chunk cache cleaning logic and improve logging during chunk fetching
* refactor: optimize chunk data handling with parallel processing and improve chunk fetching logic
* fix: lints and logging
* fix: correct typo and import format
* refactor: anvil code organization and llinear chunk reading code.
* fix logging issue and refactor some locks bindings
* refactor: reduce chunk cloning, ensure drop bindings/guards
* refactor: reduce some threading overhead when not needed
* fix: clippy lints
* refactor: simplify chunk data processing in Anvil and Linear serializers
* fix: clippy lints rust 1.84.1 to 1.85.0
* add file drop guard and rework some logic
* refactor: improve AnvilChunkFile and AnvilChunkData serialization logic and docs
* fix: typos
* feature: initial async implementation of Bulk API
* fix: chunk writing with async read
* refactor: improve async handling and use asyn IO for reading files
* fix: sorry i delet a mod xd
* refactor: enhance chunk management with async locks and OneCells
* use async methods where avaliable and stream chunks instead of collecting them
* refactor: streamline chunk management by removing unnecessary mutex and optimizing cache cleanup
* Split data
* Move format related stuff from chunk
* refactor: decoupling Serializer from ChunkIO and reduce CPU usage by 25%-50% when flying as spectator at max speed
* refactor: imrpove CPU usage marking Heavy CPU tasks with tokio block_in_place and improve Docs/Comments
* refactor: improve linear format performance and imrpove some localize tokio::block_in_place for compression and uncompression in both formats, also for serialization and deserialization
* Remove parking_lot dependency from Cargo.toml
* feat: add futures crate for improved async handling and performance optimizations
* refactor: change some comments about blocking (not needed with futures::future::join_all) and zstd compression concerns
* refactor: remove unnecessary blocking in AnvilChunkFile methods for improved performance
* refactor: change receive_chunks parameter from slice to vector for improved flexibility
* fix: add timestamp tracking for chunk updates
* refactor: remove async_trait usage from chunk serializers
* fix: handle potential errors in chunk decompression and parsing
* fix: prevent parsing of empty chunks by updating condition for sector offset and count
* fix: improve dashmap reference management
* dont log if chunk does not exist and tweak help functions
* refactor: reduce CPU and Memory usage while improving Chunk Generation performance and reducing code complexity.
* fix: improve logging for file operations and ensure proper lock management during closure
* fix extra log
* fix: chunk generation threading issue (not using rayon spawn) and a emory leak in the level cache cleanup
* fix: handle expected chunk reading errors without logging as failures
* feat: add intial benchmark for chunk I/O performance and update criterion dependencies
* remove panic for bad chunk return
* refactor: comment out benchmark code for test without BulkAPI PR
* Refactor benchmarking code and remove unused chunk_io benchmark
* fix: remove dummy benchmark file and using paths/files instead of temp dirs
* refactor: update benchmark to re-instanciate the level on each iteration avoiding caching
* fix: clippy lints and add comments
* fix: correct chunk count logging in benchmarking tests
* refactor: replace hardcoded channel size with a constant, print seed used, add one more size to bench.
* use bytes for anvil and stream deserialization
* re-add streaming w/ capacities
* tweak rayon threads
* fix deadlock, max concurrency, box chunks early
* add parallel benchmarking for chunk read/write operations
* work on chunk batching
* add chunk batching
* fix chunk batching
* add comment to lru cache
* fix test
* fix spelling
---------
Co-authored-by: Mili <santiagoogle@hotmail.com>
Co-authored-by: kralverde <github@email.kralverde.dev>
Co-authored-by: suprohub <suprohub@gmail.com>
* simplify all
* fmt
* Consider using derive macro
* Add new events
* Add events call
* minor fixes
* minor fixes
* fix
* optimize code
* big changes
* fix skill issue
* fix
* Consider not specify patch version
* make logger shared
* i want test one thing
* Add one useful function
* add little amouth of docs
* use config instead
* revert changes
* fmt
---------
Co-authored-by: Alexander Medvedev <lilalexmed@proton.me>
we only used them for collect_vec. I think its worth the "effort" to write Vec type annotations ourself and have one big crate with way too may features we don't need less
* implement populate noise step of chunk generation
* fix gitignore
* pass thru seed
* add counter for edge case
* add entire chunk test case
* remove eclipse file
* remove packet timing
* precompute float divisions
* unsafe speed up
* slight aquifer optimization
* tweak block pos packing
* optimize perlin map
* precompute aquifer random positions
* fix aquifer regression
* refactor samplers, add tests to aquifer, and move chunk extractor to the extractors
* fix base density function error and add a bunch of test cases
* ignore expensive tests by default
* remove un-needed lifetimes
* remove minecraft prefix from block lookups
* fix populate noise bench
* slight optimizations
* read files at runtime to speed up test compilation speed and remove ignores
* remove new lines from test data so people stop freaking out about line count
* make extractor the same
* leverage branch prediction
---------
Co-authored-by: Alexander Medvedev <lilalexmed@proton.me>
* Remove fastnbt as dependency
* Move to own NBT crate
* Added deserialization
Also replaced i8 bool's with bool's
* Use: fastnbt for pumpkin-world
Until i don't figure out why Arrays do not work, we can use fastnbt
* ctrlc: also handle `sigterm` and `sighup` via `termination` feature
This commit enables the `termination` feature in `ctrlc` so that
`sigterm` and `sighup` are also handled on UNIX systems. This in turn
improves Docker compatibility since containers now listen to the
`SIGTERM` signal it sends to containers when trying to stop the
container via `Ctrl-C` whilst attached.
* Replace `ctrlc` signal handling with `tokio`
This removes the `ctrlc` crate from the project, replacing (and upgrading) its functionality to also handle `sigterm` and `sighup`.
Sigterm handling is working on UNIX with this commit, I haven't tested Windows compatibility with `Ctrl-C` nor have I tested `sighup` or `sigint`.
Of course, compiles and runs on UNIX. Haven't tested Windows.
* fix: remove unused import; collapse if statement
* main: fix per-os signal handling
Will only compile Windows-specific/Unix-specific code when on their respective environments.
Handles signals in a separate tokio thread so it doesn't suspend the main thread.
Tested to work on UNIX (macOS) but not tested on Windows.
* main: cargo fmt
* main: fix setup_sighandler fn signature on windows
* main: attempt 2 at fixing windows sighandler
* main: enable ctrlc for all non-unix systems
Previously, ctrlc usage in tokio was only enabled on Windows systems
(since UNIX systems have their own signal handling in Pumpkin). Instead
of limiting ctrlc to just Windows, we can make it run on any non-UNIX
system since it should be platform independent anyhow.
This reverts commit a1b97730bd.
Due to Errors
```
thread 'tokio-runtime-worker' panicked at pumpkin-protocol/src/client/play/c_chunk_data.rs:84:25:
index out of bounds: the len is 4096 but the index is 25374
```
