Commit Graph
465 Commits
Author SHA1 Message Date
black-sliverandFabian Dill 5442df64fd MultiServer, customserver: move data package handling, optimize RAM usage (#5981)
* MultiServer, customserver: move data package handling

Create a new class that handles conversion worlds+embedded -> context data.
Create a derived class that uses static_server_data+pony instead.
There is also a not very efficient feature to deduplicate strings (may need perf testing).

By moving code around, we can simplify a lot of the world loading.
Where code lines were touched, some typing and some reformatting was added.
The back compat for GetDataPackage without games was finally dropped.
This was done as a cleanup because the refactoring touched those lines anyway.

Also reworked the per-context dicts and the RoomInfo to hopefully be more efficient
by ignoring unused games. (Generating the list of used games was required for the new
code anyway.)

Side effect of the MultiServer cache: we now load worlds lazily (but still all at once)
and don't modify the games package in place. If needed we create copies.
This almost gets us to the point where MultiServer doesn't need worlds - it still needs
them for the forbidden items.
There is a bonus optimization that deduplicates strings in name_groups that may have bad
performance and may need some perf testing if we run into issues.

* MultiServer: fix string deduplication in data package cache

* MultiServer, customserver: minor formatting fixes

* MultiServer: fix data package cache for missing checksum case

* MultiServer: improve string deduplication in games package cache

* customserver: handle missing checksum in datapackage cache

* customserver: typing cleanup in games package cache

* Test, MultiServer, customser: add tests for games package cache

* Test, MultiServer: reorder imports

Hopefully this fixes the random test failures with pytest-xdist

* customserver: make WebHost import lazy in games package cache

and fix test

* customserver: games package cache: fix py3.11 compat

* MultiServer, customserver: cache: typing improvements

* customserver: cache: reformat

* Test: move customserver tests to not interfere with webhost

* MultiServer, customserver: cache: rename module

* Multiserver: better error message for missing Datapackage

* customserver: cache: invert if for readability

* customserver: optimize non_hintable_names

* Use an empty frozenset singleton
* Only include non-empty sets in static_server_data
* Only prefill dict with played games

This should save ~2-4KB per room and slightly speedup lookup.

---------

Co-authored-by: Fabian Dill <[email protected]>
2026-09-24 23:33:30 +02:00
NewSoupVi 527f1195c9 MultiServer: Correct Bounce Target logic, add unit tests (#6417)
* Correct Bounce Target logic, write unit tests

tmp

* that is not correct

* What was I smoking

* explicitly document None

* What was I smoking

* fix weakref syntax

* ruff

* fix magicmoc

* Address Berserker's comments

* comment

* consistently use self.assertEqual

* make slot 2 on team 3 not use deathlink to test different tags for the same slot on different teams

* Improve error message
2026-09-20 18:47:27 +02:00
NewSoupVi 9ae40fd6e4 MultiServer: Allow "and" chaining of Bounce packet conditions & add teams support (#3845)
* Allow 'and' concatenation of bounce packet conditions

* Docs + small change

* Docs stuff

* update Bounced as well

* lol

* shadowing issue

* Slight rework to allow teams

* teams note

* Make the PR ready

* docs further up too

* Let's not do this actually

* Use operator & don't redefine functions

* team default

* remove unnecessarily added whitespace
2026-08-17 21:31:41 +02:00
SomeJakeGuy bf52661182 MultiServer: reject None targets on Bounce and validate types before sending Bounced (#6397) 2026-08-17 00:04:31 +02:00
Duck 838d55327e MultiServer: Update hint_points when hint cost or check points is changed (#6219) 2026-08-05 00:12:23 +02:00
qwint 74a0cd6022 MultiServer: update help doc on auto_shutdown#5917 2026-04-29 21:21:42 +02:00
Silvris 3ecd856e29 MultiServer: fix Windows compatibility (#6010) 2026-03-06 01:41:48 +01:00
black-sliver f26313367e MultiServer: graceful shutdown for ctrl+c and sigterm (#5996) 2026-03-04 00:02:12 +01:00
Fabian Dill 922c7fe86a Core: allow async def functions as commands (#5859) 2026-03-01 17:51:56 +01:00
black-sliver fcccbfca65 MultiServer: don't keep multidata alive for race_mode (#5980) 2026-02-26 18:31:39 +00:00
Doug HoskissonandDuck 092a9dc6bd Core: fix bug with missing help text (#5632)
Co-authored-by: Duck <[email protected]>
2026-01-19 20:13:43 +01:00
James White 69e83071ff Multiserver: remove dead code (#5831) 2026-01-11 16:54:12 +01:00
Jarno efd8528db0 MultiServer: Safe DataStorage .pop (#5060)
* Make datastorage .pop not throw on missing key or index

* Reworked to use logic rather than exception catching
2025-12-19 14:57:10 +01:00
Duck d089b00ad5 Core: Add spaces in concatenated strings #5697 2025-11-29 18:52:08 +01:00
NewSoupVi bdae7cd42c MultiServer: Fix hinting multi-copy items bleeding found status (#5547)
* fix hinting multi-copy items bleeding found status

* reword
2025-10-14 20:44:01 +02:00
black-sliver 5390561b58 MultiServer: Fix breaking weakrefs for SetNotify (#5539) 2025-10-12 21:46:16 +02:00
Katelyn Gigante 30ce74d6d5 core: Add host.yaml setting to make !countdown configurable (#5465)
* core:  Add host.yaml setting to make !countdown configurable

* Store /option changes to countdown_mode in save file

* Wording changes in host.yaml

* Use .get

* Fix validation for /option command
2025-10-10 15:02:56 +02:00
black-sliver e907980ff0 MultiServer: slight optimizations (#5527)
* Core: optimize MultiServer.Client

* Core: optimize websocket compression settings
2025-10-08 02:22:34 +02:00
Fabian Dill dc270303a9 Core: improve formatting on /help command (#5381) 2025-09-24 17:33:44 +02:00
Fabian Dill 6c45c8d606 Core: make countdown a "admin" only command (#5463) 2025-09-21 19:23:29 +02:00
NewSoupVi 27e50aa81a MultiServer: Make it so hint_location doesn't set an automatic priority #4713 2025-09-11 00:42:52 +02:00
qwint 8f88152532 MultiServer: Validate CreateHints status arg #5408 2025-09-03 21:01:56 +02:00
Doug Hoskisson c9ebf69e0d Core: MultiData typing (#5071) 2025-07-27 01:27:29 +02:00
a36e6259f1 Core: Add restricted_dumps helper (#5117)
* Add pickling helper that check unpicklability

* Add test condition and generation error handling

* Fix incorrect call and make imports consistent

* Fix newline padding

* Change PicklingError to directly caused by UnpicklingError

Co-authored-by: Doug Hoskisson <[email protected]>

* Revert to `pickle.dumps` for decompressed multidata

* Fix import order

* Restore pickle import in main

* Re-add for multidata in Main

* Remove multisave checks

* Update MultiServer.py

* Update customserver.py

---------

Co-authored-by: Doug Hoskisson <[email protected]>
Co-authored-by: NewSoupVi <[email protected]>
2025-07-26 23:01:40 +02:00
NewSoupVi 4e1eb78163 MultiServer: Fix LocationScouts with "only_new" broadcasting hints for found locations over and over (#4482)
* Hints PR number 42069

* Make it explicit

* clarify

* oops

* Port the change to CreateHints
2025-07-26 19:30:38 +02:00
NewSoupVi 0e4314ad1e MultiServer: CreateHints command (Allows clients to hint own items in other worlds) (#4317)
* CreateHint command

* Docs

* oops

* forgot an arg

* Update MultiServer.py

* Add documentation on what happens when the hint already exists but with a different status (nothing)

* Early exit if no locations provided

* Add a clarifying comment to the code as well

* change wording a bit
2025-07-23 13:04:07 +02:00
Fabian Dill de71677208 Core: only raise min_client_version for new gens (#4896) 2025-05-22 21:30:30 +02:00
qwint 53defd3108 MultiServer: More Guardrails for Nolocation Clients (#4470) 2025-05-10 18:51:44 -04:00
black-sliver 73964b374c MultiServer: import get_settings from the correct module (#4914)
* MultiServer: import get_settings from the correct module

* MultiServer: settings: use attr inbstead of dict access
2025-04-23 15:40:36 +00:00
NewSoupVi b62c1364a9 MultiServer.py: Another Hint Priority + Item Links bug oh boy (#4874)
Basically, hints for itemlink worlds' locations get stored in ctx.hints under
1. the location's player
2. **every individual player** that is participating in the itemlink.

Right now, the updatehint code tries to replace and resend the hint under the itemlinked player, which doesn't work.
2025-04-21 00:43:05 +02:00
Fabian Dill 1b3ee0e94f Core: require clients to support overlapping IDs (#4451) 2025-04-18 20:41:09 +02:00
Ishigh1 4571ed7e2f Core: Made want_reply follow the specs in the docs #4750 2025-04-05 18:35:00 +02:00
NewSoupVi 9a5a02b654 MultiServer Extend datastore "update" operation to work on lists as well, acting as a pseudo "set union". #4666 2025-04-05 18:05:58 +02:00
NewSoupVi b7fc96100c Revert "Core: update websockets (#4732)" (#4753)
This reverts commit 42eaeb92f0.
2025-03-19 01:39:18 +01:00
Fabian Dill 42eaeb92f0 Core: update websockets (#4732) 2025-03-16 22:13:12 +01:00
Benny D 9b3ee018e9 Core/Various Worlds: Fix crash/freeze with unicode characters (#4671)
replace colorama.init with just_fix_windows_console
2025-03-14 08:24:37 +01:00
NewSoupVi d744e086ef MultiServer: Fix hinting an item that someone else already hinted in their slot not resolving correctly (#4655)
* Fix get_hint not checking for finding_player

* Fix using the wrong variable for slot lookup
2025-02-17 15:16:18 +01:00
black-sliver 41898ed640 MultiServer: implement NoText and deprecate uncompressed Websocket connections (#4540)
* MultiServer: add NoText tag and handling

* MultiServer: deprecate and warn for uncompressed connections

* MultiServer: fix missing space in no compression warning
2025-01-29 01:42:46 +01:00
black-sliverandDoug Hoskisson cc770418f2 MultiServer: optimize PrintJSON for !release (#4545)
* MultiServer: optimize PrintJSON for !release

* MultiServer: safer comparison

Co-authored-by: Doug Hoskisson <[email protected]>

---------

Co-authored-by: Doug Hoskisson <[email protected]>
2025-01-24 23:22:33 +01:00
NewSoupVi 005a143e3e MultiServer: Add slot to SetReply packets (#3747)
* Add slot to datastorage set response

* update docs as well
2025-01-18 19:59:26 +01:00
Fabian Dill 9dac7d9cc3 MultiServer: update InvalidPacket text for location scouts (#4485) 2025-01-15 21:50:20 +01:00
NewSoupVi bedf746f1d MultiServer: Revert hints being created for already found locations #4367 2025-01-14 21:37:10 +01:00
NewSoupVi adcb2f59ca MultiServer: Correct tying of Context.groups (#4460) 2025-01-11 22:16:01 +01:00
NewSoupVi 5dd19fccd0 MultiServer/CommonClient: We forgot about Item Links again (Hint Priority) (#4314)
* Vi don't forget about itemlinks challenge difficulty impossible

* People other than Vi also don't forget about ItemLinks challenge difficulty impossible
2024-12-10 20:35:36 +01:00
Emily f26cda07db Core: Hint Priority fixes (#4315)
* Update hint priority docs

* Update network protocol.md

* Add error on `UpdateHint` trying to change to `HINT_FOUND`

* Update network protocol.md

* fix: precollected hint priority
2024-12-01 15:16:36 +01:00
josephwhiteandExempt-Medic 21dbfd2472 Multiserver: Add argument for timestamping STDOUT (#4266)
* core: add server arg for timestamping STDOUT

* Multiserver: Implicitly use default write_mode arg in init_logging

Co-authored-by: Exempt-Medic <[email protected]>

---------

Co-authored-by: Exempt-Medic <[email protected]>
2024-12-01 04:33:36 +01:00
qwint 845a604955 MultiServer: !status shows Ready status (#3598)
* Makes !status show a note if the slot is in Status Ready

* update variable name for better clarity
2024-11-30 03:40:14 +01:00
b783eab1e8 Core: Introduce 'Hint Priority' concept (#3506)
* Introduce 'Hint Priority' concept

* fix error when sorting hints while not connected

* fix 'found' -> 'status' kivy stuff

* remove extraneous warning

this warning fired if you clicked to select or toggle priority of any hint, as you weren't clicking on the header...

* skip scanning individual header widgets when not clicking on the header

* update hints on disconnection

* minor cleanup

* minor fixes/cleanup

* fix: hints not updating properly for receiving player

* update re: review

* 'type() is' -> 'isinstance()'

* cleanup, re: Jouramie's review

* Change 'priority' to 'status', add 'Unspecified' and 'Avoid' statuses, update colors

* cleanup

* move dicts out of functions

* fix: new hints being returned when hint already exists

* fix: show `Found` properly when hinting already-found hints

* import `Hint` and `HintStatus` directly from `NetUtils`

* Default any hinted `Trap` item to be classified as `Avoid` by default

* add some sanity checks

* re: Vi's feedback

* move dict out of function

* Update kvui.py

* remove unneeded dismiss message

* allow lclick to drop hint status dropdown

* underline hint statuses to indicate clickability

* only underline clickable statuses

* Update kvui.py

* Update kvui.py

---------

Co-authored-by: Silvris <[email protected]>
Co-authored-by: NewSoupVi <[email protected]>
2024-11-29 02:10:31 +01:00
qwint 48ea274655 MultiServer: persist hints even if previously found (#4214)
* change to persist all hints to ctx.hints regardless of found status

* remove if not found entirely as it seems like it was added to not double charge hint points
https://github.com/ArchipelagoMW/Archipelago/commit/9842399d8be082b757a140137706c12d84cda8c1
2024-11-19 21:16:10 +01:00
black-sliver 9be996ba0e Core: downgrade websockets and minor fixes (#4166)
* Core: downgrade websockets to 13.x

14.x currently doesn't work for MultiServer.
14.x is not supported with py3.8, so updating to 14.x should be scheduled for AP 0.6.0.

* CI: 5min timeout for hosting test

* MultiServer: properly shutdown even if ctx is invalid

* CI: increase hosting test timeout to 10min

this is 4x expected time, just to be safe.
2024-11-10 01:23:29 +01:00