Create a new class that handles conversion worlds+embedded -> context data.
Create a derived class that uses static_server_data+pony instead.
There is also a not very efficient feature to deduplicate strings (may need perf testing).
By moving code around, we can simplify a lot of the world loading.
Where code lines were touched, some typing and some reformatting was added.
The back compat for GetDataPackage without games was finally dropped.
This was done as a cleanup because the refactoring touched those lines anyway.
Also reworked the per-context dicts and the RoomInfo to hopefully be more efficient
by ignoring unused games. (Generating the list of used games was required for the new
code anyway.)
Side effect of the MultiServer cache: we now load worlds lazily (but still all at once)
and don't modify the games package in place. If needed we create copies.
This almost gets us to the point where MultiServer doesn't need worlds - it still needs
them for the forbidden items.
There is a bonus optimization that deduplicates strings in name_groups that may have bad
performance and may need some perf testing if we run into issues.
* Test: add hosting simulation test
* WebHost: add weak typing to get_app()
* MultiServer: add typing to auto_saver_thread
* MultiServer: don't cancel task, properly end it
* customserver: stop auto-save thread from saving after shutdown
and make sure it stops, another potential memory leak
* MultiServer, customserver: make datapackage small again
* customserver: collect/finish room tasks
Hopefully fixes the memory leak we are seeing
* CI: test hosting
* Test: hosting: verify autohoster saves on Ctrl+C
* customserver: save when stopping via Ctrl+C
when switching to multiple rooms per process, we ended up modifying the static server data
because that's how _load works and the data is now shared between multiple rooms.
Currently when a room shuts down while clients are connected it instantly spins back up. This fixes that behaviour categorically.
I still don't know why or when this problem started, but it's certainly wreaking havok on prod.
* Core: lazy-load worlds in unpickler
this should hopefully fix customserver's memory consumption
* WebHost: move imports around to save memory in MP
* MultiServer: prefer loading _speedups without pyximport
This saves ~15MB per MP and speeds up module import if it was built in-place.
* Tests: fix tests for changed WebHost imports
* CustomServer: run GC after setup
* CustomServer: cleanup exception handling
* WebHost: Make custom server prefer ipv4 for display
* WebHost: Make server retry saving in case of connection issues
* WebHost: fix autolaunch guardians getting stuck waiting for the oldest two rooms.
Probably not related to the issues of the system itself getting stuck, but should be fixed anyway.
* WebHost: logfile is meant to be guarded by access cookie
* WebHost: set patch target to null if port is not valid, disabling auto-connect
black-sliver