Config
Log for #openttd on 12th June 2021:
Times are UTC Toggle Colours
00:15:44  *** Progman has quit IRC
00:28:33  *** Gustavo6046 has joined #openttd
00:42:30  *** Gustavo6046 has quit IRC
01:00:37  *** Gustavo6046 has joined #openttd
01:34:11  *** Wormnest has quit IRC
02:54:08  *** D-HUND has joined #openttd
02:57:35  *** debdog has quit IRC
03:09:43  *** glx has quit IRC
04:31:22  *** WormnestAndroid has quit IRC
04:33:17  *** WormnestAndroid has joined #openttd
05:05:38  *** Flygon has joined #openttd
05:31:44  *** Flygon has quit IRC
06:21:27  *** Flygon has joined #openttd
06:30:33  *** WormnestAndroid has quit IRC
06:33:07  *** WormnestAndroid has joined #openttd
06:39:51  *** sla_ro|master has joined #openttd
06:40:08  *** paulus[m] has quit IRC
06:41:11  *** WormnestAndroid has quit IRC
06:41:54  *** WormnestAndroid has joined #openttd
06:47:01  *** tokai has joined #openttd
06:47:01  *** ChanServ sets mode: +v tokai
06:54:04  *** tokai|noir has quit IRC
06:57:39  *** andythenorth has joined #openttd
07:06:02  <DorpsGek> [OpenTTD/OpenTTD] PeterN merged pull request #9344: Change: Reduce real sprite groups if possible. https://git.io/JZn41
07:06:14  *** nielsm has joined #openttd
07:07:00  <DorpsGek> [OpenTTD/OpenTTD] PeterN updated pull request #9289: Change: Shortcut varaction chains for callbacks. https://git.io/JsHuK
07:45:39  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #9352: Codechange: use the fmt library for simpler debug formats https://git.io/JZKfv
07:54:38  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #9352: Codechange: use the fmt library for simpler debug formats https://git.io/JZKfv
08:03:50  *** outtatime is now known as whatsthetime
08:06:31  *** jottyfan has joined #openttd
08:15:25  <andythenorth> today I will mostly be watching slow CI jobs :P
08:15:32  * andythenorth works the weekend
08:15:35  <andythenorth> 22 mins for each run
08:15:46  <andythenorth> atomic commits
08:15:55  <TrueBrain> sounds like fun!
08:16:00  <TrueBrain> wait, no
08:16:02  <TrueBrain> that is the wrong word
08:16:03  <TrueBrain> eeuuhhh
08:16:05  <TrueBrain> :P
08:20:40  *** HerzogDeXtEr has joined #openttd
08:42:52  <TrueBrain> Broken savegame: list exceeds storage size
08:42:53  <TrueBrain> oops :D
08:44:34  <andythenorth> 'tried to read beyond bounds of image' is my frequent equivalent :P
08:57:03  *** Wolf01 has joined #openttd
08:57:10  *** Progman has joined #openttd
09:01:13  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain updated pull request #9322: Add: store table header for each chunk in savegame https://git.io/JGlmd
09:01:14  <TrueBrain> now companies are exportable as JSON too :D
09:01:58  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #9352: Codechange: use the fmt library for simpler debug formats https://git.io/JZKfv
09:15:25  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain updated pull request #9322: Add: store table header for each chunk in savegame https://git.io/JGlmd
09:16:48  <nielsm> https://www.tt-forums.net/viewtopic.php?f=32&t=89033   some of this is actually not a bad idea... put down buoys and then when you make a ship go to a dock or depot in the orders, do full pathfinding from previous to selected destination but try to incorporate buoys... it would probably be heavy to do, but it can be done clientside and just send a series of "add order" commands
09:18:00  <TrueBrain> "but you ruin the game, you make it too easy!!!!" :P
09:18:03  <TrueBrain> sorry, someone had to say it
09:19:05  <TrueBrain> but yeah, we have no lack of "good ideas" for this game :)
09:19:27  <Rubidium> just a lack of good developers doing the work
09:22:57  <_dp_> how about just fixing ship pf so it doesn't need bouys at all? :p
09:23:13  <nielsm> nah that would be too sensible
09:36:16  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #9352: Codechange: use the fmt library for simpler debug formats https://git.io/JZKfv
09:39:21  *** Samu has joined #openttd
09:45:15  *** Progman has quit IRC
09:52:33  *** tokai|noir has joined #openttd
09:52:33  *** ChanServ sets mode: +v tokai|noir
09:58:42  <Samu> are subsidies disabled when cargodist is in place?
09:59:33  *** tokai has quit IRC
09:59:49  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #9352: Codechange: use the fmt library for simpler debug formats https://git.io/JZKfv
09:59:49  <Samu> damn, that was it!
10:00:13  <Samu> there is no mention of it anywhere :(
10:02:41  <Rubidium> there definitely is in the logs of this channel
10:05:17  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #9352: Codechange: use the fmt library for simpler debug formats https://git.io/JZKfv
10:14:23  *** outtatime has joined #openttd
10:14:27  *** whatsthetime has quit IRC
10:19:14  *** frosch123 has joined #openttd
10:23:21  * andythenorth oof 100% diff to read
10:23:27  <andythenorth> that's almost same as 0%?
10:25:01  <DorpsGek> [OpenTTD/OpenTTD] frosch123 commented on pull request #9289: Change: Shortcut varaction chains for callbacks. https://git.io/JZMsG
10:27:46  <Samu> just got a 1.11.2 crash
10:29:53  <DorpsGek> [OpenTTD/team] frosch123 commented on issue #228: [pt_BR] Translator access request https://git.io/JZVdR
10:32:51  <DorpsGek> [OpenTTD/OpenTTD] SamuXarick opened issue #9353: Crash Report https://git.io/JZMcn
10:33:52  *** virtualrandomnumber has joined #openttd
10:34:23  *** virtualrandomnumber has quit IRC
10:43:43  <Wolf01> I got lured again into a treasure hunt in google earth
10:59:42  <DorpsGek> [OpenTTD/OpenTTD] SamuXarick commented on issue #9353: Crash Report https://git.io/JZMcn
11:02:27  <Samu> i think garbage collector was running
11:02:50  <Samu> but i dunno what could go wrong
11:07:10  *** andythenorth has quit IRC
11:15:58  *** WormnestAndroid has quit IRC
11:17:23  *** WormnestAndroid has joined #openttd
11:43:25  <TrueBrain> 	_animated_tiles.clear();
11:43:25  <TrueBrain> 	_animated_tiles.resize(_animated_tiles.size() + count);
11:43:29  <TrueBrain> how is that size() useful? :P
11:45:23  <michi_cc> That's discrimination against known constants :P
11:46:54  *** HerzogDeXtEr has quit IRC
11:50:06  <TrueBrain> :D
11:56:50  <frosch123> maybe add an assert(_animated_tiles.size() == 0), just to be on the safe side
11:57:10  <TrueBrain> dunno, means saving can maybe fail :P
11:57:12  <frosch123> hmm, do we have gsl now? then ensures(...) ofc
11:57:19  <TrueBrain> we do not
11:57:38  *** andythenorth has joined #openttd
11:59:22  *** tokai has joined #openttd
11:59:22  *** ChanServ sets mode: +v tokai
12:06:23  *** tokai|noir has quit IRC
12:19:49  <TrueBrain> SLE_ARR(LoggedChange, revision.text,     SLE_UINT8,  GAMELOG_REVISION_LENGTH),
12:19:53  <TrueBrain> I wonder why that is not a SLE_STR ..
12:22:26  <frosch123> there was a similar thing in the ai saveload stuff
12:22:36  <frosch123> i wondered whether SLE_STR is somehow newer
12:22:48  <TrueBrain> another GameLog thing does use SLE_STR
12:22:57  <frosch123> ok :p
12:23:04  <TrueBrain> but the file got moved
12:23:08  <TrueBrain> so hard to see if they are introduced at the same moment
12:34:28  <TrueBrain> debugging SaveLoad stuff remains really really difficult
12:43:17  <TrueBrain> SLE_CONDNULL(4, SLV_6, SL_MAX_VERSION)
12:43:21  <TrueBrain> I forgot we still had these gems
12:48:29  <frosch123> is that for inserting stuff in stable releases without savegame bump?
12:48:42  <TrueBrain> we used to have those in all structs, so indeed we could add fields freely
12:48:44  <TrueBrain> most have been removed
12:48:47  <TrueBrain> but a few have remained
12:57:42  *** glx has joined #openttd
12:57:42  *** ChanServ sets mode: +v glx
13:04:39  <TrueBrain> @base 10 16 26
13:04:39  <DorpsGek> TrueBrain: 1A
13:04:47  <TrueBrain> shit, I should hav elooked at my DHCP settings, sooorrryyyy
13:04:49  <TrueBrain> not a real nerd :(
13:05:01  <frosch123> just wanted to write that :)
13:05:10  <TrueBrain> :D
13:05:26  <glx> oh you were in gamelog area
13:05:28  <TrueBrain> it still cracks me up
13:06:18  <glx> it's so C stuff
13:06:29  <TrueBrain> its really weird
13:06:35  <TrueBrain> but I am going to make it a bit more pretty :)
13:07:46  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain updated pull request #9322: Add: store table header for each chunk in savegame https://git.io/JGlmd
13:08:05  <TrueBrain> right, fixed that all lists now have a gamma before the list
13:08:13  <TrueBrain> 10000x easier to read now for external tools :P
13:13:47  <glx> Samu: your crash is weird as ScriptObject::GetActiveInstance() is never supposed to return nullptr, there's even an assert ensuring that in non stable release builds
13:14:48  <Samu> i think it's related to garbage collector being run on non active instances
13:15:29  <Samu> trying to figure out where instances get active and deactivated
13:15:30  <_dp_> TrueBrain, does it allow unknown chunk types btw?
13:15:44  <glx> but the instance is active when it's started, it's if (Company::IsValidAiID(cid)) Company::Get(cid)->ai_instance->CollectGarbage();
13:17:16  <TrueBrain> _dp_: let me find my mind-reading device again ... nope, still broken, sorry!
13:17:48  <Samu> in debug mode, the assert happens somewhere else
13:17:57  <Xaroth> I think you left your crystal ball at the bar, TrueBrain.
13:18:27  <Samu> line 53 of script_object.cpp assert(ScriptObject::ActiveInstance::active != nullptr);
13:18:31  <TrueBrain> error: cannot convert ‘std::_Bit_reference*’ to ‘bool*’ <- lol @ error :)
13:18:58  <_dp_> TrueBrain, you ignore unknown fields in chunks but what happens if chunk type itself is not known?
13:19:00  <glx> Samu: yes, because asserts are disabled for non beta releases
13:19:40  <TrueBrain> _dp_: invalid chunk types is in any sense unrecoverable
13:20:19  <Samu> I reproduced the issue in current master
13:20:23  <Samu> still happens
13:20:32  <_dp_> there is nothing to recover, it just shouldn't be the fatal error by the same logic
13:20:43  <TrueBrain> that makes 0 sense
13:20:54  <TrueBrain> here you have a blob of data you know NOTHING about, not the length, not the content, nothing
13:21:02  <_dp_> yes, skip it
13:21:03  <TrueBrain> but ... "read" it anyway?
13:21:12  <TrueBrain> if you don't know the type, you don't know the length
13:21:16  <_dp_> and you know length
13:21:17  <TrueBrain> so .. magic?
13:21:36  <glx> Samu: is it easy to reproduce ?
13:22:19  <Samu> hmm, yes, and no. Seems to happen during garbage collection
13:22:32  <Samu> when the AI finishes a pathfind run
13:22:38  <Samu> which is not that often
13:23:16  <Samu> basically, just run several Ludiai afterfixes v18
13:23:25  <Samu> and wait for it to trigger
13:24:21  <_dp_> TrueBrain, basically if patches add fields to existing objects it's mostly fine and will be ignored
13:24:35  <_dp_> but if there is no suitable object to put stuff in it can't just add another chunk
13:25:29  <Samu> i've been lucky maybe, with 3 Ai's in the first 2 years, i got a crash
13:25:46  <Samu> could take longer though
13:26:13  <TrueBrain> _dp_: you seem to heavily confuse chunk-type with chunk-ids
13:26:23  <TrueBrain> but no, OpenTTD is backwards compatible, not forwards compatible
13:26:27  <Rubidium> Samu: do you have more of the stack trace somewhere?
13:26:29  <TrueBrain> my changes will not change that
13:27:17  <_dp_> TrueBrain, neither did they change backward-compatibility :p
13:28:13  <_dp_> at least, hopefully xD
13:28:29  <Samu> nop, i posted a crash though
13:29:27  <Samu> im trying to have it crash again, will make a better trace
13:32:03  *** jottyfan has quit IRC
13:32:10  *** Progman has joined #openttd
13:34:26  <_dp_> TrueBrain, and pr even has no motivation besides the settings chunk so it's unclear why you did it at all when it hardly changes anything for the game itself :p
13:34:37  <glx> my best guess is some ScriptObject deletion happening earlier as that will modify ScriptObject::active
13:34:51  <TrueBrain> @kick _dp_ stop the endless gatekeeping
13:34:51  *** _dp_ was kicked by DorpsGek (stop the endless gatekeeping)
13:34:51  *** _dp_ has joined #openttd
13:35:16  <_dp_> and for settings same could be achieved with other ways like adding explicit id
13:35:31  <_dp_> wtf? you're pr is lacking motivation and I'm gatekeeping?
13:35:45  <TrueBrain> you can ASK for things, but you are not ASKING
13:36:01  <_dp_> it's a good to make format better but it's mostly useful for external tools not the game itself
13:36:03  <TrueBrain> I am done with having to "defend" my stuff
13:36:10  <TrueBrain> ask questions if you are curious
13:36:18  <TrueBrain> stop assuming random stuff and making it a truth and forcing me to defend it
13:36:29  <TrueBrain> it is stupid, unproductive and not constructive in any way
13:37:13  <_dp_> TrueBrain, and I'm ASKING, what's the motivation of your pr
13:37:30  <TrueBrain> you were not ASKING anything. You stated a lot of things
13:37:32  <_dp_> because to me it goes way beyond what it declares
13:38:00  <_dp_> but when I ask about possible uses you ignore them
13:38:50  <TrueBrain> just read back how you talk lately .. it rarely is in any form of a question
13:38:54  <TrueBrain> you judge, claim things
13:38:56  <TrueBrain> and demand an answer
13:38:58  <TrueBrain> it is not fun
13:39:00  <Rubidium> to me it reads like: "As I was messing about with the settings, to make things easier I wanted to make the settings chunk self-declarative. But then figured out making the whole savegame self-declarative would be better in the long run"
13:39:01  <TrueBrain> not in the slightest
13:39:24  *** norri has joined #openttd
13:39:30  <_dp_> TrueBrain, ffs, you even build a full savegame explorer using pr that only pretends to ease up managing settings
13:41:10  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain updated pull request #9322: Add: store table header for each chunk in savegame https://git.io/JGlmd
13:41:13  <TrueBrain> right, animated-tiles done ... now for gamelog blob ... oof ..
13:41:59  <Rubidium> _dp_: you're wrong... he's doing it to add STUN and TURN... ;)
13:42:11  <_dp_> oh xD
13:43:18  *** jottyfan has joined #openttd
13:43:26  <TrueBrain> do we have any tooling that reads the gamelog currently?
13:44:33  <TrueBrain> I guess crash.log is our main way of reading it :D
13:44:37  <Rubidium> maybe some crashlog decoder thing?
13:45:37  <glx> there are console commands IIRC
13:48:02  <_dp_> TrueBrain, btw, speaking of fun, I have a lot of code that works with current savegame format. and while new format in theory can make managing it easier it can also just break things completely if those possibilities are not realised
13:48:18  <_dp_> and it's not fun when someone breaks a lot of your stuff for no good reason
13:52:28  *** Tirili has joined #openttd
13:54:16  <nielsm> regardless of making a new, easier to parse format, the parser for the old format needs to remain too
13:55:32  <_dp_> nielsm, depends on what you're talking about. game needs compatibility for sure, but, for example savegame diff tool for desync debugging can go with complete switch
13:57:47  <Rubidium> is that diff "intelligent" in that it can say which vehicle ID has the difference, or is it for the human to figure that out based on a hex dump?
13:59:03  <Rubidium> as with the self-descriptive chunk definition the information what the bits mean would get added to the savegame and implementing it once in the diff tool means that (barring and significant format changes), you do not need to update the diff tool when a field gets added, changed or removed
13:59:30  <Rubidium> you might even be able to start diffing between different savegame versions, if you ignore the fields that are not in common between both
14:00:38  <_dp_> well, I was talking some tool in general, but mine for example does stuff like this: https://citymania.org/static/files/misc/desync20210117.txt
14:02:20  <_dp_> and I fully agree that self-descriptive format can make this tool much simpler
14:02:32  <_dp_> but it's hard to tell for sure with intentions of format change being unclear
14:02:46  <_dp_> and my tool already works fine with existing format
14:09:54  <Samu> im not being lucky this time, still waiting for a crash
14:11:07  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #9352: Codechange: use the fmt library for simpler debug formats https://git.io/JZKfv
14:11:57  <glx> I can see how it could crash, but not exactly why
14:12:03  <glx> and I can't reproduce
14:12:33  <Samu> try a small map
14:12:38  <Samu> 256x256
14:12:48  <glx> that's the size I'm using
14:13:03  <glx> but it's running at less than 1.0x
14:13:11  <peter1138> That's a normal size map, not small :p
14:13:13  <Samu> running 3 AIs, disable aircraft
14:13:56  <Samu> max loan is 300k, not sure if it matters
14:14:25  <Samu> im unsure how garbage collector works
14:14:49  <Samu> sometimes I'm out of money, but the AI doesn't cancel the pathfinding, so the priority queue remains alive
14:15:02  <Samu> it waits for money before resuming pathfind
14:15:20  <TrueBrain> lol @ peter1138 :)
14:21:49  <Samu> glx, try the crash.sav and type restart, it just crashed for me 2 years in
14:22:53  <DorpsGek> [OpenTTD/OpenTTD] SamuXarick commented on issue #9353: Crash Report https://git.io/JZMcn
14:26:57  <glx> the trace doesn't show the cause, only the consequence
14:27:13  <Samu> i don't have anything else
14:27:27  <glx> I know
14:30:00  <glx> and with FFWD having no effect it's hard to debug
14:30:22  <TrueBrain> game not fast enough? :)
14:30:52  <glx> running at less than 1x
14:31:02  <TrueBrain> off
14:31:03  <TrueBrain> oof
14:31:06  <TrueBrain> typing, hard
14:32:37  <Samu> my ai randomizes some settings on load, makes it even harder to reproduce
14:32:56  <TrueBrain> frosch123: ah, indeed, SLE_STR reads/writes a "char *" .. and I do not think we have anything else to store char[] with :)
14:33:08  <TrueBrain> so it does make sense .. somewhat :P
14:33:09  <Samu> a long standing bug that you didn't fix :(
14:33:55  <Samu> got a crash again! 24th nov 1952
14:35:44  <Rubidium> yeah, the ActiveInstance does not get set when collecting the garbage there
14:36:05  <glx> yes, but sometimes it seems to work
14:36:31  <frosch123> TrueBrain: didn't rb add std::string for everything?
14:36:35  <glx> probably using the worng instance then
14:36:37  <TrueBrain> frosch123: support, yes
14:36:38  <Rubidium> so probably the priority queue was used somewhere in a set of structures that has a loop, so it would not be freed "normally". Only the garbage collector could solve that
14:36:55  <TrueBrain> we just have places to change :)
14:37:35  <Rubidium> so the normal "goes out of scope" collection would usually take care of releasing/freeing the priority queue. Only in this situation / for this AI the location where it is stored differs and boom
14:37:46  <DorpsGek> [OpenTTD/OpenTTD] Turtle-PL started discussion #9354: /Discord screenshot channel (not competition)/ https://git.io/JZyez
14:38:40  <peter1138> Discord bugs now, what.
14:39:45  <Rubidium> the biggest problem with solving the "active instance" issue is figuring out where to change the const(s)
14:40:03  <_dp_> you made it "official", deal with it! :p
14:40:25  <_dp_> also looks like someone missed #trainboard
14:43:00  <Rubidium> Samu/glx: https://gist.github.com/rubidium42/4956ee7a09dcc06a2dce77a42bec6405 <- a potential solution for that issue
14:44:24  <Samu>  will try, but... i still don't have a sure method to replicate
14:44:26  <glx> makes sense yes
14:46:09  <glx> the issue is it's hard to reproduce the crash, so testing a fix won't be easy
14:51:24  <TrueBrain> meh ... back to battles SaveLoad errors :P
14:52:01  *** Wormnest has joined #openttd
14:52:02  <Rubidium> the trick is to make something with an internal circular reference that also references the priority queue, and then releasing the "stack" reference to that. So it could probably be implemented as a regression quite easily... if only one could trigger the garbage collection at the right time (tm)
14:52:25  <glx> at least with this change CollectGarbage() is similar to other members calling this->engine functions
14:54:13  <glx> and seems more sane
14:54:18  <glx> (and safe)
14:55:02  <DorpsGek> [OpenTTD/OpenTTD] glx22 commented on discussion #9354: Discord request: screenshot channel (not competition) https://git.io/JZyez
14:55:06  <Samu> no crash so far, 1954
14:55:16  <Samu> will restart again
14:56:58  <TrueBrain> 			SlObject(la, _glog_action_desc); // has to be saved after 'DATE'!
14:57:02  <TrueBrain> during loading ..
14:57:06  <TrueBrain> some comments are weird
14:57:10  <glx> AI caused crashes are the worst to trace, because it's often random
14:57:37  <glx> TrueBrain: it's clearly not saved after DATE as GLOG is the first chunk ;)
14:57:56  <TrueBrain> exactly :)
14:58:22  <glx> I think originally it was, then we moved the chunk
14:59:29  <TrueBrain> I am staring at this gamelog chunk code .. maybe I should just add debug lines instead ..
14:59:55  <glx> all the ReallocT
15:00:09  <TrueBrain> it reads too few or too many bytes in my new code
15:00:10  <TrueBrain> so annoying
15:00:14  <TrueBrain> as you have 0 info to go on
15:00:14  <glx> when a vector should be easier to understand
15:00:52  *** tokai|noir has joined #openttd
15:00:52  *** ChanServ sets mode: +v tokai|noir
15:01:25  <glx> but when you save a game you can load it back ?
15:01:37  <TrueBrain> no, that part fails :P
15:02:00  <glx> ha so it fails for all saves, even the old ones
15:02:06  <TrueBrain> nope
15:02:15  <TrueBrain> only my new save code fails
15:02:38  <glx> oh weird, usually it's the opposite
15:03:03  <TrueBrain> I have a pretty good handle on the old stuff now :P
15:03:18  <glx> code is visible somewhere, or not pushed yet ?
15:04:28  <TrueBrain> not till I fixed it :)
15:05:08  <TrueBrain> haha, it reads an optional struct as "103" (instead of 0/1)
15:05:10  <TrueBrain> that .. is not right :D
15:06:32  <glx> so much magic in length calculation (well not magic, but still)
15:06:47  <TrueBrain> yeah ... and after your and mine change, we can remove a lot of that weirdness
15:06:56  <TrueBrain> as most of it is completely unneeded complex
15:07:01  <TrueBrain> to support weird scenarios I now removed :P
15:07:44  *** tokai has quit IRC
15:08:34  <glx> like +=4 in the loop (for action type, desc, and final none change), then a +1 for final none action
15:10:34  <TrueBrain> okay, that was just stupid what I did .. lol
15:10:56  <glx> inserted your stuff in the right place ?
15:11:12  <TrueBrain> very hard to explain whatI fucked up, but it was a brainfart :P
15:12:01  <glx> GLOG saveload code looks complex, but the format itself is quite simple
15:13:36  <glx> and the complexity is mostly because Gamelog structs are weird :)
15:13:59  <TrueBrain> okay, now I have to add headers ... then I can normally view GLOG too .. that should be simple
15:15:35  <glx> because it's C stuff it uses unions and dynamic C array when vector and subclasses would be simpler I think (but refactoring is not easy)
15:15:47  <Rubidium> GLOG should be similar in functionality/layout as stations/vehicles, but due to the union it's not. Though practically each of the logged change types could be its own class. Are you going into that direction? Though I guess... maybe not in scope?
15:15:51  <TrueBrain> refactoring is mostly just time
15:16:08  <TrueBrain> I changed the saveload code to be like vehicles
15:16:24  <TrueBrain> changing the rest in a similar way I leave as exercise to the reader :P
15:22:28  *** Tirili has quit IRC
15:26:03  <Samu> not getting any crash so far, i restarted 3 times already
15:30:49  <TrueBrain> haha, SL_STR are now a list of bytes in my reader :D
15:30:51  <TrueBrain> that looks funny :D
15:32:29  <TrueBrain> euh, no, it was an SL_ARR from SLE_UINT8s
15:32:31  <TrueBrain> so it is not wrong
15:32:34  <TrueBrain> just looks really weird :P
15:32:45  <TrueBrain> I think it is the only field doing this
15:33:17  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain updated pull request #9322: Add: store table header for each chunk in savegame https://git.io/JGlmd
15:33:17  <TrueBrain> okay, gamelog done ..
15:34:07  <TrueBrain> that leaves ... gamescript translations and linkgraph
15:34:14  <TrueBrain> well, and map-chunks, ofc
15:34:41  <TrueBrain> like really close now .. and then some splitting up over more than 2 PRs :D
15:42:23  <TrueBrain> "Broken savegame - Invalid gamelog action type" <- at least 1 bug .. it is funny that 100+ savegames loaded fine
15:42:23  <TrueBrain> but 1 failed
15:44:45  <frosch123> is there a bounty for "find a savegame that loads in master, but fails in tb's branch"?
15:46:03  <TrueBrain> yes
15:46:10  <TrueBrain> I found one
15:46:13  <TrueBrain> can I collect the bounty now?
15:46:17  *** AmaNet has joined #openttd
15:46:25  <AmaNet> So this is IRC
15:47:11  <frosch123> yes, 5 people talking, 153 people idling
15:47:41  <AmaNet> Cool
15:47:52  <AmaNet> Never used something like this before
15:47:57  <glx> frosch123: in which group are you placing DorpsGek ?
15:48:12  <frosch123> dorpsgek is best boy
15:48:31  <TrueBrain> so sad there is no more @say :P
15:49:06  <AmaNet> what is +glx
15:52:06  *** AmaNet has quit IRC
15:57:31  <TrueBrain> funny, only an emergency save fails loading :D
15:57:35  <TrueBrain> well, who needs to load those anyway?
15:59:08  *** iSoSyS has joined #openttd
16:01:06  <_dp_> indeed, who needs to debug desyncs? >:-)
16:01:08  <Samu> hmm, 24th Nov 1952 this is semi-reproducible
16:01:15  <Samu> got a crash on this day again
16:04:33  <Samu> 17th Apr 1952, nop, not reproducible
16:05:47  <Samu> with rubidium fix, it has yet to crash
16:06:04  <Samu> im now testing without it, trying to make it reproduceable
16:08:48  <norri> is there something else that should be submitted together with NewGRF feature PR? I've got a NML patch ready, but I'm thinking that's better to do only after the PR is merged?
16:09:30  <TrueBrain> I see no reason not to create it already; just mark it that it depends on merging of another PR :)
16:12:43  <glx> it's even better that way, as it allows testing :)
16:12:55  <TrueBrain> and shows you mean business :D
16:13:01  <TrueBrain> not just a drive-by-commit :P
16:13:11  <norri> alright, brb then!
16:13:50  <TrueBrain> right, lets see if I fixed the emergency savegame load issue ... pam pam pammmmmm
16:13:59  <TrueBrain> seems so :) Now to test the other 500 savegames :P
16:14:13  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain updated pull request #9322: Add: store table header for each chunk in savegame https://git.io/JGlmd
16:17:06  *** sla_ro|master has quit IRC
16:17:08  *** iSoSyS has quit IRC
16:20:20  <TrueBrain> hmm .. how to use jq with a dot in the name ..
16:21:32  <DorpsGek> [OpenTTD/nml] vituscze opened pull request #222: Feature: Support for rail vehicle property 0x2E, curve_speed_mod https://git.io/JZySd
16:22:19  <glx> \. ?
16:23:09  <TrueBrain> https://gist.github.com/TrueBrain/bbf0cca594580323e16a651968fc0af8 <- quickly find what versions a savegame is saved in over the years :P
16:24:42  <TrueBrain> I do have to make the Python code faster :P
16:24:47  <TrueBrain> some savegames take for-ever
16:26:24  <frosch123> how big are the json files? writing 200mib of json is not fast
16:26:57  <TrueBrain> well, without JSON export it is very slow too :P
16:27:10  <TrueBrain> and only 30MB of JSON, that is not too bad
16:27:42  <TrueBrain> can't measure the time it takes to JSON dump that
16:27:42  <frosch123> really? we had savegames with 10mb compressed. so 30mb sounds very little
16:27:46  <TrueBrain> it is below the noise :P
16:27:59  <TrueBrain> Sorry, I didn't test all savegames in existence :P
16:28:02  <TrueBrain> lol
16:28:22  <TrueBrain> and do remember the map-chunks are not in the JSON (yet? Dunno)
16:28:46  <TrueBrain> this savegame is 1MB on disk btw
16:29:04  <_dp_> a lot of savegame size is map array, if json skips that it mostly depends on amount of stuff in the game
16:29:20  <TrueBrain> frosch123: but the slowest part seems to be the binreader I "borrowed" from you
16:29:26  <TrueBrain> it is recreated every entry
16:29:31  <TrueBrain> and Python doesn't seem to fancy that too much
16:29:54  <TrueBrain> I didn't really try to optimize it either :D
16:30:13  <TrueBrain> pretty sure if I use a bytesview it becomes significantly faster :)
16:30:25  <TrueBrain> s/bytesview/memoryview/, whatever :P
16:33:07  <TrueBrain> not completely sure why you were thinking I was testing with a 10MB savegame frosch123 :)
16:33:38  <frosch123> hmm, never heard of memoryview, but apparently it's ancient
16:34:03  <frosch123> TrueBrain: i forgot that you used coop savegames, they mostly use sane map sizes
16:34:13  <Samu> queue.stack got 20317 items in it :p
16:34:22  <TrueBrain> in general, testing with extremes isn't fun :P
16:34:50  <TrueBrain> but biggest savegame I have is 3MB
16:35:00  <TrueBrain> called "biggest town ever"
16:35:09  <glx> high number of trains ?
16:35:18  <TrueBrain> I never loaded it in graphically :P
16:35:39  <glx> you can count vehicules with jq ;)
16:35:51  <TrueBrain> but I would guess it is a savegame with a really big town
16:36:30  <TrueBrain> 52M
16:36:31  <TrueBrain> lol
16:36:34  <glx> town size itself doesn't impact savegame as it's all in the map
16:36:35  <frosch123> towns are only maparray though
16:37:03  <TrueBrain> lets see how much the JSON is :P
16:37:26  <TrueBrain> just 175KB
16:37:27  <TrueBrain> owh yeah
16:38:01  <TrueBrain> I wonder if we should export map-chunks as JSON
16:38:08  <TrueBrain> not sure it is useful
16:38:26  <frosch123> for the savegame-diff you need everything
16:38:34  <frosch123> but for maparray i can only think of hexdump :p
16:38:36  <TrueBrain> can do a CRC only, or what-ever
16:39:04  <TrueBrain> well, in an ideal world, you can see what is on a certain tile
16:39:12  <TrueBrain> but .. we don't store things like that in the savegame :D
16:39:13  <glx> decoded map could be nice but it's not that easy
16:39:18  <frosch123> does python impose any sizelimit on strings in json?
16:39:24  <TrueBrain> not that I know
16:39:26  <dwfreed> no
16:39:26  <TrueBrain> and I did some crazy shit
16:39:34  <TrueBrain> there has to be some limit somewhere
16:39:40  <TrueBrain> but .. yeah .. memory runs out before that time :P
16:39:42  <dwfreed> the only limit is your RAM
16:39:43  <frosch123> a single string with 32MB :p
16:39:59  <TrueBrain> 32MB?
16:40:23  <dwfreed> technically you probably couldn't have a string larger than 2^64-1, but good luck reaching that
16:40:40  <TrueBrain> dwfreed: lot of virtual ram :P
16:40:42  <TrueBrain> SSDs
16:40:43  <TrueBrain> lots of them
16:41:10  <dwfreed> Linux isn't even able to address that much
16:41:24  <dwfreed> your page tables probably wouldn't even fit in RAM
16:41:38  <TrueBrain> anyway, ideally I would like to add a header to the map-chunks too, but I just don't see a sensible way of doing that atm
16:41:41  <TrueBrain> ideas are welcome
16:41:47  *** andythenorth has quit IRC
16:42:01  <TrueBrain> otherwise I can just make my reader combine the map-chunks and output per tile the total value in hex or what-ever
16:44:08  <glx> maybe binary output matching landscape_grid.html order
16:44:29  <TrueBrain> that would be a REALLY large JSON output :P
16:44:50  <glx> so it can give a partial detail of the content
16:45:11  <TrueBrain> we also have to balance practicality :)
16:45:35  <TrueBrain> @calc 256 * 256 * (64 + 3)
16:45:35  <DorpsGek> TrueBrain: 4390912
16:45:48  <glx> well that's a display job, not a json job I guess
16:45:54  <TrueBrain> for normal maps it is fine :P
16:45:59  <TrueBrain> if the JSON is 10GB
16:46:05  <TrueBrain> ... ;)
16:46:13  <TrueBrain> @calc 2048 * 2048 * (64 + 3)
16:46:13  <DorpsGek> TrueBrain: 281018368
16:46:28  <TrueBrain> 281MB for 2kx2k if we do the bare minimum JSON
16:46:39  <TrueBrain> so I do not think binary representation is a good idea :)
16:47:05  <_dp_> there's already quite good map array viewer
16:47:09  <_dp_> ? tool in the game :P
16:47:25  <TrueBrain> glx: I think it is a bit the other way around .. JSON needs to output it, a display should show it in a useful format :)
16:47:49  <TrueBrain> owh, we are 96 bits long
16:47:50  <glx> yeah json can output in hek
16:47:50  <TrueBrain> oof
16:47:57  <glx> *hex
16:48:09  <TrueBrain> 24 nibbles long that would be
16:48:18  <TrueBrain> needs 0x and "", as otherwise JSON will do it as integer
16:49:11  <TrueBrain> but anyway, I was more thinking if we should add a header to the map-chunks :)
16:49:26  <TrueBrain> needs any of the newmap branches merged I guess :P
16:49:45  <glx> in current map array, header is useless
16:50:27  <TrueBrain> for diffing, ideal it is {"1": <tile-1-in-hex>, "2": <tile-2-in-hdex>, ...}
16:50:34  <TrueBrain> or maybe per row
16:51:29  <glx> and an option to skip map from json output ;)
16:52:35  <_dp_> for diffing it's somewhat useful to diff map array layers separately
16:52:49  <_dp_> coz some are just noise
16:54:19  <TrueBrain> glx: haha, yes, without doubt :)
16:54:24  <TrueBrain> off by default even
16:54:48  <_dp_> ideal would probaly diff on tile type first and then on fields
16:55:03  <_dp_> filtering out noise somehow
16:56:16  <glx> for that you need output like an array of tiles then
16:57:22  *** jottyfan has quit IRC
16:58:11  <glx> {id, m1, m2, ...} for each tile, then you can filter based on fields
17:02:17  *** snail_UES_ has joined #openttd
17:03:46  *** gelignite has joined #openttd
17:05:54  <Samu> who's michael lutz
17:06:36  <Samu> he created ScriptInstance "in_shutdown"
17:06:48  <Samu> seems to be only used for PriorityQueue
17:08:04  <TrueBrain> Want a pitchfork with that? :D
17:08:38  <Xaroth> How about a torch? we're running specials on torches
17:08:40  <glx> it's to prevent double destruction of stuff
17:08:43  <Xaroth> buy 2 pay for 3.
17:09:45  <glx> anyway it's not the issue, it just triggers it
17:15:17  <Samu> during garbage collection, active instance is always null
17:15:36  *** HerzogDeXtEr has joined #openttd
17:16:20  <Samu> wondering if there's an alternative
17:16:25  <Samu> without relying on instance
17:17:34  <Samu> seems kind of the wrong way to fix to activate the instance for garbage collection just so it won't crash on deleting priorityqueue items
17:18:07  <Samu> how are the items deleted for other things than priority que
17:18:07  *** snail_UES_ has quit IRC
17:18:27  <glx> no it's the right fix
17:20:21  <DorpsGek> [OpenTTD/OpenTTD] michicc opened pull request #9355: Codechange: Use dynamic string list for contents of land info window. https://git.io/JZSsI
17:22:58  <michi_cc> Samu: The associated e-mail in the commit might give a hint :)
17:30:51  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 opened pull request #9356: Fix #9353: [Script] Garbage collecting on priority queues could crash the game https://git.io/JZSCv
17:32:40  <DorpsGek> [OpenTTD/OpenTTD] glx22 approved pull request #9356: Fix #9353: [Script] Garbage collecting on priority queues could crash the game https://git.io/JZSWd
17:38:21  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain updated pull request #9322: Add: store table header for each chunk in savegame https://git.io/JGlmd
17:38:36  <TrueBrain> right, time to check if we can make the reader a bit faster :D
17:41:25  *** andythenorth has joined #openttd
17:41:48  *** Flygon has quit IRC
17:43:12  <Samu> thx for the fix :)
17:43:43  <Samu> I think my AI is still the only one using AIPriorityQueue yet
17:45:32  <Samu> https://github.com/SamuXarick/LuDiAI-AfterFix/blob/master/AyStar.nut#L7
17:58:18  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 closed issue #9353: Crash Report https://git.io/JZMcn
17:58:21  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 merged pull request #9356: Fix #9353: [Script] Garbage collecting on priority queues could crash the game https://git.io/JZSCv
18:18:47  <TrueBrain> from 3s to 1.5s already .. getting there
18:33:11  * Rubidium wonders what the conceptual difference between an error and a warning is in console_cmds. In one place being in the menu is an error and nothing gets done, in another it's a warning (and again nothing is done)
18:33:48  <TrueBrain> I have the same with debug levels :D
18:34:01  <TrueBrain> imo, an error doesn't continue, a warning corrects and continues
18:34:05  <Rubidium> and interestingly you can configure it in such a manner that warnings are not shown, so doing something stupid in a command that does warnings shows you no feedback you did something wrong
18:39:11  <TrueBrain> LZMA doesn't like small reads
18:39:12  <TrueBrain> interesting :)
18:39:20  <DorpsGek> [OpenTTD/OpenTTD] Rsmid opened issue #9357: Crash Report of AI https://git.io/JZSQV
18:52:06  *** D-HUND has quit IRC
18:54:52  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain commented on issue #9357: Crash Report of AI https://git.io/JZSQV
18:54:55  <DorpsGek> [OpenTTD/OpenTTD] TrueBrain closed issue #9357: Crash Report of AI https://git.io/JZSQV
18:56:11  <DorpsGek> [OpenTTD/OpenTTD] DorpsGek pushed 1 commits to master https://git.io/JZSAC
18:56:13  <DorpsGek>   - Update: Translations from eints (by translators)
18:56:25  <TrueBrain> wait, eints before 2100? Wowwww
18:57:29  <frosch123> it was also on time yesterday
18:57:42  <TrueBrain> "one time" is still a stretch
18:57:43  <frosch123> probably the dutch announcement to ban bitcoin
18:57:45  <TrueBrain> but yeah :P
18:58:38  <TrueBrain> hmm .. I have nothing anymore to change really ... it is now just slow because it is doing a lot
18:58:45  <TrueBrain>    803576    0.271    0.000    0.563    0.000 binreader.py:71(uint8)
18:58:52  <TrueBrain> almost a million uint8 calls :P
18:59:18  <TrueBrain> I could in theory make one huge struct.unpack per header, but .. meh
19:04:15  <Samu> just happened to reproduce the crash scenario, it no longer crashes!
19:10:38  <Samu> https://i.imgur.com/V5ZFqXy.png - went through to 25th Nov 1952
19:14:18  <Samu> ScriptAllocatorScope is called twice now, is that a problem?
19:25:46  *** jottyfan has joined #openttd
19:28:02  <andythenorth> enough working
19:28:08  * andythenorth wonders what to do now :P
19:31:55  <Samu> I reported something 2 days ago or so, I forgot what it was, lol
19:33:06  <Samu> ah, i remember, the Mungo AI thing with empty files in a .tar archive
19:33:19  <Samu> should I open an issue for it?
19:36:12  <glx> and I told you fileio.cpp:545 can probably be removed without issue
19:44:21  <DorpsGek> [OpenTTD/OpenTTD] SamuXarick opened issue #9358: Bug Report https://git.io/JZ93n
19:44:29  <Samu> ew, the title is ugly
19:51:51  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 opened pull request #9359: Consolidate IConsolePrint functions and improve some of the messages https://git.io/JZ9ZR
19:52:24  *** sla_ro|master has joined #openttd
19:57:23  <glx> Rubidium: you may want to link #8894 in your PR
19:58:35  <glx> and #8853
20:00:03  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 commented on pull request #9355: Codechange: Use dynamic string list for contents of land info window. https://git.io/JZ9Cw
20:00:19  <glx> and nice clang warnings :)
20:09:12  <DorpsGek> [OpenTTD/OpenTTD] michicc commented on pull request #9355: Codechange: Use dynamic string list for contents of land info window. https://git.io/JZ94L
20:13:10  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #9359: Consolidate IConsolePrint functions and improve some of the messages https://git.io/JZ9ZR
20:16:13  <DorpsGek> [OpenTTD/OpenTTD] michicc updated pull request #9355: Codechange: Use dynamic string list for contents of land info window. https://git.io/JZSsI
20:17:05  <Rubidium> glx: yeah, I thought about linking them though technically those do more or less the opposite of what I attempt to accomplish ;)
20:17:46  <glx> but they can be auto closed by your PR
20:19:24  <DorpsGek> [OpenTTD/OpenTTD] rubidium42 approved pull request #9355: Codechange: Use dynamic string list for contents of land info window. https://git.io/JZ90F
20:26:55  *** J0anJosep has joined #openttd
20:30:41  <DorpsGek> [OpenTTD/OpenTTD] michicc merged pull request #9355: Codechange: Use dynamic string list for contents of land info window. https://git.io/JZSsI
20:34:50  <TrueBrain> right, 1s .. that is 3 times as fast
20:34:51  <TrueBrain> guess I take it
20:44:01  *** Speeder has joined #openttd
20:53:17  <DorpsGek> [OpenTTD/OpenTTD] glx22 opened pull request #9360: Codechange: [Actions] Improve MSYS2 setup time https://git.io/JZ9D1
20:55:23  <TrueBrain> urwid doesn't like a lot of fields in a single entry
20:55:24  <TrueBrain> shocker :P
20:55:37  *** Samu has quit IRC
20:56:32  <TrueBrain> ottdc games have way too many trains :P
20:56:41  <TrueBrain> 15300, by the looks, in this one game
20:56:45  <TrueBrain> well, vehicles
20:56:47  <TrueBrain> not trains
20:56:47  <TrueBrain> ofc
20:57:06  <DorpsGek> [OpenTTD/OpenTTD] J0anJosep updated pull request #8480: Feature: Multitile depots https://git.io/JL5Hh
20:57:09  <frosch123> sounds like no newgrf
20:57:22  <TrueBrain> huh?
20:57:48  <frosch123> take the early 4/8 vehicles, and you need twice as many wagons for the same train length
20:58:35  <TrueBrain> well, plenty of NewGRFs in the game :P But .. I think ottdc always tries to find some limit :)
20:58:46  <frosch123> also, the steam smoke counts as vehicles
20:59:37  <TrueBrain> how ever you flip the cookie, 15300 vehicles
20:59:40  <frosch123> we increased the vehicle id to 24bit somewhen, because people hit the old 64k limit
20:59:42  <TrueBrain> and urwid has a hard time rendering that :)
21:00:43  <TrueBrain> okay, gave the reader some improvements
21:00:48  <TrueBrain> not only in speed, but also how things render
21:00:53  <frosch123> http://urwid.org/_images/graph2.png  <- "enable unicode graphics" xD
21:01:23  <TrueBrain> mainly as I really didn't want to look into the linkgraph saveload code :D
21:02:12  <TrueBrain> there is also blobs of data after each (running) AI and GS, but that is not for this PR :)
21:02:20  <frosch123> i guess when you are done with converting all chunks, you should add some code to prevent people creating new custom chunks :p
21:02:28  <TrueBrain> yeah
21:02:32  <TrueBrain> a lot of things can also be deduplicated
21:02:40  <TrueBrain> and with classes made more understandable
21:02:45  <TrueBrain> read: less room to fuck up
21:02:52  <_dp_> btw, visual effects being vehicles is quite a wtf
21:03:19  <TrueBrain> most chunks are a list of things, and each item is a "pre-filter", that can reject the item, and a "post-load-filter" to fix stuff up
21:03:29  <TrueBrain> so I kinda only want chunks to define the pre and post stuff
21:03:34  <TrueBrain> so the rest is forced to be correct :)
21:03:41  <TrueBrain> but ... lets first get this done :)
21:05:20  <frosch123> poor gog. the number of reviews is depressing, when comparing with steam
21:05:39  <TrueBrain> you expected another 500k users? :P
21:06:24  <frosch123> i did not expect steam to be more active after 9 weeks, than gog at launch
21:07:03  <TrueBrain> Steam is seriously popular :P
21:07:20  <TrueBrain> I would be curious what Epic would do, popularity-wise
21:07:24  *** nielsm has quit IRC
21:08:47  <TrueBrain> Steam Community page did finally die down a bit
21:09:03  <frosch123> i heard about origin and uplay, but never about the epic client
21:09:19  <TrueBrain> you completely missed the Epic Launcher? :D
21:09:28  <TrueBrain> they pushed in some serious money to make it work
21:09:32  <TrueBrain> lot of free games
21:09:43  <TrueBrain> games only on their platform for a year (like Satisfactory)
21:10:02  <frosch123> hey, i have bought 3 games in the past 15 years
21:10:02  <TrueBrain> Origin and Uplay are like the Blizzard launcher
21:10:05  <TrueBrain> "cute", at best
21:10:09  <TrueBrain> hahaha :D
21:10:14  <TrueBrain> okay, I have a tiny bit more games :P
21:11:57  <frosch123> also, did anyone push kamnet to signup at gog forums, or did he do that on his own? :p
21:12:09  <TrueBrain> he even asked if there was anything to moderate
21:12:19  <TrueBrain> he really picked up being the community manager of Steam and GOG
21:12:29  <TrueBrain> which I really appreciate :D
21:13:02  <TrueBrain> okay, JSON diff is always different, because of GLOG chunk :D
21:13:06  <frosch123> i liked that guy who prefered playing ttd in dosbox, because ottd has no ai
21:16:25  <TrueBrain> seeing the old AI is also fun :D
21:16:28  <TrueBrain> terrible, but fun :P
21:16:36  <TrueBrain> hmm .. jq is nice
21:16:40  *** sla_ro|master has quit IRC
21:16:43  <TrueBrain> jq 'del(.chunks.GLOG)
21:16:45  <TrueBrain> works :P
21:16:53  <TrueBrain> well, with closing ' ofc
21:20:43  <TrueBrain> weird, jsondiff is slow with 5MB JSON diffs :P
21:24:02  *** J0anJosep has quit IRC
21:25:18  <TrueBrain> I am still amazed that loading a savegame, saving it, loading both the old and new savegame, running it for 256, gives exactly the same JSON (minus GLOG) .. for most games at least :D
21:25:20  <TrueBrain> pretty sweet
21:27:29  <TrueBrain> lol, I have 1 game that has a different date .. lol, how? :P
21:28:29  *** andythenorth has quit IRC
21:33:30  <TrueBrain> well, I will let this run for ~2 hours, just to see if we really can still load old games .. but it sure does look that way :D Amazing :)
21:34:47  <TrueBrain> @calc 10 * 20 / 60
21:34:47  <DorpsGek> TrueBrain: 3.3333333333333335
21:34:52  <TrueBrain> make that ~3.5 hours
21:35:00  <TrueBrain> ha, I want to see a DHCP server do that!
21:35:16  <frosch123> add ais, they may do different thigns
21:35:28  <TrueBrain> jq 'del(.chunks.GLOG) | del(.chunks.AIPL) | del(.[] | select(. == {}))'
21:35:30  <TrueBrain> :D
21:35:38  <TrueBrain> (remove GLOG, remove AIPL, remove chunks if empty)
21:37:13  <_dp_> ha, AIPL, if AI really does something differently your whole diff is gonna explode :p
21:39:36  <TrueBrain> boy, this is slow ...
21:39:46  <TrueBrain> mainly jsondiff takes a long time
21:41:04  <TrueBrain> again a savegame where both savegames didn't run for the same amount of ticks
21:41:06  <TrueBrain> really odd
21:41:16  <TrueBrain> (as the null-drivers does exactly N ticks :P)
21:45:16  <DorpsGek> [OpenTTD/OpenTTD] JGRennison opened issue #9361: Off by one error in Packet::CanWriteToPacket https://git.io/JZHJy
21:45:58  <frosch123> hmm, there were many changes to realtime, gametime and other ticks
21:46:07  <frosch123> maybe null driver broke along that
21:46:15  <TrueBrain> good point
21:48:08  <TrueBrain> ah, found a faster jsondiff
21:48:10  <TrueBrain> that is much better
21:50:14  <TrueBrain> it is funny to see a whole game stored in JSON :D
21:50:29  <TrueBrain> lot of zeros :D
21:50:49  <glx> <TrueBrain> lot of free games <-- one every week
21:51:03  <glx> thursday it will be overcooked 2
21:51:49  <glx> but epic is windows only, even for games available on linux
21:51:50  <TrueBrain> btw frosch123 , struct in struct with SQLite really is now rather easy, it just needs to make a table for each struct. As everything is a CH_TABLE, everything has an index, en every SL_STRUCT is forced into a SL_STRUCTLIST, which also has an index
21:52:33  <TrueBrain> I now use this information too in the savegame-reader, to make struct in structs look a lot better
21:53:58  <frosch123> so a bunch of index columns :)
21:54:01  <TrueBrain> yes
21:54:10  <TrueBrain> one for each depth :)
21:54:23  <TrueBrain> not pretty, but should work
21:54:54  <frosch123> i assume the index in the substructs restarts at 0 for every poolitem? so the substruct table would have two columns: pool index + local index?
21:55:10  <frosch123> hmm, though that breaks if a poolitem ever has the same struct in different places
21:55:40  <TrueBrain> well, substructs don't have an index as such, but they have a length and are in the order of 0 .. N
21:55:47  <TrueBrain> so they have a "virtual" index? :P
21:56:06  <TrueBrain> what do you mean: "same struct in different places"?
21:56:46  <frosch123> towns had "delivered" and "produced" cargo or something
21:57:22  <frosch123> both used the same struct, though i guess you treat them as "different" structs in your format
21:57:34  <frosch123> yeah, i guess i overthought that :p
21:57:40  <TrueBrain> yeah, there are 2 things there: SaveLoad deals with names
21:57:52  <TrueBrain> and you cannot have 2 fields with the same name in a single SaveLoad description
21:57:58  <frosch123> it's not necessary to use the same table just because two fields have the same table structure
21:58:01  <TrueBrain> and indeed, each entry gets its own instance of that Handler
21:58:07  <TrueBrain> no :)
21:58:18  <TrueBrain> if you want that, you end up somewhere weird and scary :P
21:58:43  <frosch123> nah, i don't think i want it anymore :p
21:58:50  <TrueBrain> :D
21:58:53  <frosch123> it was just my c++ template mind
21:59:07  <TrueBrain> but I have to make an instance of each handler every time they are used anyway
21:59:08  *** jottyfan has quit IRC
21:59:11  <TrueBrain> as I need to load the header per entry
21:59:15  <TrueBrain> and they might be the same class now
21:59:21  <TrueBrain> but they might not have been the same when saving
21:59:30  <TrueBrain> (and it stores the table in the instance)
22:00:00  <frosch123> i probably told you about that c programmer who was too lazy to define structs, so he stored everything in uint32 arrays, and casted everything (including pointers), in every access to the array
22:00:15  <frosch123> the equivalent in c++ is to use std::tuple for everything
22:00:23  <glx> looks like some part of openttd
22:00:37  <glx> (but not intentionally)
22:01:09  <TrueBrain> frosch123: yeah ... no :P
22:01:31  <glx> hey using array is cast is more work than defining structs
22:01:41  <glx> s/is/cast/
22:01:50  <glx> and
22:01:55  <TrueBrain> we already have some really tricky parts in the SaveLoad code
22:01:55  <glx> I can't type
22:02:07  <TrueBrain> I also found out today that std::vector<bool> does tricks :)
22:02:25  <frosch123> yeah, that one is pretty unpopular :p
22:02:27  <glx> it's a special case
22:02:31  <glx> IIRC
22:02:36  <TrueBrain> I blacklisted it from the SaveLoad system
22:02:45  <TrueBrain> you cannot have a std::vector<bool> :P
22:02:56  <frosch123> people use std::vector<uint8> or std::basic_string<bool> instead
22:03:04  <TrueBrain> basic_string, lolz
22:03:06  <_dp_> bitset! :p
22:03:11  <frosch123> bitset is fixed size
22:03:28  <TrueBrain> but yeah, a lot of our SaveLoad code first casts stuff to void*
22:03:35  <TrueBrain> in the hope it had enough information to cast it back to what-ever it was properly
22:03:42  <TrueBrain> with.. surprisingly little type-checking
22:03:55  <TrueBrain> so you can easily try to store a std::deque via std::vector
22:04:03  <glx> oh don't look in the dll magic thing I added then ;)
22:04:10  <frosch123> TrueBrain: std::basic_string<T> is similary to std::vector<T>, but it features small-vector optimisation and operator+ :)
22:04:29  <TrueBrain> I don't even ....
22:07:07  <glx> usually a void* cast is used to prevent warning for the "real" cast
22:07:29  <TrueBrain> it could really use classes and not casting :)
22:08:13  <glx> but saveload use generic pointers, because macros
22:10:17  <TrueBrain> nothing that has to last :D
22:13:07  <TrueBrain> hmm .. here a savegame that is on the same date, but is slightly different
22:13:46  <TrueBrain> they are around the same version of which I also have variants that fail to load
22:13:47  <TrueBrain> might be related
22:14:26  <TrueBrain> so I load&save the game, which I call original
22:14:40  <TrueBrain> euh, no
22:14:45  <TrueBrain> I call the old savegame original
22:14:53  <TrueBrain> I load&save&load it, and I call that main
22:15:01  <TrueBrain> original "max_train_length" == 100
22:15:04  <TrueBrain> main "max_train_length" == 64
22:15:17  <TrueBrain> in other words: loading of an old game and saving it, seems to give 100
22:15:22  <TrueBrain> load + save + load + save gives 64
22:15:28  <TrueBrain> I think we miss a clamp somewhere :P
22:15:55  <TrueBrain> (as the hard-coded max really is 64)
22:19:39  <TrueBrain> for (Train *t : Train::Iterate()) _settings_game.vehicle.max_" target="_blank">game.vehicle.max_train_length = std::max<uint8>(_settings_game.vehicle.max_" target="_blank">game.vehicle.max_train_length, CeilDiv(t->gcache.cached_total_length, TILE_SIZE));
22:19:43  <TrueBrain> might be related :P
22:20:18  <TrueBrain> so very nice that the afterload.cpp makes the value so your current trains are valid
22:20:24  <TrueBrain> but the next save/load it clamps to 64 again
22:20:28  <TrueBrain> "temporary" solution? :D
22:21:29  <frosch123> haha, so you have a testgame with a train longer than 64 tiles? :p
22:21:41  <TrueBrain> I would guess so, yes :)
22:21:54  <TrueBrain> bit funny, 80% of the games that fail are supplied by Aro :P
22:22:06  <frosch123> typical developer
22:22:08  <TrueBrain> clearly his games are weird :P
22:22:29  <TrueBrain> (which is a really good thing :D)
22:22:57  <TrueBrain> and I have 1 game where a few road vehicles made other choices
22:23:58  <TrueBrain> even supplied stations differently
22:25:36  <TrueBrain> and one game that will 100% desync if the server loaded the old savegame and had someone else join it
22:25:43  <TrueBrain> no clue what it is doing, but it is two totally different things :P
22:27:07  <TrueBrain> remove: /chunks/VEHS/98/effect/0 / add: /chunks/VEHS/98/roadveh/0
22:27:39  <TrueBrain> bit confused by the same index .. as that suggests that roadveh is created after the effect stopped?
22:28:31  <frosch123> autoreplace/renew?
22:28:37  <TrueBrain> possibly
22:29:19  <TrueBrain> so far all the savegames that show a real diff, seem to be roadveh related
22:31:20  <TrueBrain> guess the next step is making it save a savegame every tick :P
22:33:19  <_dp_> if seed changed vehicle diff may not mean anything
22:33:37  <_dp_> especially stuff like busses that instantly desyncs because of house production
22:37:40  <TrueBrain> https://gist.github.com/TrueBrain/675fca01ceee6022c9e6e82d78f104f8 <- one of the weirdest diff I have so far
22:37:59  <TrueBrain> in the load/save/loaded game, some vehicles no longer exist after 256 ticks :P
22:38:27  <glx> effects ?
22:38:46  <TrueBrain> it says roadveh
22:38:51  <TrueBrain> I haven't verified any of it btw
22:39:34  <TrueBrain> owh, it is not the vehicles that are not there, just the path that is different
22:40:44  <TrueBrain> path-finder cache
22:41:07  <TrueBrain> so the state is not different enough to say boom, but is getting close to that point, I would guess :P
22:41:53  <TrueBrain> given by the other diffs, if I would have to guess, something around the roadveh pathfinder cache is not completely okiedokie
22:43:04  <frosch123> aw, replace only shows the new value, not the old one
22:43:37  <TrueBrain> "path.td": [], "path.tile": [] vs  "path.td": [1], "path.tile": [32208] for vehicle 3011 :)
22:43:56  <frosch123> well, "frame" is different, so the rv already drove differently
22:44:36  <frosch123> oh, motion_counter is different
22:44:41  <TrueBrain> yeah, seems 2477 did something different for sure
22:44:45  <frosch123> that's incremented everytime the vehicle moves
22:44:45  <glx> any crossed junction will change path cache
22:44:59  <glx> s/will/may/
22:45:03  <TrueBrain> glx: these are 2 savegames that are suppose to be identical from start :)
22:45:17  <TrueBrain> only difference is afterload yes/no :P
22:46:06  <TrueBrain> it is funny, most of these games are different but in multiplayer won't have desync'd yet :)
22:46:45  <_dp_> yeah, it takes quite some time to desync sometimes
22:47:30  <TrueBrain> in good news, what-ever this is, it is deterministic :)
22:48:30  <_dp_> I've see vehicle go across the map desynced and only fail when it started loading/unloading
22:49:54  <TrueBrain> okay, all real diffs I have so far have at least 1 roadveh with a now-empty path.td
22:51:34  <TrueBrain> still running, but so fart only 11 out of the 450 games it did :P
22:51:52  <frosch123> coop is not known for rv :p
22:52:01  <TrueBrain> they had a few games, but indeed
22:52:07  <TrueBrain> owh, and a diff on aricraft cargo age
22:52:11  <TrueBrain> that is also interesting :)
22:53:40  <TrueBrain> but okay, before really investigating, I first need to validate that the savegames are what I think they are :)
23:00:22  <TrueBrain> lol, it even happens on a TTDp savegame :P
23:01:11  <TrueBrain> cool, 1.9 titlegame fails too :D
23:01:36  <TrueBrain> 18 failures in ~500 savegames
23:01:49  <TrueBrain> so in general our afterload stuff seems rather solid :)
23:01:59  *** WormnestAndroid has quit IRC
23:02:12  *** WormnestAndroid has joined #openttd
23:02:38  <glx> when done correctly
23:02:55  <glx> I think there was a case of broken conversion
23:03:04  <TrueBrain> multiple :P
23:03:21  <TrueBrain> but I am surprised it still works .. I wouldn't be surprised if loading a 0.4 savegame didn't work, for example
23:03:25  <TrueBrain> as who tries that in 2021 :P
23:04:06  <TrueBrain> okay, I fixed the date issue .. disabled threading for savegames
23:05:23  <TrueBrain> takes btw 1 hour to run this test-suite :)
23:09:09  *** Wolf01 has quit IRC
23:14:09  <frosch123> lol, so null driver actually now does different amount of ticks :)
23:14:43  <TrueBrain> code-wise it shouldn't
23:16:32  <TrueBrain> the other possibility is that "game_start.scr" isn't always doing the same thing
23:18:02  <TrueBrain> lol, I now have a diff where on one of the two games the pause was never removed
23:18:07  <TrueBrain> in the "game_start.scr" is "unpause"
23:18:52  <TrueBrain> (the roadveh issue remains btw, and seems to have nothing to do with this)
23:19:33  <TrueBrain> PM_PAUSED_LINK_GRAPH
23:20:06  <TrueBrain> so it does unpause
23:20:11  <TrueBrain> but one of the two is paused because of linkgraph
23:23:03  <TrueBrain> guess for this kind of testing I should compile completely without threads
23:25:02  *** outtatime has quit IRC
23:25:12  *** outtatime has joined #openttd
23:27:09  <frosch123> night
23:27:11  *** frosch123 has quit IRC
23:29:31  <TrueBrain> okay, linkgraph was indeed the one giving the date differences
23:29:32  <TrueBrain> makes sense
23:29:54  <TrueBrain> I mean, it does indeed indicate one of the two games did a tick more or what-ever :P
23:49:23  *** norri has quit IRC

Powered by YARRSTE version: svn-trunk