Times are UTC Toggle Colours
02:46:57 *** D-HUND has joined #openttd 02:50:20 *** debdog has quit IRC 03:29:00 *** keikoz has joined #openttd 04:33:15 <DorpsGek> [OpenTTD/OpenTTD] marcossouza1232 opened pull request #11012: fix #10995 https://github.com/OpenTTD/OpenTTD/pull/11012 04:51:19 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #11007: Codechange: make creating temporary StringParameters easier https://github.com/OpenTTD/OpenTTD/pull/11007 05:43:35 *** merni has joined #openttd 05:43:35 <merni> In the https://www.openttd.org/news/2023/06/10/openttd-13-2.html , the changelog only shows the changes in 13.2.1 and not those in 13.2 06:33:33 *** Flygon has quit IRC 06:47:49 <DorpsGek> [OpenTTD/OpenTTD] 2TallTyler commented on pull request #11012: fix #10995 https://github.com/OpenTTD/OpenTTD/pull/11012#pullrequestreview-1480744061 07:16:04 *** HerzogDeXtEr has joined #openttd 07:44:44 *** audunm has joined #openttd 07:44:44 <audunm> Sorry in advance. Testing if I can talk to myself from discord to matrix via IRC. 07:45:02 <audunm[m]> I can! 07:54:35 *** _aD has joined #openttd 08:03:42 <FLHerne> that's neat 08:04:24 <FLHerne> Is there a global OFTC irc-matrix bridge then? 08:11:58 <Rubidium_> no you can't talk to yourself; the discord bridge doesn't support private messages 08:50:26 <DorpsGek> [OpenTTD/OpenTTD] 2TallTyler approved pull request #10687: Codechange: Simplify drawing of timetables https://github.com/OpenTTD/OpenTTD/pull/10687#pullrequestreview-1481043999 09:07:58 *** _aD is now known as Guest3155 09:08:02 *** _aD has joined #openttd 09:09:58 *** Guest3155 has quit IRC 09:12:49 <TrueBrain> okay; I got the hang of how to scale up and down with this new cluster, and fixed the stupid cgroup bug for now .. after lunch, let's deploy this lovely bananas to production 🙂 09:16:34 *** _aD is now known as Guest3156 09:16:39 *** _aD has joined #openttd 09:17:19 *** Guest3156 has quit IRC 09:18:58 <TrueBrain> wow, the wiki is using 337 MB of RAM .. I had it limited to 384, but that is cutting it rather close 😄 09:20:03 <TrueBrain> most of that memory is the metadata, like all categories, files, links, templates etc that exist .. 09:20:19 <TrueBrain> seems it has grown a bit over the years 🙂 09:21:30 <LordAro> TrueBrain: TrueWiki is inefficient! 09:21:39 <TrueBrain> yes, let's go with that 😛 09:22:40 <pickpacket> TrueBrain: isn't that a lot of RAM though? I would've assumed that stuff like that was in a DB 09:22:50 <pickpacket> or cached 09:22:58 <TrueBrain> and a DB would be magic and won't consume RAM? 09:23:01 <TrueBrain> or a cache won't? 09:23:19 <pickpacket> cache as in static files 09:23:31 <TrueBrain> and how do you cache a tree of information in a static file? 😄 09:23:32 <pickpacket> and a DB is usually pretty optimised 09:23:36 <pickpacket> true 09:23:45 <TrueBrain> I dare you to find a DB that would hold this information for less memory 🙂 09:24:05 <pickpacket> Challenge not accepted! 💪 09:24:19 <pickpacket> 😛 09:24:41 <pickpacket> how big is the wiki in total? I think you mentioned the other day but my IRC search is bad 09:25:17 <TrueBrain> ~400MB of plain text 09:25:24 <TrueBrain> before rendering etc 09:32:23 <petern> https://cdn.discordapp.com/attachments/1008473233844097104/1118835388501463071/image.png 09:32:27 <petern> I think that might be lying... 09:32:43 <TrueBrain> at least you know when it is getting up already 09:39:11 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 commented on pull request #11001: Feature: Transparency option for cost and income indicators https://github.com/OpenTTD/OpenTTD/pull/11001#pullrequestreview-1481147862 09:45:19 <TrueBrain> ah, okay, TrueWiki only actually uses ~150MB RAM, the rest is Python stuff; after rendering a few big pages, it doesn't free the memory directly 09:45:42 <TrueBrain> (rendered pages are cached on disk, and served from disk from there on; but it ofc takes Python a bit of time to go like: owh, I can free this memory now :P) 09:46:09 <TrueBrain> the `git clone` does consume a bit of memory on startup 😄 I should add a fetch-depth, I guess ... 10:42:13 *** _aD has quit IRC 10:44:19 <DorpsGek> [OpenTTD/OpenTTD] orudge updated pull request #11010: Change: [Actions] Use notarytool for notarization instead of gon https://github.com/OpenTTD/OpenTTD/pull/11010 10:48:25 <DorpsGek> [OpenTTD/OpenTTD] TrueBrain approved pull request #11010: Change: [Actions] Use notarytool for notarization instead of gon https://github.com/OpenTTD/OpenTTD/pull/11010#pullrequestreview-1481270891 11:58:37 <TrueBrain> right .. time to flip the switch .. let's think a bit how to do that with the least amount of downtime .. hmm 12:02:43 *** _aD has joined #openttd 12:07:19 <pickpacket> TrueBrain: what could possibly go wrong? Just do it! 12:07:27 <TrueBrain> well, it serves an empty page now 12:07:30 <TrueBrain> clearly that can go wrong 😛 12:07:34 <pickpacket> :D 12:09:07 <petern> I'm broken. nullptr does not work in C#... 12:09:07 <Rubidium_> just pause the RL-hypervisor temporarily :D 12:09:32 <TrueBrain> turns out you cannot change a hostname with Pulumi / Terraform in Cloudflare 12:09:33 <TrueBrain> funny bug 12:11:04 <Rubidium_> petern: what about something like `public static class CXXDev { public const object nullptr = null; }` and in the other files `using static CXXDev;`? 12:11:21 <petern> Yes definitely doing that... 12:13:07 <petern> Hmm, GRF scanning is funny, it goes through the effort to build a sorted linked-list on the fly, and then once it's done that, it copies all the item pointers into a vector, sorts that, and then relinks the linked-list from it. 12:17:40 <TrueBrain> okay, I think everything is still working as it should 12:22:04 <DorpsGek> [OpenTTD/BaNaNaS] TrueBrain opened pull request #137: Fix: [CI] migrate to new backend for reload triggers https://github.com/OpenTTD/BaNaNaS/pull/137 12:23:32 <TrueBrain> 13.3 works, master works, I can login, I can upload files .. I think the flipping of the switch went okay 🙂 12:23:43 <DorpsGek> [OpenTTD/BaNaNaS] TrueBrain merged pull request #137: Fix: [CI] migrate to new backend for reload triggers https://github.com/OpenTTD/BaNaNaS/pull/137 12:23:54 <TrueBrain> the one thing I cannot test, if uploading new content on the production version actually works correctly .. on preview it does at least 🙂 12:26:35 <DorpsGek> [OpenTTD/BaNaNaS] TrueBrain opened pull request #138: Fix: [CI] use the correct service name for the reload triggers https://github.com/OpenTTD/BaNaNaS/pull/138 12:26:46 <DorpsGek> [OpenTTD/BaNaNaS] TrueBrain merged pull request #138: Fix: [CI] use the correct service name for the reload triggers https://github.com/OpenTTD/BaNaNaS/pull/138 12:27:25 <glx[d]> Again ? 12:27:30 <TrueBrain> yes 12:27:31 <TrueBrain> and 3 more times to go 😛 12:27:37 <TrueBrain> I am incapable of doing this correct, it turns out 12:27:42 <TrueBrain> but I am happy you point it out 😛 😄 😄 12:28:35 <TrueBrain> hmmm ... one of the secrets on GitHub is wrong .. that is odd 12:33:20 *** _aD has quit IRC 12:35:40 <DorpsGek> [OpenTTD/BaNaNaS] TrueBrain opened pull request #139: Fix: [CI] Use the API token for the API reload https://github.com/OpenTTD/BaNaNaS/pull/139 12:35:41 <TrueBrain> it is not my day ... 12:35:53 <DorpsGek> [OpenTTD/BaNaNaS] TrueBrain merged pull request #139: Fix: [CI] Use the API token for the API reload https://github.com/OpenTTD/BaNaNaS/pull/139 12:35:59 <pickpacket> TrueBrain: I believe in you ❤️ 12:36:45 <TrueBrain> right, now it finally works 🙂 12:37:46 <pickpacket> yay! 12:40:16 <DorpsGek> [OpenTTD/bananas-api] TrueBrain opened pull request #390: Fix: [CI] use the correct service name for a release deployment https://github.com/OpenTTD/bananas-api/pull/390 12:40:19 <DorpsGek> [OpenTTD/bananas-server] TrueBrain opened pull request #343: Fix: [CI] use the correct service name for a release deployment https://github.com/OpenTTD/bananas-server/pull/343 12:40:22 <DorpsGek> [OpenTTD/bananas-frontend-web] TrueBrain opened pull request #215: Fix: [CI] use the correct service name for a release deployment https://github.com/OpenTTD/bananas-frontend-web/pull/215 12:40:23 <TrueBrain> at least it was a consistent mistake 😄 12:45:17 <TrueBrain> `ReadableStream.tee() buffer limit exceeded` .. oops, can't serve big files, it seems .. 12:48:41 <petern> Just remove [az]Base 🙂 12:48:50 <TrueBrain> it is yeti that triggered the issue 😛 12:49:57 <petern> Okay, std::forward_list<T> is pretty unusable. I'm still weighing up std::list<T> or std::vector<T *> 12:53:29 <TrueBrain> ah, YETI is 182MB 12:56:00 <petern> Host everything on Steam workshop 😉 12:56:07 <TrueBrain> ghehe 12:56:40 <TrueBrain> it is a bit annoying .. every documentation mentions you should fetch the result, clone it, put one clone into cache, return the other to theu ser 12:56:50 <TrueBrain> but ... the clone copies the whole body, instead of streaming it 12:57:04 <TrueBrain> and the workers are limited to 128MB RAM .. and 182 > 128, in case you didn't know 🙂 12:57:37 <petern> They probably mean large files like 10KB 12:57:54 <TrueBrain> you can store up to 512MB, so it should kinda work honestly 13:06:14 <Rubidium_> petern: the current GRF "list" is essentially a std::forward_list, isn't it? 13:17:56 <petern> Conceptually, however the interface for wrangling with it is different, and different enough from std::list/std::vector to require learning a new set of skills :p 13:25:00 <petern> emplace() doesn't exist, emplace_after() will work but doesn't return the item you've just emplaced, so unless everything can be done in the constructor, it seems to be painful to get back the item you've just added. 13:29:15 <TrueBrain> there, fixed the issue with streaming large files .. as extra bonus, downloads start sooner now 🙂 13:31:14 <Rubidium_> that indeed makes it more annoying to use 13:33:54 <Rubidium_> and then having to keep the iterator to the end, or keep looking for the last element... std::list seems indeed a lot simpler 13:39:03 <Rubidium_> though I'd just go with std::vector 13:39:20 <petern> Yes, that's what I'm leaning to. 13:39:50 <petern> Especially as there's a place in networking where a grflist is either loaded from network packets, or it's the actual live _grfconfig. 13:40:44 <petern> Using either a std::forward_list or std::list would mean having to make a copy of _grfconfig, or use (and worse, manage) pointers to lists... 13:44:49 <TrueBrain> right, seems BaNaNaS is about 5GB an hour atm .. so about 4TB a month, seems about right .. let's see how this works out 🙂 13:47:00 <petern> I remember when I used to host a mirror (back when we did that) but had to remove it as the traffic swamped $employers' real traffic. 13:47:13 <TrueBrain> I remember that too 😄 13:47:17 <TrueBrain> you were not alone in that issue .. 13:47:49 <TrueBrain> several offers of mirrors where I told people the expected bandwidth, they said: no way, lol, you are not doing that bandwidth, demenaded to be connected .. within a month were like: yeah, I can't afford this 13:47:50 <petern> Later I had an OVH host that could've mirrored but it was basically when we were already using OVH so wouldn't've added much. 13:50:24 <TrueBrain> lol, CDN cache hit/miss ratio so far: 1350 misses, 36 hits 13:50:39 <petern> Just warming up? 13:50:43 <TrueBrain> 😄 It might take a while for the cache to become useful .. if things aren't purged every time before it is rerequested 🙂 13:50:47 <TrueBrain> I really hope so 🙂 13:51:01 <TrueBrain> `public, max-age=31536000, immutable` hopefully that hints enough it can be cached, really, it can 13:52:06 <DorpsGek> [OpenTTD/OpenTTD] orudge merged pull request #11010: Change: [Actions] Use notarytool for notarization instead of gon https://github.com/OpenTTD/OpenTTD/pull/11010 13:52:13 <DorpsGek> [OpenTTD/bananas-frontend-web] TrueBrain merged pull request #215: Fix: [CI] use the correct service name for a release deployment https://github.com/OpenTTD/bananas-frontend-web/pull/215 13:52:25 <TrueBrain> let's see if releases finally work 🙂 13:57:12 <TrueBrain> it is funny .. we replaced BaNaNaS, what, 3 years ago? 4? And people still try the old URL from time to time 🙂 13:57:30 <TrueBrain> like: don't give it! YOUR DAY MAY COME! 13:58:03 <DorpsGek> [OpenTTD/bananas-server] TrueBrain merged pull request #343: Fix: [CI] use the correct service name for a release deployment https://github.com/OpenTTD/bananas-server/pull/343 14:01:42 <TrueBrain> `/index.php/photo-gallery-a-movie/a-few-from-summer-2011/summer5-290` .. do I want to know why someone requests that URL from our servers? 14:01:44 <TrueBrain> feels weird 😛 14:06:20 <DorpsGek> [OpenTTD/bananas-api] TrueBrain merged pull request #390: Fix: [CI] use the correct service name for a release deployment https://github.com/OpenTTD/bananas-api/pull/390 14:09:12 <TrueBrain> okay ... so tomorrow ... we port a few more services over, I guess 🙂 Should go a lot quicker now, given all infra should be there now 😄 14:09:50 <TrueBrain> the release CDN will be interesting to port over .. there is a lot of data in there 14:10:47 <TrueBrain> sadly, AWS S3 doesn't tell you at all .. Cloudflare R2 at least does tell you those basic things which they use to charge you with 14:11:02 <TrueBrain> AWS is really stupid in those regards .. like .. "you will find out when the bill lands" 14:11:38 <TrueBrain> but, on estimate .. 150GB of data on that bucket ... 100k files .. every nightly we ever produced is in there 😛 14:24:12 *** _aD has joined #openttd 14:32:35 *** Kitrana has joined #openttd 14:35:29 *** nielsm has joined #openttd 14:36:29 *** Kitrana1 has quit IRC 14:42:09 *** Kitrana has quit IRC 14:43:22 *** felix_ has joined #openttd 14:45:34 *** felix has quit IRC 14:49:37 *** Kitrana has joined #openttd 14:52:06 *** Kitrana1 has joined #openttd 14:53:09 <DorpsGek> [OpenTTD/OpenTTD] zephyris updated pull request #11001: Feature: Transparency option for cost and income indicators https://github.com/OpenTTD/OpenTTD/pull/11001 14:54:05 <DorpsGek> [OpenTTD/OpenTTD] zephyris commented on pull request #11001: Feature: Transparency option for cost and income indicators https://github.com/OpenTTD/OpenTTD/pull/11001#pullrequestreview-1481782578 14:57:39 *** Kitrana has quit IRC 15:00:13 *** gelignite has joined #openttd 15:13:34 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 opened pull request #11013: Codechange: move StringParameters to strings_internal.h https://github.com/OpenTTD/OpenTTD/pull/11013 15:25:32 *** Eddi|zuHause has quit IRC 15:26:14 *** Eddi|zuHause has joined #openttd 15:38:46 <petern> > Segmentation fault (core dumped) 15:38:47 <petern> > error MSB3073: The command "webcompiler wwwroot/scss -r -c webcompilerconfiguration.json" exited with code 139. 15:38:50 <petern> Well, that's not what I want 😦 15:40:21 <TallTyler> That does sound less than ideal 15:43:19 <petern> Hmm, worked second time with no changes. That's also not good, but at least it is going through. 15:58:12 <glx[d]> hmm eints is still on old workflows, so a release is needed, but maybe I should wait 15:59:04 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 approved pull request #11001: Feature: Transparency option for cost and income indicators https://github.com/OpenTTD/OpenTTD/pull/11001#pullrequestreview-1481938120 15:59:50 <DorpsGek> [OpenTTD/OpenTTD] LordAro commented on pull request #11001: Feature: Transparency option for cost and income indicators https://github.com/OpenTTD/OpenTTD/pull/11001#issuecomment-1593340484 16:00:08 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 approved pull request #10890: Feature: Create group of vehicles from manage vehicle list button. https://github.com/OpenTTD/OpenTTD/pull/10890#pullrequestreview-1481941263 16:34:14 <DorpsGek> [OpenTTD/website] merni-ns opened issue #265: Wrong changelog for 13.2 https://github.com/OpenTTD/website/issues/265 16:39:53 <DorpsGek> [OpenTTD/website] TrueBrain opened pull request #266: Fix: mention the changelog for both 13.2 and 13.2.1 https://github.com/OpenTTD/website/pull/266 16:39:58 <TrueBrain> So people actually use those links; surprising 😛 16:40:27 <DorpsGek> [OpenTTD/website] LordAro commented on pull request #266: Fix: mention the changelog for both 13.2 and 13.2.1 https://github.com/OpenTTD/website/pull/266#issuecomment-1593394924 16:41:02 <TrueBrain> meh; do we actually care enough to erase it from existence? Fine .... 16:41:53 <DorpsGek> [OpenTTD/website] TrueBrain updated pull request #266: Fix: mention the changelog for both 13.2 and 13.2.1 https://github.com/OpenTTD/website/pull/266 16:49:39 <TrueBrain> pfff, he can make a comment within a minute, but can't even approve in 8? I am dissapointed LordAro 😛 😛 😛 16:54:17 <TrueBrain> https://cdn.discordapp.com/attachments/1008473233844097104/1118946592481886350/image.png 16:54:17 <TrueBrain> slowly growing, what is cached and what is not 🙂 17:11:48 <LordAro> TrueBrain: i went home :p 17:12:01 <TrueBrain> shitty excuses 😛 17:12:04 <DorpsGek> [OpenTTD/website] LordAro approved pull request #266: Fix: remove any mention of 13.2.1 https://github.com/OpenTTD/website/pull/266#pullrequestreview-1482064710 17:12:07 <TrueBrain> 😄 😄 17:12:08 <DorpsGek> [OpenTTD/website] TrueBrain merged pull request #266: Fix: remove any mention of 13.2.1 https://github.com/OpenTTD/website/pull/266 17:12:11 <TrueBrain> Tnx 🙂 17:12:15 <DorpsGek> [OpenTTD/website] TrueBrain closed issue #265: Wrong changelog for 13.2 https://github.com/OpenTTD/website/issues/265 17:12:31 <TrueBrain> haha, Hashicorp (Nomad) is going to send me a t-shirt for fixing that bug I found yesterday 🙂 17:12:34 <LordAro> :D 17:13:08 <TrueBrain> I like how they handle their Open Source as a business; it is nice 🙂 17:13:49 <LordAro> i did a bit of reading up of nomad today 17:15:21 <TrueBrain> it is a more understandable way of kubernetes; and a less shitty way of AWS ECS 😛 17:15:33 <LordAro> definitely interesting 17:15:51 <LordAro> but as previously, i just don't have the need for a) cloud-based things and b) such high availability 17:16:12 <TrueBrain> that is why I use OpenTTD as my garden 😄 17:16:30 <TrueBrain> otherwise I wouldn't have a need for it either; but it is a good way to understand how these technologies are evolving 17:16:58 <TrueBrain> I (strongly) advise companies to not switch to kubernetes, unless they want to invest 2 FTE; which mostly scares them away 17:17:03 <TrueBrain> but the next question they ask: but what else? 17:17:26 <TrueBrain> Azure Containers are pretty decent .. AWS Fargate is .. and now Nomad is pretty high on that list 🙂 17:18:42 <LordAro> i do get the feeling of 'being left behind' from time to time 17:19:08 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 opened pull request #11014: Codechange: remove a number of unneeded c_str() calls https://github.com/OpenTTD/OpenTTD/pull/11014 17:19:11 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 opened pull request #11015: Codechange: make SQString::Create that supports std::string and use that https://github.com/OpenTTD/OpenTTD/pull/11015 17:19:15 <TrueBrain> that is a pretty normal feeling to have, and one you should have; IT is going so quickly .. if you stand still, you become a specialist in a certain field, and that is it 🙂 17:19:34 <LordAro> all of the containers i have are just "mostly automated" 17:19:40 <TrueBrain> I worked with too many people and teams the last few years where they thought their ideas of 5 years ago were modern ... they weren't 17:21:11 <DorpsGek> [OpenTTD/OpenTTD] TrueBrain approved pull request #11014: Codechange: remove a number of unneeded c_str() calls https://github.com/OpenTTD/OpenTTD/pull/11014#pullrequestreview-1482078786 17:21:39 <DorpsGek> [OpenTTD/OpenTTD] LordAro commented on pull request #11015: Codechange: make SQString::Create that supports std::string and use that https://github.com/OpenTTD/OpenTTD/pull/11015#issuecomment-1593458538 17:21:57 <LordAro> i'm pretty sure lxd was modern when we started using it 6 years ago 17:22:07 <LordAro> it had bugs in it and everything 17:22:18 <TrueBrain> and by now it is old news 😄 17:22:35 <TrueBrain> runc is the new kid on the block 17:22:48 <TrueBrain> (it isn't new btw; which is the fun part) 17:23:05 <TrueBrain> first release seems to be from 2015 .. lol 17:24:29 <LordAro> interesting 17:24:54 <TrueBrain> what I also find interesting, and what I said 3 years ago already, that Docker is now finally also actually going away 17:25:00 <TrueBrain> we start to call it OCI containers 17:25:02 <LordAro> mm 17:25:13 <TrueBrain> and Docker is mentioned less and less in reference to containerization 17:26:05 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #11015: Codechange: make SQString::Create that supports std::string and use that https://github.com/OpenTTD/OpenTTD/pull/11015 17:26:06 <LordAro> i've never been quite sure we've been doing lxd right anyway, tend to just treat them as lightweight VMs, all stateful and just upgrading them in place - some of them still say "image: ubuntu:16.04" or similar in the config 17:26:24 <TrueBrain> oof 17:26:33 <TrueBrain> I know of little companies that use lxd, I have to admit 17:28:38 <TrueBrain> one of the reasons why I kinda like the old and new OpenTTD infra, is because everything, except for a few things like the DorpsGek chat logs, is not actually stored in the cluster 17:28:44 <TrueBrain> that is to say, if the cluster burns down 17:28:46 <TrueBrain> you spin up a new one 17:28:50 <TrueBrain> and you still have everything as you had before 17:28:56 <TrueBrain> nothing is lost .. like .. nothing 17:29:06 <TrueBrain> I really like that kind of container idea 17:29:25 <TrueBrain> but I still see way too many companies build stateful and persistent payloads in their clusters 17:30:14 <LordAro> so all the actual data is stored on some "thing" somewhere else, and all the containers/nodes/whatever just point to the right part of it? 17:30:27 <TrueBrain> yup 17:30:34 <TrueBrain> like BaNaNaS is on GitHub and a S3-compatible bucket 17:30:35 <LordAro> the equivalent in my head is a centralised database server, rather than each container running their own 17:30:38 <LordAro> (which we also do..) 17:30:41 <TrueBrain> so the actual instance has no actual value in terms of content 17:30:59 <TrueBrain> so the containers can crash all night long, but it doesn't matter, like, at all 17:31:09 <LordAro> aye 17:31:11 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 commented on pull request #11015: Codechange: make SQString::Create that supports std::string and use that https://github.com/OpenTTD/OpenTTD/pull/11015#issuecomment-1593474913 17:31:37 <TrueBrain> what I like most of it, is that few things are emergencies .. does a container crash when a user visits a certain URL? I will fix it tomorrow 17:31:51 <TrueBrain> (the wiki had that, for months .. once every week or so someone visited a certain link, crashing the wiki) 17:32:03 <LordAro> the issue i don't really know the solution to is how to achieve that sort of storage solution (without "magical clouds" doing it for you) 17:32:14 <LordAro> for us at work, we're just not big enough to justify something like ceph 17:32:27 <TrueBrain> OpenTTD also isn't big .. but it is being a bit creative 🙂 17:32:52 <LordAro> it's "bigger" in terms of services :) 17:32:53 <TrueBrain> having an NFS shared mount on all hosts could already be a step, for example 17:33:07 <TrueBrain> or having a redis cluster 17:33:12 <LordAro> i've had so many bad experiences with NFS mounts :p 17:33:20 <TrueBrain> NFSv4 is pretty good 17:33:37 <TrueBrain> AWS EFS is basically NFS, and the amount of issues .. I can still count them on my hands 17:33:46 <JGR> With NFS it's very easy to end up in a huge muddle with uid/gid mapping issues 17:34:02 <LordAro> that too 17:34:07 <TrueBrain> yeah ... you need to know what you are doing, that is for sure 🙂 17:34:25 <TrueBrain> but yeah, if you are talking about ceph, you are talking about self-hosted, and that is just ... shitty in 2023 17:34:34 <LordAro> trying to route AD auth through it all has also proven difficult 17:34:41 <LordAro> TrueBrain: aerospace, got to be on-site 17:34:46 <TrueBrain> when I was security consultant I did one too many talks about explaining to companies that self-hosting is terrible for security 17:35:19 <LordAro> are they available anywhere? :D 17:35:36 <TrueBrain> haha, I hope not 😄 17:35:59 <LordAro> got to run, thanks for some interesting ideas 17:36:06 <LordAro> i don't have any time to do anything with any of them, but even so 17:36:07 <TrueBrain> haha, enjoy running 😛 17:36:32 <TrueBrain> (and he just got home!) 17:37:49 <LordAro> cinema time! 17:37:56 <TrueBrain> w00p! 17:39:19 <TrueBrain> lol, reading random tt-forums topic (yes, I know), about issues connecting to the coordinator .. I see we should make our logs a tiny bit better 17:39:30 <TrueBrain> when you only have IPv4 17:39:35 <TrueBrain> it tells you connections over IPv6 fail 17:39:38 <TrueBrain> (to be expected) 17:39:44 <TrueBrain> but it doesn't mentions it was also trying IPv4 🙂 17:40:03 <TrueBrain> (the IPv6 fails with an error, and in this case the IPv4 timed out) 17:40:22 <TrueBrain> so the IPv6 error is printed verbose, and the timeout is handled more graceful .. hihi 17:40:29 <TrueBrain> making logs make sense in most cases is hard! 17:43:00 <DorpsGek> [OpenTTD/bananas-api] TrueBrain updated pull request #373: Fix: bring the image size limit of PIL in line https://github.com/OpenTTD/bananas-api/pull/373 17:44:00 <TrueBrain> finally I can now test that PR myself, without setting up a local setup 😄 17:47:06 <TrueBrain> lol, I need to assign more memory to this preview instance .. can't even attempt a 10kx10k map 😛 17:47:22 <TrueBrain> something for another day 🙂 18:11:56 *** Flygon has joined #openttd 18:35:12 <TrueBrain> wauw, analyzing a 10kx10k heightmap is ... taking a bit of time. Why did we want to allow such big maps again? 18:35:51 <Rubidium_> because maybe JGRPP supports them? 18:35:59 <Rubidium_> as in, maps of that size 18:36:25 <TrueBrain> but it is just completely unrealistic to process such large heightmaps on the backend 18:37:08 <discord_user_03329cf> Ngl i wanna try 16k x 16k 18:37:23 <discord_user_03329cf> Or at least 8k x 8k 18:37:49 <discord_user_03329cf> 4x4 feels too small to have continents 18:38:43 <TrueBrain> 16kx16k consumes ~1.2GB of RAM on the backend .. and takes over 30s to process .. lol. I wonder if the image actually needs to be completely in memory .. PIL does that, but does it have to? Hmm .. 18:39:08 <JGR> 16k x 16k is not really usefully playable 18:39:31 <discord_user_03329cf> Maybe 8x8 then 18:40:01 <discord_user_03329cf> Made up of stitched together klutz maps 18:40:48 <DorpsGek> [OpenTTD/bananas-api] TrueBrain dismissed a review for pull request #373: Fix: bring the image size limit of PIL in line https://github.com/OpenTTD/bananas-api/pull/373#pullrequestreview-1447234447 18:41:54 <DorpsGek> [OpenTTD/OpenTTD] DorpsGek pushed 1 commits to master https://github.com/OpenTTD/OpenTTD/commit/aae8f40b9fa9f5eebf006940ddc2ba65b5243bee 18:41:55 <DorpsGek> - Update: Translations from eints (by translators) 18:42:45 <DorpsGek> [OpenTTD/bananas-api] TrueBrain commented on pull request #373: Fix: bring the image size limit of PIL in line https://github.com/OpenTTD/bananas-api/pull/373#issuecomment-1593557620 18:42:59 <TrueBrain> even requesting the image size via PIL triggers the OOM 18:42:59 <TrueBrain> lol 18:44:50 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 updated pull request #10733: Codechange: use standard int types https://github.com/OpenTTD/OpenTTD/pull/10733 18:45:51 <TrueBrain> so `open` is suppose to be lazy .. this is weird 🙂 18:46:15 <TrueBrain> owh, md5sum is calculated with an `fp.read()` 18:46:20 <TrueBrain> yeah .. that is already not smart 🙂 18:46:39 <TrueBrain> should be chunk-based ofc 18:47:11 <TrueBrain> but a 8kx8k image is just 1MB, so that shouldn't actually be a problem .. well, this will be interesting 18:52:11 <DorpsGek> [OpenTTD/bananas-api] TrueBrain commented on pull request #373: Fix: bring the image size limit of PIL in line https://github.com/OpenTTD/bananas-api/pull/373#pullrequestreview-1482205339 18:55:06 <Rubidium_> TrueBrain: how hard would it be to determine the highest difference in orthoganally adjacent pixels in your classifier script? I wonder whether the big heightmaps still have quite high differences between pixels, so they can't be shown correctly. Going even more pixels will only make that problem worse, so I'm not sure that >4kx4k is really necessary 18:55:29 <Rubidium_> since the majority of the information is going to be thrown away anyway to fit within the limits of our maps 18:55:54 <TrueBrain> I really do not know 18:56:12 <TrueBrain> I don't like backends being the limitation of systems; that I do know 🙂 18:57:28 <TrueBrain> we have some 8kx8k uploads on BaNaNaS currently 18:58:17 <TrueBrain> the only thing I can imagine, that people would like a heightmap to have 1-on-1 resolution with the biggest map you can create with JGRPP 18:59:11 <TrueBrain> (which brings us to 256M tiles) 19:00:36 <petern> Urgh, compilation errors that don't show you the line number... 19:04:51 <TrueBrain> meh; Pillow really just loads the image in memory in full 19:04:57 <TrueBrain> such a waste .. I just want to see the bytes once .. 19:07:00 <Ahyangyi> Yeah, if 16k maps cause OOM, I'll give up and submit 8k heightmaps instead 19:07:16 <TrueBrain> 8k even cause OOM 😛 I just need to look into this 🙂 19:07:22 <Ahyangyi> Weird 19:07:27 <TrueBrain> not really 19:07:35 <TrueBrain> we never really used to read the content of heightmaps; we added that recently 19:07:47 <TrueBrain> and .. well .. not many people want to upload very large maps 😄 19:08:27 <discord_user_03329cf> Oom? 19:08:40 <Ahyangyi> discord_user_03329cf: shorthand for "out of memory" 19:09:08 <discord_user_03329cf> Aaa 19:13:51 <TrueBrain> Ahyangyi: I think a library like `pyvips` instead of Pillow might really help 19:13:57 <TrueBrain> it explicitly doesn't keep the whole image in memory 19:15:52 <TrueBrain> https://github.com/libvips/libvips/wiki/Speed-and-memory-use suggests a difference of 9x memory usage 😄 19:17:50 <glx[d]> nml could maybe use that too 19:19:46 <TrueBrain> it doesn't have the best documentation, but that happens a lot with these kind of libraries 🙂 19:20:28 <andythenorth> libvips eh 19:20:51 <andythenorth> I did check recently, Pillow isn't as slow as I expected for a lot of common tasks 19:20:56 <andythenorth> I assumed 'eh python is slow' 19:21:05 *** Wolf01 has joined #openttd 19:21:06 <andythenorth> but some benchmarks suggested Pillow isn't 19:21:22 <TrueBrain> speed I did not look into .. memory footprint however is insane 🙂 19:21:27 <TrueBrain> I can't find any way to stream an image 19:21:33 <TrueBrain> although most image backends seem to support it just fine 19:21:42 <TrueBrain> but at some point, all pixels are loaded in a single buffer, it seems 19:22:18 <andythenorth> Horse image generation is now nearly 5s 19:22:23 <andythenorth> and that's using an MP pool 19:23:16 <andythenorth> 24s single threaded 19:23:38 <andythenorth> 2k LOC in my image processor though 😛 19:23:42 <andythenorth> not sure I want to convert 19:25:08 <Ahyangyi> libvips does come with a historgram function 19:25:25 <Ahyangyi> which seems to be the main thing we do with the heightmap in bananas-api? 19:25:31 <TrueBrain> Pillow has one too .. it was rubbish 😛 19:25:41 <TrueBrain> main thing is that we convert RGB in a certain way 19:26:09 <TrueBrain> which is one of the standard ways to convert RGB to a single colour btw 19:26:15 <TrueBrain> so "it depends" 😄 19:26:19 <DorpsGek> [OpenTTD/OpenTTD] JGRennison opened issue #11016: [Bug]: Use after free in network invalid packet error path https://github.com/OpenTTD/OpenTTD/issues/11016 19:26:29 <Ahyangyi> I see, that's the hard part (when memory is a concern) 19:26:51 <TrueBrain> JGR: lol .. we fixed more than one bug that looks very similar to what you describe .. the gift that keeps on giving 😄 19:27:03 <TrueBrain> Ahyangyi: well, we just need a stream of pixel-data, honestly 19:27:20 <TrueBrain> it can even be in any order, as long as we have seen all pixels in the end 😛 19:28:08 <andythenorth> wonder if anyone could port my Pillow code 😛 19:28:18 <andythenorth> it's probably just method and parameter replacement 19:28:38 <andythenorth> at first glance, libvips appears to have the majority of methods I'm using 19:28:39 <TrueBrain> pyvips' documentation is not really telling me how to use it ... lol 19:28:59 <andythenorth> it's more obvious if you've done 10 years of PIL 😛 19:29:14 <andythenorth> it probably has an image stream of raw pixel data 19:29:21 <andythenorth> and an Image wrapper around that 19:29:29 <andythenorth> with save / load etc 19:29:35 <andythenorth> it has crop, insert, transpose, rotate 19:29:38 <andythenorth> colour map 19:29:47 <andythenorth> it has a point reader 19:29:54 <andythenorth> it has draw tools, but they come with a performance warning 19:30:14 <TrueBrain> okay, the SourceCustom we most likely need to wrap around our `fp` .. so that at least is possible .. I found out by an example of theirs, ironically 🙂 19:35:36 <TrueBrain> guess one could always just save the image as BMP and use the TargetCustom to capture data .. but I am sure there is some other function to call .. haven't found it yet 😄 19:36:05 <TrueBrain> anyway, I was going to watch a movie .. Ahyangyi if you feel up for looking into this, would be appreciated. Otherwise I will toy with this a bit soon .. it has to be possible to not use an absurd amount of memory to just read through an image once 🙂 19:36:17 <TrueBrain> (and yes, the backend is memory-limited .. that is just what it is 😄 ) 19:38:11 <TrueBrain> I also wonder if we actually check the upload is in a format we support in OpenTTD .. like not that someone uploads a TIFF or PDF 😛 19:40:55 <TrueBrain> seems you need to use `crop` to get smaller sections of an image to avoid memory usage 19:45:23 <glx[d]> multiple load+crop ? 19:47:16 <TrueBrain> Seems crop returns a new object.. dunno, needs testing 🙂 19:51:00 <TrueBrain> Owh, sink screen seems promising too, but doesn't seem to have a py wrapper .. 20:02:24 <petern> Hmm 20:02:25 *** gelignite has quit IRC 20:03:59 <petern> Weird, 1) ZeroedMemoryAllocator doesn't seem to have any effect with std::make_shared, and 2) the std::string constructor of GRFConfig doesn't even invoke ZeroedMemoryAllocator() anyway, so how does that work in master... 20:05:35 <petern> Although I'm happy to not use ZeroedMemoryAllocator anyway, as it's bad form when structs contain stl things. 20:07:04 <Rubidium_> yeah, eventually ZeroedMemoryAllocator should go away 20:08:20 <JGR> The point of std::make_shared is to avoid calling the allocator for the new object 20:08:32 <Rubidium_> and ZeroedMemoryAllocator allocates (new) zeroed memory. It doesn't have a constructor zero-ing stuff 20:08:37 <JGR> So that the control block and the data can go in the same allocation 20:09:17 <petern> Ah ha, I see. 20:10:06 <petern> So the fact that the copy constructor DOES include a call to ZeroedMemoryAllocator() is a red herring, and either isn't necessary, or is there to shut up a compiler warning. 20:10:38 <petern> I wasn't sure if make_shared called new behind the scenes, or if that's all implementation specific... 20:10:50 <JGR> It calls placement new 20:11:36 <petern> Fun. 20:11:46 <Rubidium_> petern: yes, that seems to be a red herring 20:12:33 <TrueBrain> Some more googling .. vips has regions which does exactly what we want / need .. it reads a region of pixels .. so seems vips can help reduce memory 🙂 20:14:49 <DorpsGek> [OpenTTD/OpenTTD] rubidium42 merged pull request #11014: Codechange: remove a number of unneeded c_str() calls https://github.com/OpenTTD/OpenTTD/pull/11014 20:55:49 *** HerzogDeXtEr has quit IRC 20:56:41 *** Wolf01 has quit IRC 20:56:44 <LordAro> cinema was broken, so we went bowling instead and went to the arcade 20:56:58 <LordAro> i won enough tickets for 7 mini bags of haribo 20:58:05 <TrueBrain> End the night with a sugar high? 😄 21:00:11 <petern> Desserts are like that :/ 21:03:12 <petern> Nice, removing ZeroedMemoryAllocator from certain places, and that means alloc_func.hpp is no longer needed... 21:03:17 <petern> At least, no longer directly needed. 21:03:55 <petern> Include hell :/ 21:06:12 <petern> alloc_type.hpp rather. 21:22:07 *** keikoz has quit IRC 22:15:35 *** _aD has quit IRC 22:52:49 <DorpsGek> [OpenTTD/OpenTTD] zephyris opened pull request #11017: Fix #10838 https://github.com/OpenTTD/OpenTTD/pull/11017 23:12:52 *** nielsm has quit IRC 23:17:39 *** Eddi|zuHause has quit IRC 23:18:31 *** Eddi|zuHause has joined #openttd 23:23:04 *** Eddi|zuHause has quit IRC 23:23:46 *** Eddi|zuHause has joined #openttd 23:33:00 <petern> That's a terrible PR title :p 23:41:02 *** D-HUND is now known as debdog 23:42:13 <petern> How did 00:42 happen... oops.