Log for #openttdcoop.devzone on 14th June 2014:
Times are UTC Toggle Colours
00:16:46  *** yorick has quit IRC
01:01:42  *** gelignite has quit IRC
06:26:39  *** Supercheese has quit IRC
06:31:38  *** tycoondemon has joined #openttdcoop.devzone
06:34:09  *** phatmatt has joined #openttdcoop.devzone
06:34:39  *** tycoondemon2 has quit IRC
06:34:39  *** Guest12532 has quit IRC
06:35:08  *** phatmatt is now known as Guest13572
08:10:35  <DevZone> Project Finnish Trainset build #61-push: SUCCESS in 3 min 12 sec:
08:19:44  *** gelignite has joined #openttdcoop.devzone
08:22:05  <DevZone> Yippee, build fixed!
08:22:06  <DevZone> Project 2ccts build #152-push: FIXED in 1 min 23 sec:
11:18:06  *** frosch123 has joined #openttdcoop.devzone
11:35:32  <DevZone> Project Finnish Trainset build #62-push: FAILURE in 8.2 sec:
11:37:08  <DevZone> Project Finnish Trainset build #63-push: STILL FAILING in 12 sec:
11:43:20  <DevZone> Yippee, build fixed!
11:43:21  <DevZone> Project Finnish Trainset build #64-push: FIXED in 2 min 48 sec:
11:43:21  *** Alberth has joined #openttdcoop.devzone
12:12:29  *** yorick has joined #openttdcoop.devzone
13:08:08  <DevZone> Project Finnish Trainset build #65-push: SUCCESS in 2 min 52 sec:
13:08:15  <DevZone> Project Finnish Trainset build #66: FAILURE in 6.2 sec:
13:42:24  *** ODM has joined #openttdcoop.devzone
13:54:50  <DevZone> Project Finnish Trainset build #67: STILL FAILING in 6.4 sec:
13:55:36  <frosch123> what was again the right syntax for subrepositories to work both on devzone and clients?
13:56:31  <planetmaker> you mean in .hgsub ?
13:56:37  <frosch123> yes
13:56:54  <frosch123> finnishtrainset specifies them using http
13:56:57  <frosch123> but eints needs ssh
13:57:10  <frosch123> likely the same for hudson?
13:57:15  <planetmaker> you need to specify the ssh:// then, too
13:57:35  <planetmaker> you can define it multiple times
13:57:43  <planetmaker> ssh://   = ssh://
13:57:47  <planetmaker>   =
13:57:51  <planetmaker> graphics   = graphics
13:58:49  <planetmaker> hm, so eints needs the ssh:// one?
13:58:52  <planetmaker> I guess
13:59:14  <frosch123> yes
13:59:22  <frosch123>
13:59:27  <DevZone> Project Finnish Trainset build #68: STILL FAILING in 5.9 sec:
13:59:41  <planetmaker> but jenkins uses http:// for cloning, IIRC
14:00:03  <planetmaker> let's check...
14:00:13  <frosch123> iirc eints vm cannot use http
14:00:13  <Alberth> jenkins doesn't write changes :)
14:00:24  <frosch123> and usually it needs ssh anyway for committing
14:00:29  <frosch123> though technically not for subrepos
14:01:00  <planetmaker> jenkins - in the default template - also uses ssh:// to clone and pull
14:02:05  <DevZone> Project Finnish Trainset build #69-push: STILL FAILING in 14 sec:
14:02:12  <juzza1> hmm
14:03:08  <juzza1> so whats the correct one, if i want to have make-nml-juzza1-common as subrepo for finnishtrainset, and symlink its makefile to finnishtrainset root directory?
14:03:34  <planetmaker> you need all three
14:04:05  <planetmaker> sub repos are a bit sucky :)
14:04:28  <planetmaker> this is one of the rough edges of subrepos
14:06:17  <planetmaker> the relative path is for local operations
14:06:34  <planetmaker> the ssh for push/pull via ssh obviously. As is the http relation for those who pull via http
14:07:23  <planetmaker>
14:08:28  <planetmaker> with a different path on the server in principle the ssh and http [subpath] sections could be avoided... yet that needs more work on that end there...
14:10:05  <planetmaker> did I install xcf2png on devzone?
14:10:16  <juzza1> ok, ill try fixing the hgsub
14:10:18  <juzza1> yes :)
14:10:24  <planetmaker> ok :)
14:11:10  <planetmaker> xcftools 1.0.7 is it probably. Heck, I even documented it :D
14:14:42  <planetmaker> juzza1, I considered to promote using make-nml as sub-repo for projects. But I decided against
14:15:51  <planetmaker> mostly on grounds of adding the two-repo complexity for very little gain
14:20:27  <DevZone> Project Finnish Trainset build #70-push: STILL FAILING in 16 sec:
14:20:27  <juzza1> yeah, probably the best that way
14:21:41  <planetmaker> you still miss the http:// redirection
14:21:51  <juzza1> oh
14:21:58  <planetmaker> the /home/hg/... one is not needed
14:22:14  <planetmaker> as it's not used anywhere afaik. Hm... maybe in rhodecode
14:22:40  <juzza1> ok, copied those from zbase
14:23:14  <planetmaker> oh, I see :) zBase might have those for long(er)
14:23:26  <planetmaker> but opengfx-mars works without those
14:23:47  <DevZone> Project Finnish Trainset build #71-push: STILL FAILING in 15 sec:
14:24:16  <juzza1> works now, only some errors in the actual code
14:25:28  <planetmaker> k :)
14:30:56  <DevZone> Yippee, build fixed!
14:30:56  <DevZone> Project Finnish Trainset build #72-push: FIXED in 3 min 1 sec:
16:02:35  <DevZone> Project xussrset - Trains from Russia build #320-push: SUCCESS in 3 min 19 sec:
16:21:59  <DevZone> Project Japanese Buildings build #231-nightlies: SUCCESS in 19 sec:
16:34:01  <DevZone> Project Finnish Rail Infrastructure - Rails build #345-nightlies: SUCCESS in 8 min 20 sec:
16:41:23  <DevZone> Project road-hog build #231-nightlies: SUCCESS in 42 sec:
16:48:05  <DevZone> Project Dutch Trackset build #28-push: SUCCESS in 40 sec:
16:48:24  <DevZone> Project Iron Horse build #910-nightlies: SUCCESS in 1 min 44 sec:
16:49:46  <DevZone> Project 2ccts build #153-push: SUCCESS in 1 min 22 sec:
16:51:34  <DevZone> Project xussrset - Trains from Russia build #321-push: SUCCESS in 3 min 29 sec:
19:25:38  <frosch123> Alberth: no idea what you are heading for, but just in case
19:25:50  <frosch123> i started working on a grfcodec replacement
19:26:11  <planetmaker> o_O
19:26:22  <frosch123> some library which reads and writes grfs, in the hope that it can attach to various input files other than nfo
19:26:29  <frosch123> possible some intermediate nml input
19:26:42  <Alberth> I am not sure either
19:27:10  <Alberth> but profiling showed that nml time is spend in scanning + parsing
19:27:24  <Alberth> I don't see a nice way around that
19:27:46  <Alberth> partial compile would help, but you still have the same amount of nml code
19:27:54  <frosch123> partial compiles :) some intermediate object format which still has identifers
19:28:25  <Alberth> as would having a template mechanism behind the scanner + parser, instead of in cpp
19:28:51  <frosch123> how big are the nml files after cpp?
19:29:07  <planetmaker> just check bundles server. It has those nml files
19:29:21  <planetmaker> firs is 50k or so iirc
19:29:27  <Alberth> 13493660 May 31 11:07 firs.nml
19:29:43  <planetmaker> hm, lines of code :)
19:30:15  <frosch123> hmm, interesting. so nml is not the issue, but the way it is abused :p
19:30:31  <Alberth> planetmaker:  379983 <-- lines
19:30:37  <planetmaker> oh...
19:30:42  <planetmaker> got quite a bit bigger then
19:32:03  *** Supercheese has joined #openttdcoop.devzone
19:32:37  <Alberth>    just a mostly random collection of nml files, most are not up-to-date
19:33:46  <frosch123> yeah, then indeed it looks like we need multiple input files
19:34:12  <frosch123> or some "import" command instead of "include" which imports some precompiled thingie
19:35:01  <Alberth>  profile of firs compile
19:35:15  <planetmaker> <-- not so random size distribution of nml files
19:37:07  <frosch123> Alberth: so, if we just dump the parser tree from the yacc output into some intermediate file, which can be imported by other modules?
19:37:25  <frosch123> "just" :)
19:37:43  <Alberth> I considered writing a lex specification
19:38:08  <Alberth> it would speed things up considerably, at the cost of having a piece of C code running
19:38:23  <Alberth> which makes deployment of nml a big issue
19:38:41  <planetmaker> wouldn't make it a bigger one than OpenTTD deployment
19:39:01  <planetmaker> or would it?
19:39:27  <frosch123> would your nml windows binary be able to link it?
19:39:47  <Alberth> depends on how you use it, as stand-alone executable, or as subroutine in Python
19:40:13  <frosch123> what would be the point of stand-alone?
19:40:29  <frosch123> then it can only dump to a file, which needs reading again
19:40:51  <frosch123> i don't think reading that one would be much faster
19:40:52  <Alberth> push it through a pipe :)
19:41:31  <frosch123> i mean you need a format that is easier lexed than pure nml :p
19:41:37  <Alberth> ply/  works with RE on a (for firs) 13MB text string
19:41:44  <frosch123> which still contains all info like line information
19:41:50  <Alberth> json?
19:42:21  <frosch123> hmm, you mean because python has a built-in parser for that?
19:42:51  <Alberth> I don't know if it is coded in C or in python itself
19:43:31  <Alberth> but the cuplrit is probably RE
19:44:34  <Alberth> but a stand-alone exe has to be run from python as child and get connected to a pipe
19:44:47  <Alberth> (aside from questions about speed gains)
19:45:29  <planetmaker> do pipes work on windoze?
19:45:38  <Alberth> a python subroutine is potentially much easier, but you have to build with whatever compiler the Python interpreter is built
19:46:12  <Alberth> iirc there are automagic solutions for that, at least for unix
19:46:26  <Alberth> no idea how that ports to windoze
19:47:57  <Alberth> from a performance point of view, it may be easiest if we do lex+yacc in C, and the C functions then call the original Python functions that are now in the nml parser
19:48:11  <Alberth> ply is yacc, so the routines should just work
19:48:38  <Alberth> it's just a matter of getting the arguments correct
19:48:38  <planetmaker> mercurial itself has also some binary part(s), but I dunno which. A hybrid solution might be the way to go...
19:48:51  <planetmaker> ... but more painful to maintain, I guess
19:49:24  <Alberth> not really, scanner and parser are small and simple in structure
19:49:48  <frosch123> planetmaker: well, we can still drop win platform and go for the remote compile service :)
19:50:07  <planetmaker> yes, we could indeed
19:50:11  <Alberth> nml syntax is very generic, almost everything is an expression, the actual cases are all handled in the type checking code after parsing
19:50:31  <frosch123> for the smaller projects plain python is fast enough
19:50:50  <planetmaker> it basically even exists...
19:51:19  <planetmaker>
19:51:29  <Alberth> it's called "hg push" :p
19:51:51  <planetmaker> it has awesome many security issues so far. Thus I can't really make it public as-is
20:18:36  *** oskari89 has joined #openttdcoop.devzone
20:30:25  <Alberth> can we conclude anything here? :)
20:38:38  <planetmaker> tja, good question :)
20:51:12  <planetmaker> Alberth, maybe we can conclude that any which makes nml faster would be highly useful.
20:52:01  <Alberth> sure, like the UN :)
20:52:26  <Alberth> lex+yacc in c++ looks doable to me, in unix
20:52:36  <Alberth> euhm, in C, probably
20:53:25  <Alberth> but I fear the windows + python3 + windows build for python3 stuff will not
20:54:03  <Alberth> it's probably best for performance
20:54:58  <Alberth> another option is to make a stand-alone executable
20:55:13  <Alberth> scanner only is simplest
20:55:35  <Alberth> but the question is what you gain
20:56:08  <Alberth> building is probably simpler, as it's a normal binary
20:58:21  <planetmaker> I feel somewhat out of the bounds of my knowledge to decide this question... I don't have any expertise there to judge a path
20:58:43  <planetmaker> I would not mind so much a 2-part solution if it helps us on
20:58:57  <planetmaker> maybe it could even be combined with frosch's backend. Dunno
20:59:37  <Alberth> could be, but until now it's not the bottleneck, probably due to the graphics caching
20:59:47  *** ODM has quit IRC
21:00:07  <Alberth> dropping that would perhaps be useful at some point in time :)
21:00:24  <Alberth> building a scanner as executable is quite trivial
21:00:36  <Alberth> we could start with that
21:12:30  *** Alberth has left #openttdcoop.devzone
22:15:39  *** oskari89 has quit IRC
22:26:50  *** frosch123 has quit IRC
23:58:35  *** yorick has quit IRC

Powered by YARRSTE version: svn-trunk