[03:54:25] * ChanServ sets mode: +o temporalfox [08:43:21] * ChanServ sets mode: +o purplefox
[09:21:12] *** ChanServ sets mode: +o purplefox
[10:13:30] <purplefox> cescoffier: temporalfox: morning
[10:13:37] <cescoffier> good morning
[10:13:38] <temporalfox> hi purplefox
[10:13:42] <temporalfox> hi everyone
[10:48:30] <s0va> purplefox: i'm debugging strange vertx-hazelcast related memory leak
[10:49:42] <s0va> i've wrote an email on mail on this topic, but didn't receive any useful reply
[10:50:38] <s0va> here's a situation: the same code works okay on single-node vertx…
[10:51:11] <s0va> on clustered version it leaks memory, heap gets filled in production in circa two days.
[10:51:19] <s0va> http://shrani.si/f/3m/A3/36TPQAHG/vertx-memleak-heap-compa.jpg
[10:56:25] <s0va> here's a yourkit snapshot screenshot
[10:56:53] <s0va> here's the deal: when websocket client connects, 3-5 event bus consumers are registered
[10:57:11] <s0va> … when client disconnects, consumers are unregistered
[10:57:39] <s0va> heap is filled with lots of hazelcast choosable sets instances
[10:57:57] <s0va> what am i doing wrong=
[11:05:17] <purplefox> s0va: do you have a link to the mail?
[11:11:26] <s0va> just a sec
[11:12:13] <s0va> https://groups.google.com/forum/?fromgroups#!searchin/vertx/memory$20leak/vertx/zzfleuzdCHo/nnEl0q3PELoJ
[11:12:25] <s0va> that was for version 2.1.6
[11:12:42] <s0va> as my last resort i ported app to 3.0, but nothing changed
[11:13:05] <s0va> i can also provide heap dumps
[11:13:09] <s0va> or even code
[11:56:27] <purplefox> s0va: it should be fairly straightforward in your profiler to see which object(s) are holding on to the large number of objects in your report
[11:57:09] <purplefox> s0va: I'd recommend finding out what is holding on to io.vertx.core.impl.ConcurrentHashSet
[11:57:20] <purplefox> and tracing back through references
[11:57:34] <purplefox> s0va: that should be pretty easy with any modern profiler
[11:58:09] <purplefox> s0va: looks like you're using yourkit.. this is very easy
[12:02:02] <Sticky> would have thought it is this one: http://grepcode.com/file/repo1.maven.org/maven2/io.vertx/vertx-hazelcast/2.1RC3/org/vertx/java/spi/cluster/impl/hazelcast/HazelcastAsyncMultiMap.java#57
[12:02:22] <purplefox> is that a guess?
[12:02:29] <Sticky> totally
[12:02:47] <purplefox> I'd rather see the data from the profiler. That class is used in several places :)
[12:03:02] <purplefox> you may well be right, but I don't like guess work
[12:03:38] <Sticky> yeah its just based on there being a single instance of HazelcastAsyncMultiMap but a retained set size the same as the possible “leak”
[12:11:21] <purplefox> Sticky: if that's what's holding on to things then it's likely there are simply a lot of event bus handlers that have been registered by the user
[12:15:45] <Sticky> somewhere around 2million
[12:18:39] <purplefox> s0va: Sticky: that, in itself is not a huge number, but it depends whether it was desired or not. I don't see any evidence of a leak as such. I suggest s0va looks through your code for places where you register event bus handlers but don't remove them later.
[12:21:26] <purplefox> s0va: some basic techniques for tracking down this kind of thing - global search through your code for where you register handlers, and every time you do increment a global counter. when you unregister them decrement your global counter. then have a periodic task that displays the count.
[12:25:03] <purplefox> Sticky: having said that I don't see anything that looks like handlers in the report, i guess they could be on other nodes of the cluster
[12:25:20] <purplefox> s0va: how many nodes in the cluster here? do your other nodes register a lot of handlers?
[12:28:41] <purplefox> s0va: also, one thing to check - may sure you're using the correct version of hazelcast for the vert.x release, using a different version might break the cluster - i know previous versions had bugs in the multimap implementation
[12:31:03] <Sticky> oh, hmmm, this is a websocket app, another thing to look at, does what is on the end of that websocket register lots of listners?
[12:36:41] <purplefox> Sticky: shouldn't do. at least, not _clustered_ handlers
[13:02:38] <s0va> purplefox: 10x for hint, will check out
[13:02:39] <s0va> purplefox: only 3 nodes
[14:02:54] <s0va> purplefox: is this normal: http://pastebin.com/TfGhQ69Y
[15:49:49] <aetherius> hi. using the HttpClient from vert.x i'm trying to connect to an https host. using ssl set to true and trusting all certificates (generally unsafe, but it's just a toy example) i still get following exception: javax.net.ssl.SSLHandshakeException: Failed to create SSL connection. does anyone know how to fix this “problem”? do i have to install specific certificates?
[15:51:43] <Sticky> well debugging ssl is a bitch
[15:52:03] <Sticky> that error dosnt give us much to go on
[15:52:08] <aetherius> yep :(
[15:52:27] <Sticky> try turning on -Djavax.net.debug=all
[15:52:50] <Sticky> may help a bit, but yeah, javas ssl warnings suck
[15:53:27] <aetherius> hm ok. i found some general information overriding the internal trustmanagers and hence accept all certificates, but with no luck/success
[15:55:18] <Sticky> testing with the commandline thing of “openssl s_client -connect 127.0.0.1:443” can sometimes give more info too
[15:56:49] <aetherius> ok. will try that as well .. (the first hint didn't gave much insight … yet)
[16:22:34] <aetherius> hmm is it possible to create exception with a higher level of verbosity?
[16:22:42] <aetherius> *exceptions
[16:29:12] <aetherius> solution is / was simple, and i didn't thought of that: the port parameter, the first in client.getNow() is important and needs to be set as 443 (https default port) .. well ..
[16:29:21] <aetherius> thanks for the help and hints!
[16:29:44] <s0va> purplefox: i'm looking at memory snapshots in yourkit
[16:30:07] <s0va> purplefox: everything traces back to vertx-hazelcast.
[16:30:35] <s0va> … if i deploy all verticles in the same vertx instance and disable clustering, memory leak does not occur. which is really weird
[16:30:49] <s0va> used vertx: 3.0, used hazelcast: 3.5
[16:36:11] <aetherius> good bye
[16:45:39] <s0va> purplefox: here are two memory snapshots (first one taken shortly after vertx app startup, another after some time)
[17:27:49] <melvinross> so i've been using vertx without grade or maven for doing test. I'm trying to figure out how to include modules without using either. Do i just need to build the projects as fat jars and put them on the class path?
[17:41:28] <s0va> melvinross: yep
[17:41:43] <s0va> or put all jars to classpath
[17:43:58] <melvinross> thanks
[17:56:00] <melvinross> i ended up just putting the project into maven
[17:56:42] <Sticky> yeah working without some build or dependency management tool is just a nightmare
[18:11:13] <melvinross> it wasn't by choice
[18:13:18] <melvinross> i'm fighting a bit of a FUD war
[18:36:20] <melvinross> has anyone tried to use the kafka verticle from cyanogen?
[19:59:10] <melvinross> i'm attempting to deploy a vertices from JS
[19:59:43] <melvinross> vertx.deployVerticle(“com.cyngn.kafka.MessageProducer”,kafkaConfig, kafkaComplete);
[20:00:06] <melvinross> but the deployed vertical gets no information when it calls config()
[20:00:48] <melvinross> just an empty object, so it looks like the data isn't actually being passed
[20:27:52] <BadApe> melvinross, sorry i am using java only, but it is very quiet in here
[20:50:04] <BadApe> TSharedRef addr = ISocketSubsystem::Get(PLATFORM_SOCKETSUBSYSTEM)→CreateInternetAddr(); has TSharedRef been deprecated
[20:58:53] <BadApe> oh maybe it is ISocketSubsystem that no longer has SetIp
[21:03:24] <BadApe> oops wrong place
[22:52:30] <bytor99999> melvinross will probably miss this. But in Vert.x 3 you have to put in kafkaConfig, a property in the root called “config” then you can put your values under that property.