This version (2017/05/27 13:44) is a draft.
Approvals: 0/1

[02:25:58] *** ChanServ sets mode: +o temporal_

[08:18:10] <m2k> hello there. I'm wondering if there's a limit of active async calls from RedisClient. I'm making a lot (thousands) of RedisClient calls out of a java loop.

[09:12:49] <temporal_> m2k as far as I know the only limit should be the connection to Redis

[09:13:20] <temporal_> m2k perhaps you want to pace the requests ?

[09:13:38] <temporal_> is it what you mean ?

[09:17:19] <temporal_> m2k looking at it, it looks like eerything you will do will be sent to redis on the connection

[09:18:07] <temporal_> perhaps you want to batch your commands

[09:18:25] <temporal_> and use “ping” command to pace the batches

[09:18:38] <temporal_> for example you send 500 commands

[09:18:38] <temporal_> then a ping command

[09:18:44] <temporal_> when you get the ping result you send another batch

[15:51:28] <joem86> So far vert.x has been really fun to work with. I am wondering though, I have several verticles that need to stream bytes to separate UDP endpoints, and one verticle that populates those bytes sent in over UDP. My first approach had each packet published on the event bus, and the repeater verticles subscribing to that event, but that strikes me as inefficient. Could I somehow share an in-memory byte buffer among several verticles?

[15:53:38] <cescoffier> joem86 : you can use a distributed map to share this data among verticles

[15:53:47] <cescoffier> vertx.sharedData()

[15:56:21] <joem86> hmmmm, since the data is mutable (a Buffer), wouldn't the entire buffer need to be copied for every verticle each time data is added?

[15:58:23] <cescoffier> shared data should not be used for mutable data

[16:00:05] <joem86> Gotcha. So another idea would be to write the data to a file in the filesystem, and have each of the verticles constantly read data from that file.

[16:06:53] <temporal_> joem86 it depends if you have lot of data or not!

[16:07:45] <temporal_> in this kind of situation, some would use memory mapped files

[16:10:35] <joem86> hmmmm, sounds close to what I have in mind. The main concern is the real-time repeating of the data, so there's no other reason to store it anywhere other than to give other verticles access to it.

[16:11:30] <Sticky> I believe java chronical has some stuff for sharing data via memory mapped files

[16:11:36] <joem86> So theoretically I could have a memory-mapped-file that is of constant size (think circular buffer) that each verticle is “pumping” out to its UDP endpoint

[16:23:05] <joem86> Chronicle Queue (recommended by the java chronicle github page) seems interesting, but it sounds like it's trying to solve the same problem as the event bus.

[17:14:37] <tsegismont> joem86, if there is a verticle which prepares udp packets already, why not send them straight from it?

[17:28:11] <joem86> tsegismont, there are multiple endpoints to send to, so the idea was to have one verticle to receive the packets and another to send to a listener. If I placed the responsibility for receiving/broadcasting into a single verticle it would be single-threaded, and performance would be impacted based on the number of listeners.

[17:28:44] <joem86> Does that make sense? Am I misunderstanding something?

[17:33:39] <tsegismont> It may make sense to split work between verticles. My (personal) feeling is to do something like you describe in a single verticle if that simplfies the impl. And then check if your performance goals are met.

[17:33:50] <tsegismont> If they are, no need to make things more complex

[17:33:51] <tsegismont> :)

[17:38:01] <joem86> Sure. I'm actually migrating an app written in node.js that currently is single-threaded. I saw an opportunity with vert.x to split up the work asynchronously, so the packets are always handled as fast as possible.

[17:39:01] <joem86> It may have been possible to do that in node, but I'm a Groovy/Grails developer, so writing a non-blocking event driven server in Groovy is very attractive to me.

[17:43:50] <tsegismont> joem86, I understand. My gut feeling is that publishing datagrams over the event bus is not the improvement you're looking for. Hard to say more without a deeper understanding of your use case.

[18:25:06] <joem86> Agree, the event bus sounds more like it's better suited for coordination among verticles instead of streaming raw bytes