[09:00:06] * ChanServ sets mode: +o temporalfox [09:04:19] <magabiztos> hi all! [09:18:15] <temporalfox> hi ppatiern [09:18:27] <ppatiern> hi temporalfox ! [09:19:19] <temporalfox> so I can have a look a bit this morning at the polyglotisation of mqtt [09:19:43] <temporalfox> I would like that you tell me the scope of this component and its role [09:19:58] <temporalfox> i.e define what it does and what it does not [09:20:12] <ppatiern> ok [09:20:36] <ppatiern> the main objective is to handle communication with remote MQTT clients [09:20:59] <temporalfox> I see it is named mqtt-server now [09:21:05] <ppatiern> it means that the MQTT server is able to handle all the MQTT messages defined by the spec (MQTT 3.1.1) like CONNECT, PUBLISH and so on [09:21:10] <temporalfox> does a “regular” server handle more than this ? [09:21:34] <ppatiern> my mqtt-server component ISN'T a full MQTT broker [09:21:52] <temporalfox> so it's called server now [09:22:00] <ppatiern> yes [09:22:07] <temporalfox> because it's not a broker [09:22:12] <ppatiern> just because as a server it can handle communication with clients [09:22:13] <ppatiern> yes [09:22:17] <ppatiern> it's not a broker [09:22:18] <temporalfox> so this server an be used to implement a broker [09:22:29] <ppatiern> yes … it can be used to do that [09:22:34] <temporalfox> so what a broker would do in addition of being a server ? [09:22:55] <ppatiern> being a full broker means to implement all the MQTT spec [09:22:57] <ppatiern> for example … [09:23:17] <temporalfox> so more messages ? [09:23:25] <ppatiern> nope [09:23:29] <ppatiern> let me try with an example [09:24:16] <ppatiern> will testment : when a client CONNECT to a broker it sends information about a “will” message that should be send when the client goes offline due to a brute disconnection. The “will” information are in the CONNECT message. The broker needs to store this information in some way and then publish the message when needed (as I said before) [09:24:17] <temporalfox> ok [09:24:38] <temporalfox> so persistence ? [09:24:46] <ppatiern> for example [09:24:51] <ppatiern> my mqtt-server just provide the way to an application on top of it to know about the CONNECT message is received and doing what it needs to do [09:25:07] <ppatiern> a good example is protocol translation for example [09:25:36] <temporalfox> so it means that with an mqtt-broker, someone wanting to do protocol translation needs to be an client of the mqtt-broker [09:25:42] <ppatiern> using the MQTT server to receive connections from MQTT clients … accepting the CONNECT message .. extract the will … and translating it on top of another protocol for example [09:25:47] <temporalfox> and in this case it's simpler, it's an handler of the mqtt-server [09:25:54] <ppatiern> yes [09:26:03] <temporalfox> so the use cases are [09:26:41] <ppatiern> yes absolutely [09:26:50] <ppatiern> these are the main use cases I see [09:26:52] <temporalfox> with the current vertx stack, what interactions could be done ? [09:27:05] <temporalfox> like we will have soon kafka client [09:27:09] <temporalfox> we have amqp support [09:27:11] <temporalfox> rabiit mq [09:27:26] <temporalfox> using these with the server would make somehow sense ? [09:27:32] <temporalfox> or just the event bus [09:27:33] <temporalfox> too [09:27:37] <ppatiern> you could use for example Vert.x Proton (AMQP) for building a custom logic on top of the mqtt-server and bridging mqtt to amqp [09:27:46] <ppatiern> absolutely even the event bus for example [09:28:09] <ppatiern> the same for kafka for example [09:28:17] <temporalfox> ok [09:28:24] <ppatiern> you could decide that a feature like the “will” message become a message to store into kafka [09:28:26] <ppatiern> for example [09:28:33] <temporalfox> ok [09:28:56] <temporalfox> looks enough for now [09:28:59] <ppatiern> it just provides the underlying communication with MQTT clients … exposing fluent API for handling received messages and sending new ones [09:29:02] <temporalfox> i.e to understand the role [09:29:17] <temporalfox> it could also use websockets ? [09:29:21] <temporalfox> I'm guessing [09:29:35] <ppatiern> for communication with MQTT clients you mean ? [09:29:47] <temporalfox> for relaying messages to other servers [09:29:53] <ppatiern> oh ues [09:29:55] <ppatiern> yes [09:29:55] <temporalfox> does aeron makes sense too ? [09:30:07] <temporalfox> I've been doing an aeron client recently [09:30:14] <temporalfox> for vertx [09:30:32] <ppatiern> I have never used aeron [09:31:17] <temporalfox> it's an IPC layer [09:31:18] <temporalfox> that can use multicast [09:31:18] <ppatiern> but the main thing is that you can handle incoming messages … you of course need to reply in the correct way (as MQTT spec says, i.e. CONNECT received … CONNACK to send) but in the middle you can do whatever you want [09:31:21] <temporalfox> too [09:31:25] <temporalfox> using UDP mainly [09:31:37] <ppatiern> on top of my head I don't see limitations [09:32:02] <ppatiern> it's important that receiving MQTT messages you reply in the “correct” way (as I said i.e. receive CONNECT, need to send CONNACK) [09:32:11] <temporalfox> yes [09:32:13] <ppatiern> and between them you can put the logic you want in your application [09:32:17] <temporalfox> obviously [09:32:35] <temporalfox> now I see 15 tests [09:32:38] <temporalfox> they run [09:32:47] <temporalfox> one fails but it's a base abstract class [09:32:57] <ppatiern> one fail ? [09:32:58] <temporalfox> I'm going to spend quickly some time today making it polyglot [09:33:03] <temporalfox> initializationError(enmasse.mqtt.test.MqttBaseTest): No runnable methods [09:33:17] <ppatiern> oh yes [09:33:19] <ppatiern> right [09:33:29] <ppatiern> this is a base for the other test classes [09:33:34] <ppatiern> I'll check it [09:33:52] <temporalfox> it should be abstract class probably [09:33:57] <ppatiern> yes [09:34:12] <ppatiern> thanks [09:34:17] <temporalfox> so I'm going to do this polyglot thing [09:34:21] <temporalfox> because it's quite important for being in vertx stack [09:34:28] <ppatiern> I understand [09:34:47] <ppatiern> I'll continue to work on that on MQTT message handling side [09:34:56] <temporalfox> ok [09:35:06] <temporalfox> I hope we won't have much conflicts [09:35:31] <ppatiern> I hope so [09:36:49] <ppatiern> where I can read something about how to make polyglot a Vert.x component (just for the sake of curiosity) [09:38:00] <temporalfox> this document does not exist yet [09:38:18] <temporalfox> we do have this doc [09:38:18] <temporalfox> https://github.com/vert-x3/vertx-codegen/blob/master/src/main/asciidoc/java/index.adoc [09:38:22] <temporalfox> about code generation in general [09:38:25] <temporalfox> it will give you an idea [09:38:35] <temporalfox> I plan to write such doc though [09:38:39] <temporalfox> along with the netty guide [09:38:47] <temporalfox> you already had [09:39:03] <ppatiern> great thanks [09:39:29] <temporalfox> ppatiern one issue with netty mqtt codec [09:39:35] <temporalfox> is that the factory for mqttmesssages [09:39:36] <temporalfox> is final [09:39:40] <temporalfox> and uses static method [09:39:56] <temporalfox> so it means eventually we cannot override with custom mqttmessage impl for vertx [09:40:08] <temporalfox> so you should contribute such thing to netty-mqtt [09:40:18] <temporalfox> the goal being having a different impl of mqttmessage [09:40:21] <temporalfox> in the codec [09:45:21] <ppatiern> for being ployglot do you need new classes derived from MqttMessage ? [09:46:07] <temporalfox> it would help yes to have a custom mqttmessage implemnetation [09:46:17] <temporalfox> for performance reason [09:46:30] <temporalfox> you said you want to keep Mqttmessage without translation to a vertx mqtt messgae right ? [09:46:46] <ppatiern> but then we need to map/copy content from any received MqttMessage (Netty) to MqttMessage (Vertx) [09:46:55] <ppatiern> it could be better [09:47:00] <temporalfox> well [09:47:10] <ppatiern> but I could miss some points on Vertx you know better than me [09:47:27] <temporalfox> I'm tryhing to figure out if we can have [09:47:33] <temporalfox> MqttEndpointBase [09:47:38] <temporalfox> MqttEndpointBase<M> extends ReadStream<M>, WriteStream<M> [09:47:50] <temporalfox> and then have a Netty MqttEndpointBase extension that does not do translation [09:47:55] <temporalfox> and have a Vertx that does translation [09:48:00] <temporalfox> only the Vertx would be polyglot [09:48:07] <temporalfox> it complicates a bit the OO design [09:48:09] <temporalfox> but it would work [09:48:27] <ppatiern> but what are the main needs to be polyglot ? [09:48:39] <temporalfox> because vertx is polyglot [09:49:19] <ppatiern> no I mean … what are the main implementation needs in order to have a Vert.x polyglot component [09:49:26] <ppatiern> derive from some class [09:49:29] <ppatiern> some annotations [09:49:33] <ppatiern> something like that ? [09:49:35] <temporalfox> use a @VertxGen type [09:49:44] <temporalfox> and then simple types like primitive [09:49:45] <temporalfox> enum [09:49:50] <temporalfox> look at the restriction [09:50:02] <temporalfox> https://github.com/vert-x3/vertx-codegen#api-constraints [09:50:36] <temporalfox> I think one way would be to have a netty server [09:50:47] <temporalfox> and wrap it with a vertx server [09:50:53] <temporalfox> the netty server would implement most of the logic [09:51:01] <temporalfox> the vertx thin wrapper would only do protocol translation [10:02:46] <ppatiern> temporalfox whatever you consider the simpler and quicker way … you are the guru on that. For now I'm on the way to continue adding other messages handling [10:03:02] <temporalfox> I hoe you can rebase your code on what I will do [10:04:04] <ppatiern> how much time do you think is needed for adding polyglot ? [10:10:17] <temporalfox> today should be ok I think [10:11:22] <ppatiern> are you going to fork the repo right ? [10:12:51] <temporalfox> I guess [10:17:21] <ppatiern> ok I'm here for any questions ;) [10:23:17] <temporalfox> no problem [10:36:32] <temporalfox> ppatiern here is a first shot https://github.com/vietj/mqtt-server [10:36:40] <temporalfox> it decouples the Netty server from the interfaces [10:36:53] <temporalfox> and the interfaces have been codegen-ified [10:36:56] <temporalfox> but they are not implemented [10:37:10] <temporalfox> in second shot I will do this wrapper that uses the Netty server [10:37:22] <temporalfox> Netty server is almost full implementation but that exposes Netty types [10:37:40] <temporalfox> the wrapper on top of it will only do type translation when necessary [10:37:45] <temporalfox> let me know how that sound to you [10:40:02] <temporalfox> ppatiern ping [10:40:21] <ppatiern> yes I'm reading your changes [10:40:38] <temporalfox> ok [10:41:53] <ppatiern> in any case I see you needed to re-implement MqttMessage [10:43:52] <ppatiern> didn't you figure out the way to avoid this ? [10:44:28] <temporalfox> no it's not possible [10:44:35] <temporalfox> but the vertx mqtt messages will be simpler I think [10:44:38] <temporalfox> in design [10:44:53] <temporalfox> like from subscribe message [10:44:54] <temporalfox> List<MqttTopicSubscription> payload(); [10:44:56] <temporalfox> directly [10:45:01] <temporalfox> instead of having an object for this [10:45:04] <temporalfox> like netty does [10:45:24] <temporalfox> but in your case you can use the NettyMqttServer directly [10:45:26] <ppatiern> but we need to copy any received message (or any message to send) [10:45:29] <temporalfox> with no message translation [10:45:47] <temporalfox> that's why there is NettyMqttServer for your case when you don't want copy [10:45:49] <ppatiern> so you want to provide a component useable in both ways ? [10:45:54] <temporalfox> yes [10:46:05] <temporalfox> unless you don't care about copying [10:46:09] <temporalfox> but it seems you do [10:48:39] <ppatiern> to me it could be a confusion for the user to see two ways … I think that I'd like to have only one way (the Vertx) and considering using the copy message [10:50:30] <temporalfox> ok [10:50:34] <temporalfox> as you want [10:50:51] <temporalfox> do you want me to finish this then ? [10:51:07] <temporalfox> i.e to continue with the current codegen api [10:51:11] <temporalfox> and use it in the server [10:53:30] <ppatiern> yes please … we need polyglot [10:53:35] <ppatiern> btw I think I can't continue [10:53:42] <ppatiern> too much conflicts on that [10:54:40] <temporalfox> so ? [10:54:47] <temporalfox> I will finish as soon as possible [10:54:52] <ppatiern> no worries [10:55:05] <ppatiern> please continue [10:55:29] <ppatiern> ;) [10:56:33] <ppatiern> btw if it changes something on what you have in mind [10:57:08] <ppatiern> please follow the way of providing only one way using Vert.x server and not Netty server even if with message copy [10:57:27] <ppatiern> imo it's better than user doesn't see a different way with direct netty server [10:58:22] <ppatiern> does it make sense to you ? [11:14:38] <temporalfox> ys [11:14:39] <temporalfox> yes [11:14:40] <temporalfox> I'm doing that [11:14:48] <temporalfox> almost there [11:15:00] <temporalfox> there will be some improvements [11:15:02] <temporalfox> to do [11:15:11] <temporalfox> like taking care of buffer lifecycle [11:15:19] <temporalfox> to avoid leaks [11:15:24] <temporalfox> but that's something that can be done later [11:16:29] <ppatiern> yes … I'd like to have this working with my related AMQP project and then improving it [11:16:50] <ppatiern> btw I'm going ahead committing some changes … I'll have conflicts of course … I'll fix them [11:16:58] <temporalfox> i.e now it's quick and dirty [11:17:04] <temporalfox> but I will improve it later [11:17:08] <temporalfox> I'm done actually [11:17:47] <temporalfox> https://github.com/vietj/mqtt-server [11:18:01] <temporalfox> I removed the endpoint stream for now [11:18:03] <temporalfox> to keep simple [11:18:27] <temporalfox> the message translation is quick and dirty [11:18:32] <temporalfox> should be improved later [11:18:38] <temporalfox> but it works [11:19:11] <temporalfox> let me know how it sound [11:26:31] <ppatiern> your already executed unit tests ? [11:27:22] <temporalfox> yes [11:27:52] <ppatiern> what do you think to open a PR so that I can merge and continue my work with your strcuture … it looks good to me ! [11:33:48] <ppatiern> it makes more sense to continue on your refactoring than my not polyglot code [11:36:49] <temporalfox> yes [11:36:57] <temporalfox> will do now [11:37:07] <ppatiern> thanks ! [11:37:53] <temporalfox> there is a conflict though [11:38:01] <temporalfox> weird [11:38:03] <temporalfox> because I rebased [11:38:18] <ppatiern> I committed one thing after your fork [11:38:24] <ppatiern> but I'm resetting it now [11:38:30] <ppatiern> just a sec [11:38:37] <ppatiern> few lines I can rewrite after [11:38:40] <ppatiern> later [11:39:26] <temporalfox> ok [11:39:41] <ppatiern> try it now … I reset to the previous commit [11:39:47] <ppatiern> when you forked [11:41:48] <ppatiern> I see your PR [11:42:30] <ppatiern> what's the dataobjects.adoc you added ? [11:43:23] <temporalfox> it's automatic [11:43:27] <temporalfox> the code generator [11:43:28] <temporalfox> does it [11:43:33] <temporalfox> so it makes documentation for data object [11:43:43] <temporalfox> that is then available for other languages [11:43:46] <temporalfox> like javscript [11:43:52] <temporalfox> and also as cheatsheet [11:44:01] <temporalfox> like a recap of options [11:44:08] <ppatiern> ah ok [11:45:11] <ppatiern> thanks for your today work [11:45:18] <temporalfox> no problem [11:45:34] <temporalfox> we can so submit this project to the vertx commiters [11:45:47] <temporalfox> that we have defined its scope [11:45:52] <temporalfox> and it seems like a useful add to vertx umbrella [11:46:02] <temporalfox> will allow more synergies and expand capabilities [11:46:17] <temporalfox> can you send me a little bio ? [11:46:23] <temporalfox> like a couple of lines [11:46:31] <ppatiern> and MQTT is a protocol used a lot under Eclipse IoT [11:46:32] <temporalfox> in particular what you have done also with vertx [11:46:38] <temporalfox> yes good point [11:46:51] <ppatiern> it's the MAIN protocol in Eclipse IoT land [11:47:05] <ppatiern> yes I'll send it via email [11:56:46] <ppatiern> temporalfox bio just sent [12:12:44] <ppatiern> temporalfox: ping [16:38:16] <temporalfox> ppatiern hi again [16:38:33] <ppatiern> temporalfox hi ! [16:38:39] <temporalfox> I'm reading a bit about mqtt [16:38:57] <temporalfox> so the wildcards would be parsed by the mqtt server ? [16:39:03] <temporalfox> or it would provide helpers to parse them ? [16:39:06] <temporalfox> something like that ? [16:39:57] <ppatiern> my opinion is that the mqtt server should provide to the upper level the topic as it is … with or without wildcards [16:40:03] <ppatiern> the application should handle this [16:40:10] <temporalfox> ok [16:40:16] <temporalfox> it could provide helper for parsing them though [16:40:21] <temporalfox> i.e static methods [16:40:24] <ppatiern> oh sure [16:40:56] <temporalfox> I'm wondering also about mqtt-codec [16:41:02] <temporalfox> can it be used to build a client as well ? [16:41:45] <ppatiern> yes I think so .. but for this the Eclipse Foundation has the Paho umbrella with all MQTT client implementations (in Java as well) [16:42:02] <temporalfox> ppatiern ok [16:45:07] <ppatiern> ;) [16:45:46] <temporalfox> I see paho has also server support [16:46:14] <ppatiern> you are speaking about mosquitto ? [16:47:42] <ppatiern> “The Eclipse Paho project provides open-source client implementations of MQTT and MQTT-SN messaging protocols …” [16:48:46] <temporalfox> looking at paho page [16:48:46] <ppatiern> Mosquitto is the open source broker sponsored by Eclipse : http://projects.eclipse.org/projects/technology.mosquitto [16:49:08] <temporalfox> ok [16:50:28] <temporalfox> mosquitto is in C ? [16:51:23] <ppatiern> yes [21:46:18] * ChanServ sets mode: +o temporalfox

[22:06:11] <shvalb> Hi guys,

[22:06:34] <shvalb> I'm new here in this channel - and if it works like Slack…that would be AWESOME!

[22:10:53] <myghty> haha

[22:10:56] <myghty> well irc is irc

[22:11:00] <myghty> that does not work like slack

[22:11:18] <myghty> ircs is a chatroom. nothing more nothing less

[22:22:01] <AlexLehm> haven't used slack much, but most of the concepts like channels and direct messages are very similar to irc

[22:22:20] <AlexLehm> its not quite as colorful since its text only unless you use a web irc like irccloud

[22:23:28] <AlexLehm> shvalb: its has mentions, though

[22:23:29] <myghty> AlexLehm: well, slack is much more than that :) slack collects all the communication from different channels and puts it in one place, email, build messages, chats, documents

[22:23:32] <myghty> everything

[22:24:11] <AlexLehm> well, build messages can be posted to irc easily, e.g. from jenkins

[22:24:43] <myghty> sure :) but not documents, emails and all the stuff

[22:25:14] <AlexLehm> well, you could use a pastebin for that

[22:25:29] <myghty> exactly, but then it is already not irc anymore

[22:25:55] <AlexLehm> well yes

[22:26:33] <AlexLehm> some bot functionality in irc is quite good, for example on the #jenkins irc, they are replying with issue descriptions is someone posts a JIRA issue number

[22:26:56] <myghty> oh cool

[22:27:32] <myghty> I just started for my own project with the newest version of gitlab… and they evolved a lot since the last time, also integrated an open source slack alternative, scrum board and all that stuff

[22:28:47] <myghty> so for development purposes: pretty cool

[22:29:16] <myghty> and everything in one place

[22:31:23] <myghty> AlexLehm: what I like a lot about gitlab is their new CI feature… everyone can register a build runner, then gitlab chooses an available. This can also be a notebook of a developer which is not under load right now

[22:31:28] <myghty> so effective use of ressources

[22:32:06] <AlexLehm> putting the ci right into the git platform seems kind of odd to me

[22:32:29] <myghty> well, it is not in the git plattform itself, gitlab is modular, but well, you have one interface

[22:32:35] <AlexLehm> ok

[22:32:57] <myghty> one webinterface/platform for git repository browsing, issue tracking, build and environment status

[23:41:01] *** kornbluth.freenode.net sets mode: +o temporalfox