Approvals: 0/1
[09:00:06] * ChanServ sets mode: +o temporalfox
[09:04:19] <magabiztos> hi all!
[09:18:15] <temporalfox> hi ppatiern
[09:18:27] <ppatiern> hi temporalfox !
[09:19:19] <temporalfox> so I can have a look a bit this morning at the polyglotisation of mqtt
[09:19:43] <temporalfox> I would like that you tell me the scope of this component and its role
[09:19:58] <temporalfox> i.e define what it does and what it does not
[09:20:12] <ppatiern> ok
[09:20:36] <ppatiern> the main objective is to handle communication with remote MQTT clients
[09:20:59] <temporalfox> I see it is named mqtt-server now
[09:21:05] <ppatiern> it means that the MQTT server is able to handle all the MQTT messages defined by the spec (MQTT 3.1.1) like CONNECT, PUBLISH and so on
[09:21:10] <temporalfox> does a “regular” server handle more than this ?
[09:21:34] <ppatiern> my mqtt-server component ISN'T a full MQTT broker
[09:21:52] <temporalfox> so it's called server now
[09:22:00] <ppatiern> yes
[09:22:07] <temporalfox> because it's not a broker
[09:22:12] <ppatiern> just because as a server it can handle communication with clients
[09:22:13] <ppatiern> yes
[09:22:17] <ppatiern> it's not a broker
[09:22:18] <temporalfox> so this server an be used to implement a broker
[09:22:29] <ppatiern> yes … it can be used to do that
[09:22:34] <temporalfox> so what a broker would do in addition of being a server ?
[09:22:55] <ppatiern> being a full broker means to implement all the MQTT spec
[09:22:57] <ppatiern> for example …
[09:23:17] <temporalfox> so more messages ?
[09:23:25] <ppatiern> nope
[09:23:29] <ppatiern> let me try with an example
[09:24:16] <ppatiern> will testment : when a client CONNECT to a broker it sends information about a “will” message that should be send when the client goes offline due to a brute disconnection. The “will” information are in the CONNECT message. The broker needs to store this information in some way and then publish the message when needed (as I said before)
[09:24:17] <temporalfox> ok
[09:24:38] <temporalfox> so persistence ?
[09:24:46] <ppatiern> for example
[09:24:51] <ppatiern> my mqtt-server just provide the way to an application on top of it to know about the CONNECT message is received and doing what it needs to do
[09:25:07] <ppatiern> a good example is protocol translation for example
[09:25:36] <temporalfox> so it means that with an mqtt-broker, someone wanting to do protocol translation needs to be an client of the mqtt-broker
[09:25:42] <ppatiern> using the MQTT server to receive connections from MQTT clients … accepting the CONNECT message .. extract the will … and translating it on top of another protocol for example
[09:25:47] <temporalfox> and in this case it's simpler, it's an handler of the mqtt-server
[09:25:54] <ppatiern> yes
[09:26:03] <temporalfox> so the use cases are
[09:26:41] <ppatiern> yes absolutely
[09:26:50] <ppatiern> these are the main use cases I see
[09:26:52] <temporalfox> with the current vertx stack, what interactions could be done ?
[09:27:05] <temporalfox> like we will have soon kafka client
[09:27:09] <temporalfox> we have amqp support
[09:27:11] <temporalfox> rabiit mq
[09:27:26] <temporalfox> using these with the server would make somehow sense ?
[09:27:32] <temporalfox> or just the event bus
[09:27:33] <temporalfox> too
[09:27:37] <ppatiern> you could use for example Vert.x Proton (AMQP) for building a custom logic on top of the mqtt-server and bridging mqtt to amqp
[09:27:46] <ppatiern> absolutely even the event bus for example
[09:28:09] <ppatiern> the same for kafka for example
[09:28:17] <temporalfox> ok
[09:28:24] <ppatiern> you could decide that a feature like the “will” message become a message to store into kafka
[09:28:26] <ppatiern> for example
[09:28:33] <temporalfox> ok
[09:28:56] <temporalfox> looks enough for now
[09:28:59] <ppatiern> it just provides the underlying communication with MQTT clients … exposing fluent API for handling received messages and sending new ones
[09:29:02] <temporalfox> i.e to understand the role
[09:29:17] <temporalfox> it could also use websockets ?
[09:29:21] <temporalfox> I'm guessing
[09:29:35] <ppatiern> for communication with MQTT clients you mean ?
[09:29:47] <temporalfox> for relaying messages to other servers
[09:29:53] <ppatiern> oh ues
[09:29:55] <ppatiern> yes
[09:29:55] <temporalfox> does aeron makes sense too ?
[09:30:07] <temporalfox> I've been doing an aeron client recently
[09:30:14] <temporalfox> for vertx
[09:30:32] <ppatiern> I have never used aeron
[09:31:17] <temporalfox> it's an IPC layer
[09:31:18] <temporalfox> that can use multicast
[09:31:18] <ppatiern> but the main thing is that you can handle incoming messages … you of course need to reply in the correct way (as MQTT spec says, i.e. CONNECT received … CONNACK to send) but in the middle you can do whatever you want
[09:31:21] <temporalfox> too
[09:31:25] <temporalfox> using UDP mainly
[09:31:37] <ppatiern> on top of my head I don't see limitations
[09:32:02] <ppatiern> it's important that receiving MQTT messages you reply in the “correct” way (as I said i.e. receive CONNECT, need to send CONNACK)
[09:32:11] <temporalfox> yes
[09:32:13] <ppatiern> and between them you can put the logic you want in your application
[09:32:17] <temporalfox> obviously
[09:32:35] <temporalfox> now I see 15 tests
[09:32:38] <temporalfox> they run
[09:32:47] <temporalfox> one fails but it's a base abstract class
[09:32:57] <ppatiern> one fail ?
[09:32:58] <temporalfox> I'm going to spend quickly some time today making it polyglot
[09:33:03] <temporalfox> initializationError(enmasse.mqtt.test.MqttBaseTest): No runnable methods
[09:33:17] <ppatiern> oh yes
[09:33:19] <ppatiern> right
[09:33:29] <ppatiern> this is a base for the other test classes
[09:33:34] <ppatiern> I'll check it
[09:33:52] <temporalfox> it should be abstract class probably
[09:33:57] <ppatiern> yes
[09:34:12] <ppatiern> thanks
[09:34:17] <temporalfox> so I'm going to do this polyglot thing
[09:34:21] <temporalfox> because it's quite important for being in vertx stack
[09:34:28] <ppatiern> I understand
[09:34:47] <ppatiern> I'll continue to work on that on MQTT message handling side
[09:34:56] <temporalfox> ok
[09:35:06] <temporalfox> I hope we won't have much conflicts
[09:35:31] <ppatiern> I hope so
[09:36:49] <ppatiern> where I can read something about how to make polyglot a Vert.x component (just for the sake of curiosity)
[09:38:00] <temporalfox> this document does not exist yet
[09:38:18] <temporalfox> we do have this doc
[09:38:18] <temporalfox> https://github.com/vert-x3/vertx-codegen/blob/master/src/main/asciidoc/java/index.adoc
[09:38:22] <temporalfox> about code generation in general
[09:38:25] <temporalfox> it will give you an idea
[09:38:35] <temporalfox> I plan to write such doc though
[09:38:39] <temporalfox> along with the netty guide
[09:38:47] <temporalfox> you already had
[09:39:03] <ppatiern> great thanks
[09:39:29] <temporalfox> ppatiern one issue with netty mqtt codec
[09:39:35] <temporalfox> is that the factory for mqttmesssages
[09:39:36] <temporalfox> is final
[09:39:40] <temporalfox> and uses static method
[09:39:56] <temporalfox> so it means eventually we cannot override with custom mqttmessage impl for vertx
[09:40:08] <temporalfox> so you should contribute such thing to netty-mqtt
[09:40:18] <temporalfox> the goal being having a different impl of mqttmessage
[09:40:21] <temporalfox> in the codec
[09:45:21] <ppatiern> for being ployglot do you need new classes derived from MqttMessage ?
[09:46:07] <temporalfox> it would help yes to have a custom mqttmessage implemnetation
[09:46:17] <temporalfox> for performance reason
[09:46:30] <temporalfox> you said you want to keep Mqttmessage without translation to a vertx mqtt messgae right ?
[09:46:46] <ppatiern> but then we need to map/copy content from any received MqttMessage (Netty) to MqttMessage (Vertx)
[09:46:55] <ppatiern> it could be better
[09:47:00] <temporalfox> well
[09:47:10] <ppatiern> but I could miss some points on Vertx you know better than me
[09:47:27] <temporalfox> I'm tryhing to figure out if we can have
[09:47:33] <temporalfox> MqttEndpointBase
[09:47:38] <temporalfox> MqttEndpointBase<M> extends ReadStream<M>, WriteStream<M>
[09:47:50] <temporalfox> and then have a Netty MqttEndpointBase extension that does not do translation
[09:47:55] <temporalfox> and have a Vertx that does translation
[09:48:00] <temporalfox> only the Vertx would be polyglot
[09:48:07] <temporalfox> it complicates a bit the OO design
[09:48:09] <temporalfox> but it would work
[09:48:27] <ppatiern> but what are the main needs to be polyglot ?
[09:48:39] <temporalfox> because vertx is polyglot
[09:49:19] <ppatiern> no I mean … what are the main implementation needs in order to have a Vert.x polyglot component
[09:49:26] <ppatiern> derive from some class
[09:49:29] <ppatiern> some annotations
[09:49:33] <ppatiern> something like that ?
[09:49:35] <temporalfox> use a @VertxGen type
[09:49:44] <temporalfox> and then simple types like primitive
[09:49:45] <temporalfox> enum
[09:49:50] <temporalfox> look at the restriction
[09:50:02] <temporalfox> https://github.com/vert-x3/vertx-codegen#api-constraints
[09:50:36] <temporalfox> I think one way would be to have a netty server
[09:50:47] <temporalfox> and wrap it with a vertx server
[09:50:53] <temporalfox> the netty server would implement most of the logic
[09:51:01] <temporalfox> the vertx thin wrapper would only do protocol translation
[10:02:46] <ppatiern> temporalfox whatever you consider the simpler and quicker way … you are the guru on that.
For now I'm on the way to continue adding other messages handling
[10:03:02] <temporalfox> I hoe you can rebase your code on what I will do
[10:04:04] <ppatiern> how much time do you think is needed for adding polyglot ?
[10:10:17] <temporalfox> today should be ok I think
[10:11:22] <ppatiern> are you going to fork the repo right ?
[10:12:51] <temporalfox> I guess
[10:17:21] <ppatiern> ok I'm here for any questions ;)
[10:23:17] <temporalfox> no problem
[10:36:32] <temporalfox> ppatiern here is a first shot https://github.com/vietj/mqtt-server
[10:36:40] <temporalfox> it decouples the Netty server from the interfaces
[10:36:53] <temporalfox> and the interfaces have been codegen-ified
[10:36:56] <temporalfox> but they are not implemented
[10:37:10] <temporalfox> in second shot I will do this wrapper that uses the Netty server
[10:37:22] <temporalfox> Netty server is almost full implementation but that exposes Netty types
[10:37:40] <temporalfox> the wrapper on top of it will only do type translation when necessary
[10:37:45] <temporalfox> let me know how that sound to you
[10:40:02] <temporalfox> ppatiern ping
[10:40:21] <ppatiern> yes I'm reading your changes
[10:40:38] <temporalfox> ok
[10:41:53] <ppatiern> in any case I see you needed to re-implement MqttMessage
[10:43:52] <ppatiern> didn't you figure out the way to avoid this ?
[10:44:28] <temporalfox> no it's not possible
[10:44:35] <temporalfox> but the vertx mqtt messages will be simpler I think
[10:44:38] <temporalfox> in design
[10:44:53] <temporalfox> like from subscribe message
[10:44:54] <temporalfox> List<MqttTopicSubscription> payload();
[10:44:56] <temporalfox> directly
[10:45:01] <temporalfox> instead of having an object for this
[10:45:04] <temporalfox> like netty does
[10:45:24] <temporalfox> but in your case you can use the NettyMqttServer directly
[10:45:26] <ppatiern> but we need to copy any received message (or any message to send)
[10:45:29] <temporalfox> with no message translation
[10:45:47] <temporalfox> that's why there is NettyMqttServer for your case when you don't want copy
[10:45:49] <ppatiern> so you want to provide a component useable in both ways ?
[10:45:54] <temporalfox> yes
[10:46:05] <temporalfox> unless you don't care about copying
[10:46:09] <temporalfox> but it seems you do
[10:48:39] <ppatiern> to me it could be a confusion for the user to see two ways … I think that I'd like to have only one way (the Vertx) and considering using the copy message
[10:50:30] <temporalfox> ok
[10:50:34] <temporalfox> as you want
[10:50:51] <temporalfox> do you want me to finish this then ?
[10:51:07] <temporalfox> i.e to continue with the current codegen api
[10:51:11] <temporalfox> and use it in the server
[10:53:30] <ppatiern> yes please … we need polyglot
[10:53:35] <ppatiern> btw I think I can't continue
[10:53:42] <ppatiern> too much conflicts on that
[10:54:40] <temporalfox> so ?
[10:54:47] <temporalfox> I will finish as soon as possible
[10:54:52] <ppatiern> no worries
[10:55:05] <ppatiern> please continue
[10:55:29] <ppatiern> ;)
[10:56:33] <ppatiern> btw if it changes something on what you have in mind
[10:57:08] <ppatiern> please follow the way of providing only one way using Vert.x server and not Netty server even if with message copy
[10:57:27] <ppatiern> imo it's better than user doesn't see a different way with direct netty server
[10:58:22] <ppatiern> does it make sense to you ?
[11:14:38] <temporalfox> ys
[11:14:39] <temporalfox> yes
[11:14:40] <temporalfox> I'm doing that
[11:14:48] <temporalfox> almost there
[11:15:00] <temporalfox> there will be some improvements
[11:15:02] <temporalfox> to do
[11:15:11] <temporalfox> like taking care of buffer lifecycle
[11:15:19] <temporalfox> to avoid leaks
[11:15:24] <temporalfox> but that's something that can be done later
[11:16:29] <ppatiern> yes … I'd like to have this working with my related AMQP project and then improving it
[11:16:50] <ppatiern> btw I'm going ahead committing some changes … I'll have conflicts of course … I'll fix them
[11:16:58] <temporalfox> i.e now it's quick and dirty
[11:17:04] <temporalfox> but I will improve it later
[11:17:08] <temporalfox> I'm done actually
[11:17:47] <temporalfox> https://github.com/vietj/mqtt-server
[11:18:01] <temporalfox> I removed the endpoint stream for now
[11:18:03] <temporalfox> to keep simple
[11:18:27] <temporalfox> the message translation is quick and dirty
[11:18:32] <temporalfox> should be improved later
[11:18:38] <temporalfox> but it works
[11:19:11] <temporalfox> let me know how it sound
[11:26:31] <ppatiern> your already executed unit tests ?
[11:27:22] <temporalfox> yes
[11:27:52] <ppatiern> what do you think to open a PR so that I can merge and continue my work with your strcuture … it looks good to me !
[11:33:48] <ppatiern> it makes more sense to continue on your refactoring than my not polyglot code
[11:36:49] <temporalfox> yes
[11:36:57] <temporalfox> will do now
[11:37:07] <ppatiern> thanks !
[11:37:53] <temporalfox> there is a conflict though
[11:38:01] <temporalfox> weird
[11:38:03] <temporalfox> because I rebased
[11:38:18] <ppatiern> I committed one thing after your fork
[11:38:24] <ppatiern> but I'm resetting it now
[11:38:30] <ppatiern> just a sec
[11:38:37] <ppatiern> few lines I can rewrite after
[11:38:40] <ppatiern> later
[11:39:26] <temporalfox> ok
[11:39:41] <ppatiern> try it now … I reset to the previous commit
[11:39:47] <ppatiern> when you forked
[11:41:48] <ppatiern> I see your PR
[11:42:30] <ppatiern> what's the dataobjects.adoc you added ?
[11:43:23] <temporalfox> it's automatic
[11:43:27] <temporalfox> the code generator
[11:43:28] <temporalfox> does it
[11:43:33] <temporalfox> so it makes documentation for data object
[11:43:43] <temporalfox> that is then available for other languages
[11:43:46] <temporalfox> like javscript
[11:43:52] <temporalfox> and also as cheatsheet
[11:44:01] <temporalfox> like a recap of options
[11:44:08] <ppatiern> ah ok
[11:45:11] <ppatiern> thanks for your today work
[11:45:18] <temporalfox> no problem
[11:45:34] <temporalfox> we can so submit this project to the vertx commiters
[11:45:47] <temporalfox> that we have defined its scope
[11:45:52] <temporalfox> and it seems like a useful add to vertx umbrella
[11:46:02] <temporalfox> will allow more synergies and expand capabilities
[11:46:17] <temporalfox> can you send me a little bio ?
[11:46:23] <temporalfox> like a couple of lines
[11:46:31] <ppatiern> and MQTT is a protocol used a lot under Eclipse IoT
[11:46:32] <temporalfox> in particular what you have done also with vertx
[11:46:38] <temporalfox> yes good point
[11:46:51] <ppatiern> it's the MAIN protocol in Eclipse IoT land
[11:47:05] <ppatiern> yes I'll send it via email
[11:56:46] <ppatiern> temporalfox bio just sent
[12:12:44] <ppatiern> temporalfox: ping
[16:38:16] <temporalfox> ppatiern hi again
[16:38:33] <ppatiern> temporalfox hi !
[16:38:39] <temporalfox> I'm reading a bit about mqtt
[16:38:57] <temporalfox> so the wildcards would be parsed by the mqtt server ?
[16:39:03] <temporalfox> or it would provide helpers to parse them ?
[16:39:06] <temporalfox> something like that ?
[16:39:57] <ppatiern> my opinion is that the mqtt server should provide to the upper level the topic as it is … with or without wildcards
[16:40:03] <ppatiern> the application should handle this
[16:40:10] <temporalfox> ok
[16:40:16] <temporalfox> it could provide helper for parsing them though
[16:40:21] <temporalfox> i.e static methods
[16:40:24] <ppatiern> oh sure
[16:40:56] <temporalfox> I'm wondering also about mqtt-codec
[16:41:02] <temporalfox> can it be used to build a client as well ?
[16:41:45] <ppatiern> yes I think so .. but for this the Eclipse Foundation has the Paho umbrella with all MQTT client implementations (in Java as well)
[16:42:02] <temporalfox> ppatiern ok
[16:45:07] <ppatiern> ;)
[16:45:46] <temporalfox> I see paho has also server support
[16:46:14] <ppatiern> you are speaking about mosquitto ?
[16:47:42] <ppatiern> “The Eclipse Paho project provides open-source client implementations of MQTT and MQTT-SN messaging protocols …”
[16:48:46] <temporalfox> looking at paho page
[16:48:46] <ppatiern> Mosquitto is the open source broker sponsored by Eclipse : http://projects.eclipse.org/projects/technology.mosquitto
[16:49:08] <temporalfox> ok
[16:50:28] <temporalfox> mosquitto is in C ?
[16:51:23] <ppatiern> yes
[21:46:18] * ChanServ sets mode: +o temporalfox
[22:06:11] <shvalb> Hi guys,
[22:06:34] <shvalb> I'm new here in this channel - and if it works like Slack…that would be AWESOME!
[22:10:53] <myghty> haha
[22:10:56] <myghty> well irc is irc
[22:11:00] <myghty> that does not work like slack
[22:11:18] <myghty> ircs is a chatroom. nothing more nothing less
[22:22:01] <AlexLehm> haven't used slack much, but most of the concepts like channels and direct messages are very similar to irc
[22:22:20] <AlexLehm> its not quite as colorful since its text only unless you use a web irc like irccloud
[22:23:28] <AlexLehm> shvalb: its has mentions, though
[22:23:29] <myghty> AlexLehm: well, slack is much more than that :) slack collects all the communication from different channels and puts it in one place, email, build messages, chats, documents
[22:23:32] <myghty> everything
[22:24:11] <AlexLehm> well, build messages can be posted to irc easily, e.g. from jenkins
[22:24:43] <myghty> sure :) but not documents, emails and all the stuff
[22:25:14] <AlexLehm> well, you could use a pastebin for that
[22:25:29] <myghty> exactly, but then it is already not irc anymore
[22:25:55] <AlexLehm> well yes
[22:26:33] <AlexLehm> some bot functionality in irc is quite good, for example on the #jenkins irc, they are replying with issue descriptions is someone posts a JIRA issue number
[22:26:56] <myghty> oh cool
[22:27:32] <myghty> I just started for my own project with the newest version of gitlab… and they evolved a lot since the last time, also integrated an open source slack alternative, scrum board and all that stuff
[22:28:47] <myghty> so for development purposes: pretty cool
[22:29:16] <myghty> and everything in one place
[22:31:23] <myghty> AlexLehm: what I like a lot about gitlab is their new CI feature… everyone can register a build runner, then gitlab chooses an available. This can also be a notebook of a developer which is not under load right now
[22:31:28] <myghty> so effective use of ressources
[22:32:06] <AlexLehm> putting the ci right into the git platform seems kind of odd to me
[22:32:29] <myghty> well, it is not in the git plattform itself, gitlab is modular, but well, you have one interface
[22:32:35] <AlexLehm> ok
[22:32:57] <myghty> one webinterface/platform for git repository browsing, issue tracking, build and environment status
[23:41:01] *** kornbluth.freenode.net sets mode: +o temporalfox