Messing around with Kafka | 220713
220713
Wtf is the key used for in Kafka I saw the key inside of the go-kafka for the writer and wasn’t wtf it was meant for. TIL I think it to tell kafka what partition to use. This is a little hard for I to understand is cause I’m not really using partitions (or at least on purpose) cause I not needing the redundancy on my local machine for test/development data.
Speeed
Okay the slow kafka message receiving has been corrected and was a simple fix just update the writer from the default for Batch Size from 1000 messages and the Batch Timeout of 10 mil seconds. Granted I was getting more than a 10 mil second delay but whatever
1530
So I am at the point where I need to setup the Mux to grab the intended transaction from the event message and I am no quite sure where I should plug in at so I can try and re use the handlers for the rest. The issue with that I am going to need to pass some http.Response/http.Request which I am going to make/pass and then drop later. The other option thats in my head would be to make new ones that plug right into the model functions directly
| Use Existing Handlers | Make new Event Message Handlers |
|---|---|
| ++ Reuse code | – Make all new Handlers |
| – Passing un-needed Data | ++ Limit what function other service have access to. (cause not every service needs the ability to everything the Rest service does) |
| ++ Some Error Catching is already there | ++ Don’t Need to Deal with request/headers/ http things |
| ++Smaller Code base |
.. I am sure I forgetting or not thinking of some items
I still not sold on which is better or what I am suppose to do. I am going to try and use the existing handler with casting tom foolery to try and not have two handlers do about the same thing.
_1758
Alright that wasn’t that painful. Instead of trying to plug into services at handler level what if I just start at the beginning. Just send a HTTP Request. Granted this may cause some items that need be checked once in the cloud with CORS, TLS, and even have each services-event manger to have credentials. These are all items for future me but I think I understand how to deal atleast two of those. So sending a http request
Alright Now I Need to do the writer part to send message back to the send that all work or did not.