sauljp

Working on it

100%

I do not think it is a shit arch. I think that is an arch for some problems, but people insist on using it for uses cases it was not built for.

I guess both work :)

100%

Hey ddoolin, It's so nice to read this, yes! kaskade ignores the schema registry metadata for json and protobuf messages generated for schema registry, you do not need to provide url (only in case of using avro).

The only catch (in the protobuf case) is to generate the descriptors with protoc, but the good thing is generally we all have them, so maybe is not a big problem.

Hi! I did not implemented something like that, but I can say this year textual has a great set of features and a nice community willing to help and share. I totally recommend textual.

Thanks!

Totally agree

about the first step: if you already have the .proto file you do not need to download it from schema registry

thanks! that is awesome

Hi! kaskade is more like AKHQ than Cruise control.

We use strimzi by default, so we deploy cruise control with kafka, and it takes care of rebalancing the data across the nodes. Also you can deploy it without strimzi.

Delete crap is more complicated, usually with kafka-delete-records (this is king of new I think). The problem is the offsets. By general rule you should not delete data from topics

Hi! it validates if the message was (or not) generated for schema registry, it checks if the message has the schema registry magic bytes (this bytes has the schema id). so, it deserializes messages with or without schema registry.

This is the drawback: you have to generate the descriptor first.

1) download the schema from schema registry: http :8081/schemas/ids/<my-id>/schema

2) generate the descriptor with the schema: protoc --include_imports --descriptor_set_out=my-descriptor.desc --proto_path=. my-schema.proto

3) use kaskade: kaskade consumer -b my-kafka:9092 -x auto.offset.reset=earliest -k string -v protobuf -t my-protobuf-topic -p descriptor=my-descriptor.desc -p value=mypackage.MyMessage

Install it:

brew install kaskade

pipx install kaskade