d_t_w

I have deployed and run Cassandra myself, basically as you describe.

The first 'dev' Cassandra install was three nodes. I downloaded the .tar.gz, installed it, started each process in turn on each node with the required configuration.

That was 2012 and there is a chance that cluster is still running. It was low-volume in terms of data. TTL configured so it would never run out of disk. Never had any issues in particular. I used it for ~5 years before concluding work with that client.

The problem in that case was Cassandra proliferated in that small company, they didn't build any particular expertise beyond me, and in the end I was being pulled into discussions from different teams, different products, split across about 13 clusters. Wasn't much fun - but that wasn't the DB's fault.

Hell of a story Dan! Congratulations on the new arrival and best wishes.

The parse-*, update-keys, update-vals, and random-uuid functions are great additions to core. I look forward to dropping the custom versions from each of my projects!

It has been some time but I think it's from "Hells Angels' I learned Ken Kesey, author of One Flew Over the Cuckoo's Nest, participated in MKUltra while working as a student at Menlo Park Veteran's Hospital and the character Chief was inspired by his own hallucinations.

Hunter S. Thompson partied with the Hells Angels at Ken Kesey's house with the Merry Pranksters. There's a lot to unpack there.

Not the OP but our product falls into the 'specialty engineering software' category. We build an enterprise ready tool for teams working with Apache Kafka (https://kpow.io) and offer an hourly price.

The deliverable is a single docker container (not a desktop app!) and right from the start, nearly three years now, we've been selling on the AWS Marketplace for 0.16c/hr:

https://aws.amazon.com/marketplace/pp/prodview-5jvke6codhrsm

We have a couple of pricing models, but on the consumption based model we stick to that clear, transparent hourly price. It's good for customers that choose that model, and good for us because we're engineers and we don't care for opaque enterprise sales practices.

We do a fair number of enterprise sales too, they require a negotiation because metering usage at an enterprise level is rarely practical. Even then the price is some calculation of usage/value + support. Keeps it simple, lets us attack a big market as a small team and stay focused on technical delivery, shortens the sale/negotiation time, and our customers know they're basically paying the same price as anyone else.

Not right away no, but it will help to have an understanding of Java/JVM for Clojure and Browser/Javascript for Clojurescript if you are intending on building real systems.

Thanks for sharing - you've spotted the hole in my experience.

I tend to lead more on the JVM and I have talented team-mates who drive the architecture on the JS side. I love being able to contribute both front and back end feature-wise but I don't have too much depth in understanding the bigger issues and advantages/drawbacks of Clojure (and macros) in JS land.

I've been programming in Clojure professionally for roughly ten years now. It's a great language and extremely effective if you're building products both on the JVM and in the browser.

I think I've written a handful of macros in that time, there is one in my current codebase in a test class, and it could probably be removed.

Macros are for extending the language. Programmers who are new to Clojure tend to pick macros up and get a bit excited and create lots of DSLs and other cute little trinkets, and then they put macros down again because they realise they don't compose.

The power of Clojure is in simple datastructures, the core functions that operate on them, composition, and host interop.

The most useful thing that I made with macros was a library for opinionated exception handling for Clojure core.async, at the end of which I realised I was using core.async a bit too much and ended up removing a bunch of it.

The product I work on today is an Enterprise toolkit for Apache Kafka (https://kpow.io) that according to Github is 97% Clojure. Data-oriented, great delivery cadence, and terrific fun to work on. Having a single language on the JVM and in the Browser is enormous leverage, certainly we feel the benefit every day.

I'm not quite sure what 'housing' means in that quote because CPI specifically excludes most housing costs.

RPI takes into account housing costs such as mortgage interest payments, building insurance etc. I'm not sure what USA measure, but the UK shifted to CPI in 2003 to harmonise with EU.

So no, UK/EU/CPI inflation measures basically don't include housing related costs. Certainly not the price of houses, which might be the thrust of OPs point.

Yes, in two ways.

First - kPow also stores all telemetry in internal topics (and also keeps an audit-log in another topic for data governance). Setting up multi-cluster is fairly easy and kPow stores data in the first configured cluster - so like Confluent's suggestion you could have a dedicated kPow cluster. We really only recommend this if/when you get up to a large number of connected Kafka resources (Cluster, Schema, or Connect) and/or you have cluster/s that are very busy with other work.

Second - from v85 (next week) our v3 compute engine make kPow monitoring functionally independent of topic load / performance of the underlying cluster. We do still persist to internal topics for recovery, etc, but live stats and control are only dependent on your Kafka cluster responding to AdminClient requests (a handful every minute).

If you would like a trial license or guided tour, just say.

See https://kpow.io - if you would like a trial license let me know. 6k topics is fine, might need slight tuning which is easily done (6k groups slightly more complicated).

Multi-Tenancy is probably worth a link:

https://kpow.io/how-to/manage-kafka-visibility-with-multi-te...

kPow gives you the ability to restrict visibility of Kafka resources down to a group or topic level by user-role, then builds completely internally consistent synthetic cluster views for each tenant.

We have a number of large enterprise customers who requested multi-tenancy support, it sounded tough at first but due to how kPow is built (we compute our own metrics / no JMX) and the fact that we use Clojure in both the front/back end it turned out to be fairly straight forward to implement.

https://kpow.io - we build kPow for Apache Kafka, a drop-in replacement for Confluent Control Center + much more.

https://kpow.io/features

- Multi-Cluster, Multi-Schema, Multi-Connect.

- Multi-Tenancy, RBAC, all the Authz - LDAP, SAML, etc.

- Very fast streaming multi-topic search (~1M messages / minute)

- Built in JQ search filters (we implemented a subset of JQ in the browser and JVM)

- Live Kafka Streams / Consumer Group topology visualisation (including RocksDB metrics for state stores, etc)

- Offset management, resetting, scheduled mutations.

- Consumer Group read and Topic write freshness / n-minutes inactivity metrics.

- Heaps of admin functions like Staged Mutations, Temporary RBAC policies, Data Masking for PII, etc.

- Prometheus / OpenMetrics endpoints.

- kREPL (slightly mad programmatic interface to Kafka)

- 7k commits, 100k docker pulls, 97.4% Clojure.

- Unlimited users, unlimited features.

- Commercially supported, annual licenses or available as hourly-metered (0.16c/hr) on the AWS Marketplace or Red Hat Marketplace.

Four eyes spying on your citizens and sending reports back to your intelligence agency is sort of the point of Five Eyes (well, one of the points).

It ain't cheating if you're not spying on your own people.

It's complex.

When I first moved to Australia I rankled at compulsory voting as I believe that not participating is a valid political act. Countries without compulsory voting talk about the turnout as relative to previous years, and if it falls too far there will be questions about a mandate to govern.

In Australia that can't happen as turnout is always high, this leads to very little opportunity to overhaul the mechanism of elections to make it more applicable to the community, or to overhaul political parties entirely. Red team or Blue team are guaranteed some high proportion of the country voting for them and it gives a false sense of relevance.

However I'm now fully in favour of compulsory voting. Why?

Look at America, what an absolute shambles of a democratic system. One party spending decades attempting to stop black people voting. Voter suppression and gerrymandering (also not possible in Australia) baked into the political system. No thanks.

Australian politicians are no better, they would reach for those tactics in a heartbeat, compulsory voting stops us from heading down that path. There is no utopia, the system has problems, but the trade-off is worth it.

1. Not the main thrust of OPs point.

2. Sweden's population density is 200th on this list of 250 countries:

https://en.wikipedia.org/wiki/List_of_countries_and_dependen...

> This argument does not make sense.

The OP compares Sweden to its closest neighbours

> +87% of the Swedish population live in urban area

As do +83% +85% and +88% of the three closest neighbours OP compared Sweden.

In what way does that comparison not make sense?

For kids only? I've been working with Kafka for a decade and found this to be an absolute delight.

Honestly I can't wait for perpetual topics backed by S3 integrated with MSK. If the AWS team aren't working on it I will eat my own shoes.

Let me know how you go with kPow! If you want a guided tour just let me know. :)

ps. we're releasing a major bump to the UI later today..

We build a complete drop in replacement for Confluent's metrics/management tooling (https://kpow.io) that fills a very large gap in the engineering experience for Kafka. I say replacement but tbh we offer a lot more in terms of features other than stopping/starting clusters.

It feels obvious to me that AWS will roll-out S3 backed storage and managed Kafka Connect at some point, their recent IAM / ACL integration points to a pretty active MSK team.

I completely agree in the past there were some pretty big gaps - that's why we built kPow - my feeling is those gaps are narrowing pretty significantly.

In my experience I found the ability to produce a similar functional deliverable (e.g. a webservice or similar) in Clojure as I was able to in my previous language (Java) was the most important factor in my learning and continuing with Clojure.

Guides like this give a quick-start into that world. The book-driven big-philosophy route might work better for some people, but to take your first example of "reduce" being special - well that's never really occurred to me as important or remarkable.

In time Clojure has transformed how I program computers, so the big ideas have settled in. I've never finished a Clojure book so I can't offer much reportage on their effectiveness.

We all learn differently, I guess that's the point.

I build kPow[0], a toolkit for Apache Kafka.

I've worked with Kafka since 2012, ran a consultancy delivering systems built with Kafka and Cassandra for a while, and eventually realised our experience could be wrapped up into a great tool that covers everything Kafka from clusters to schema registries, kafka connect and more.

There was literally nothing in the market when I started, so I built what we (and our customers) needed. Originally kPow was going to be a replacement for Datastax OpsCenter but it became pretty obvious in around 2017 that Kafka was going to be more widely adopted (and tbh I love the streaming compute paradigm / kafka streams / etc).

I work with another engineer now and we spend all of our time chatting with users, shipping features, working in Clojure[1], it's a blast!

[0] https://kpow.io

[1] https://www.youtube.com/watch?v=MnvtPzEH-d8

Confluent Cloud has some nice point-and-click UI for creating associated Kafka resources like Schema Registries and Connect Clusters.

My preference is MSK but I'm very comfortable with vanilla Kafka in AWS at a good price with auto-updates.

I've been working commercially with Clojure for ten years.

My gut feeling is the language is well subscribed and growing at a flat rate. That's fine by me.

#(mapv (juxt (comp str first) count) (partition-by identity %))

This is exactly how we shipped our JQ-like product feature in 62 lines of Clojure. Just pure functions composed into a single predicate and and a decent grammar to interpret the input JQ.

We recently implemented a subset of JQ in Clojure, converting filters and functions into Clojure predicates to allow our users to apply (K)JQ queries directly to Kafka topics holding JSON-like data (e.g Avro, Json, Transit, etc). If it's maps or vectors it works.

The basic implementation is predicate composition as we support multiple combinations of filters.

I hadn't thought any deeper about 'traversal systems' or optics, just parsing a grammar, creating predicates, and composing them. This is an interesting read, perhaps I should take the Clojure impl. further.

https://www.youtube.com/watch?v=7krDIMjVzZM

Probably the most fun coding I've done this year because I had an excuse to use the wonderful Instaparse again.

Be sure to let me know when you launch, I'd love to see what you've built. Good luck!

I founded operatr.io. Our product, static site, and licensing systems are entirely built in CLJ/CLJS. JVM, brower, lambda, etc.

Like JWR I feel there's absolutely zero chance we could have built this product as effectively in my previous primary JVM languages. Clojure is just a better toolset, no comparison.

We recently enquired with Github about sponsoring as an organisation (rather than me using my personal account) and have been added to the waitlist for the alpha of organisation-sponsoring. Once that starts we'll contribute financially to the developers in our space who enable our business.

Also as a habit I send personal emails to open source contributors simply to thank them in person.

Good luck with your launch!

P.s. I'm very excited for the team at Cognitect. It's hard to start a consultancy, hard to launch a product, there are no guarantees of success, and they deserve all of it.

I'm not sure how indigenous cultures are represented in North America today but the map shows Iwi in New Zealand that are present day, not a historical artefact.