newcrobuzon

K8s is amazing piece of tech, but still there is a big cost and complexity inherent in adopting & managing it.

I enjoy using it and playing with it, but so many use cases can be addressed with something simpler - either just Docker / Swarm / AWS ECS etc. alternatives or just going for VMs with well defined CI/CD processes that let you tear down the infrastructure and set it up again easily.

What very interests me are the concepts K8s build on that are not usually recognized - to me K8s seems a lot like a JVM, just that it operates on infrastructural (and not runtime) level.

I enjoy experimenting with these concepts when applied back in the runtime world - it is for instance interesting to run 100s of servers with JVM and let them load/execute new dependencies and code at runtime (JVM is very well suited for that).

This is area that is not yet explored and would probably deserve more attention as it allows for distributed rapid computing that is infrastructure/platform independent (the downside is that it requires (just) JVM and the isolation is not perfect).

Options should be used as intended: as a hedge.

E.g. If I am net long in my portfolio and I fear some headwinds I can buy a put or two for the peace of mind. Now those puts should be always considered as worthless, and it is just the price to pay for the peace of mind.

Similarly you sell options. Trading options on the other hand is just pure gambling. Even if you get the direction right you likely won't get the timing right (or the volatility).

edit: typo

Many people don't know this, but you can still commit code that does not compile :D

> Sun spent like a billion dollars marketing Java

I guess now the question is if Nubank could/will also promote Clojure and to what extent (or if they will support Cognitect enough to do that). I would wonder if they went through that thought process and what RoIs for that are from their perspective.

It might even take much lesser investment than equivalent of $1bil from 90s... The Java/JVM market is huge so it might need just a little nudge :)

Within the JVM ecosystems it seems that basically Kotlin (as nice as it is) has been stealing some market share that might have belonged to Clojure. And all that just by selling some cheap cut syntax sugar on the corner.

There definitely is a huge desire within Java/JVM community to innovate and if approached correctly Clojure could actually shine - there are just few misconceptions and fears that could be put to sleep by some smart marketing strategy. Nubank is great first step in that direction, because it gives an example of a mature large yet innovative company relying on Clojure in production.

Cancer survivor of 20 years here. You sir seem to understand exactly how things are. This is how it is, not bad, not good, just the nature of life i guess.

Really appreciate your time looking into this and apologies, missed your original post. You got the answeres right. Re 2 - it shouldn't be that hard to add, since everything is just data. It is however partially covered by the retry property on each step. Will read through your additional comments and will respond tomorrow (busy day, plus it's midnight here in Europe)! Cheers Miro

Thanks ramon! Appreciate your kind words! Feel free to shoot me an email or reach me on twitter and let's stay in touch!

Thanks for the kind words!

Yes, I would still be careful with passwords in the public beta.

It goes via SSL and you get a unique UUID URL so all-in-all it is kinda secured (plus time-to-live of the instance is only 3 hours which limits any time to hack it) but still this is not 100% secure and I would not recommend it for any kind of production use (including any use of passwords you dont want exposed).

The free instances are not (yet) password protected (other then the UUID) - this on the other hand is useful if you want to share the envrionment with somebody, just send them the link... This is just a beginning of the public hosting so I will need to think through further improvements that would go into the hosting, security-wise and other aspects as well.

My main objective at this stage for the free hosted instances was to give people way to play with Titnaoboa or quickly test something, especially if something is not working in their local environment (say dependencies) and they want quickly re-test in vanilla environment.

If you want to add aws secrets or what not I would suggest to just download/install your local version, it is super easy: https://github.com/mikub/titanoboa#installation

Or if you need a public IP to integration just grab the docker instance and spin it in your AWS ECS or something.

Thanks Neelesh for the link, I like the way of thinking with the focus on data.

It is similar with what I am seeing - i.e. lots of older integration systems are terrible with data simply because they force you to use some way of data modeling (e.g. their OOP data models, WSDLs/XSDs etc.) while the newer ones just rely on json which is good but can lack the (sometimes necessary) complexity. To do some data cleansing on the way then seems like an unachievable task (there is certainly such a thing as overdose on XSLT ;) ).

I also like the approach you took with centralized data dictionary - it certainly is something the industry might need, I would wonder how it impacts change management though (especially in bigger companies).

Wishing you good luck!

Ooops, good catch, I will fix that :)

Oh really? Wasn't aware of that. Thanks for the link!

Do you have any links to some blog posts discussing it (the "doing it right" approach)? I would agree that it definitely depends on the use case.

I will definitely check out Syncari - just opened the landing page and it looks great!

Thanks, wasn't aware of them!

If anybody's curious about this "public beta hosting - HN bear hug" performance load test:

I just (sequentially) restarted all 4 servers behind my 2 load balancers - seems that by reaching 100 containers in each geography the provisioning threads may have died after many (unsuccessful) retries, so need to restart the thread pools.

That is one more error I need to handle (or increasing my limit in AWS).

That is a fair criticism, I am aware of the AGPL license shortcomings.

I just picked up the more restrictive license at the beginning - being a sole funder and not working on this full time etc. I simply did not want somebody (e.g. a big company with a big team) grabbing my code along the way and running away with it.

Since now Titanoboa got to the shape I envisioned it to be in I am starting to focus more on adoption, so yes I am definitely thinking about switching to less restrictive license since it will probably help.

Also at the beginning I was not aware how badly AGPL is perceived (I always thought if it was good for Mongo it could work for me, but I may have been wrong).

I mostly considered naming to be a distraction from getting the real work done.

I also am probably not good with names as my other alternative was Megalodon (4 syllables) - I just wanted to have some megafauna name.

Happy to hear what you folks would suggest :)

I would agree.

The no-code is an illusion in the enterprise realm - before you know it, you are waist deep in the custom code.

No-code can really work only for small businesses imo.

I come from enterprise background and that is one of the reasons I built Titanoboa - to make something that makes it easy to rapidly prototype new integrations on the fly.

I summed up some of my thoughts on this topic here: https://www.titanoboa.io/repl.html

The main point I am trying to test with Titanoboa is this however:

State Machines <-> Process Designers is a spectrum and one product could handle the entire spectrum (or part of it).

Titanoboa makes it possible to pre-define workflow steps and make it "no code" while also making complex custom integrations possible from the same environment with the same concepts. Plus also distributed data handling is in the mix.

I guess now the challenge is how to market this versatility or whether it could create more confusion...

To be honest Prefect just got on my radar last week, so I will need to look at it more closely, I was not aware of them till then.

Obviously this is for JVM, plus I am not sure how Prefect addresses following two points I mainly focus on:

- make rapid prototyping of new steps possible during runtime, without any need to restart/redeploy

- focus on distributed processing, where in master-less Titanoboa cluster you can have pretty much any number of nodes

A bit similar, but on JVM. The main focus of Titanoboa is however as follows:

- make rapid prototyping of new steps possible during runtime, without any need to restart/redeploy

- focus on distributed processing, where in master-less Titanoboa cluster you can have pretty much any number of nodes

Approaching 100 containers cap in US, so if something does not work, dont be mad :)

yes, I am considering this and it should not be that hard (also Kotlin, Nashorn javascript...), but at this stage Titanoboa is just a one-man-show - so patience, I might get there ;)

Thanks! Will reach out when the traffic subsides :) Or feel free to shoot me an email, my email address is on the web page.

Ok, I have just reached 100 concurrent containers in Europe, which is max at this stage and I don't think I can change the aws limit on the fly (will check though) so I might start killing the oldest instances.

Apologies for inconvenience folks, but I am sure people rarely play with each instance for the whole 3 hours anyway at this stage.

Honestly did not expect this kind of "public beta" rollout, still surprised how well the service holds so far under the load.

Well yes, this is for JVM, so maven ecosystem, sorry.

I have been contemplating also doing a clone on node.js (titanoboa is written in clojure so it would just be a move from clojure to clojurescript), but I did hear that the npm ecosystem is a bit uninviting too...

What dependencies tools would you like to see?

This is a very good point.

My current line of thinking is: A workflow is a workflow is a workflow. Titanoboa is built the way that pretty much everything it shuffles is data - and either it can be big (Airflow) or the steps can have side effects (iPaaS/Zapier).

The "no code" is achievable since adding more predefined steps is not that hard (there's not that many at the moment though) - see for instance https://github.com/mikub/titanoboa/wiki/Getting-Started-with... - you dont code, you just fill in properties, so it is very straightforward, pretty much like Zappier.

My aim in the iPaaS space is more at the enterprise level, where the predefined steps won't do anymore and you have to custom develop them anyway - so in that are I think Titanoboa shines since you really can rapidly prototype steps on the fly, a bit like Repl.it and Zapier together...

I agree that it might be good to pick one audience, at this stage I am just experimenting if somehow I can market to both and don't mess up and confuse both groups.

Thanks! If it is the recapcha string then it is just an old public string for local testing.

But I would upvote you 100x or send you a medal or something if I could, this is really super nice of you letting me know!

You can check out https://github.com/mikub/titanoboa-tasklets And also https://github.com/mikub/titanoboa-sample-workflows

There are few blog posts I wrote about some sample workflows, e.g.:

https://github.com/mikub/titanoboa/wiki/Getting-Started-with...

https://www.titanoboa.io/using-titanoboa-for-it-automation.h...

https://www.titanoboa.io/using-titanoboa-for-cloud-integrati...

https://www.titanoboa.io/java-repl.html

I am using D3. AFAIK nobody has used it for worfklow visualization so far: https://www.titanoboa.io/visualizing-workflows.html

Good point, not a native speaker so I thought this would roughly mean the same.

The main message I wanted to convey was that this runs on JVM and multiple JVM languages (java and clojure) could be used.

Wow, this has blown up a bit, so I have added few more servers (now two servers on the West coast and two in Europe), but ultimately I think there is a limit cap of 100 Titanoboa instances in each geography in parallel.

So if we break that level please don't be mad if you don't get your instance :)

Instead give me a star on github and come back later :)

Cheers Miro