4,430 karma · joined October 6, 2009
Co-founder of CircuitLab (YC W13): https://www.circuitlab.com/
Formerly I ran Growth at Triplebyte (YC S15): https://triplebyte.com/
Heii On-Call will keep alerting you with these “Critical Alerts” until you’ve manually acknowledged. (Or until it escalates to a teammate and they acknowledge…)
And at least on my phone they sound nothing like normal phone notifications, which I personally always have on vibrate and/or DND anyway.
Give it a try and I think you’ll like it.
We use iOS “Critical Alerts” and similar on Android that breaks through any Do-Not-Disturb settings. https://heiioncall.com/blog/better-alerting-for-heii-on-call... Would you be willing to give that a shot? It wakes me every time :)
(It’s configurable too; we have vibrate-only or silenced modes. Think old-school beeper.)
In the rare case that it doesn’t wake you, we have configurable escalation strategies to alert someone else on your team after a configurable number of minutes.
At Heii On-Call https://heiioncall.com/ we use:
ENV RUBY_YJIT_ENABLE=1
ENV RUBYOPT=--yjit-exec-mem-size=16
in our Dockerfile for our Rails processes.At Heii On-Call https://heiioncall.com/ we are using Active Admin https://activeadmin.info/ for Ruby on Rails, which seems quite similar to the Django admin. In my experience, it's mostly useful as a fairly basic read-only view of what's in the database. In Rails, it's so easy to whip together a custom view that we tend to do that, and the Active Admin is nice to have but I wouldn't say "tremendous value".
The Hacker News API is still up: https://github.com/HackerNews/API https://hacker-news.firebaseio.com/v0/topstories.json?print=...
To @dang or whoever is reading at Algolia: you can set up continuous monitoring for problems at the DNS/TCP/TLS/HTTP layers from just your URL at https://heiioncall.com/ and we’d definitely alert you if the DNS record disappears or it’s otherwise unreachable for more than X minutes. (Email me if you need help getting it set up.)
Edit: the subtle difference from the article is that it sounds like you want historical data, rather than present-state data.
(Agree that the “WHEN NOT MATCHED DO NOTHING” is effectively the default behavior.)
Use case is the Outbound Prober processes for https://heiioncall.com/ which can do thousands of HTTP requests per second, then aggregate all those row updates into a single query (or single transaction, which in practice is a single query).
We can do these batched updates very, very fast using INSERT … ON CONFLICT. No need to re-parse a differently shaped query, only a single index rebuild. Smooth :)
The "WHEN NOT MATCHED THEN DO NOTHING" is very appealing, and we currently implement this with a transaction and a DELETE after the upsert because we want to do updates only, not unintentionally insert new rows.
- Slow compile times are still a pain for iteration.
- The REPL / interpreter mode is still rough around the edges.
As far as companies using Crystal:
- We’re using it happily in production at Heii On-Call https://heiioncall.com/status
- Kagi is using it for their search engine backend https://news.ycombinator.com/item?id=32687071
- Other companies using it list: https://crystal-lang.org/used_in_prod/
How is this better than simply working on a cash basis, letting companies take the loss in year 1 when they build X, and then, for tax purposes, carry forward those net losses against the future revenues? Sure seems much simpler.
The constant churn of protocol deprecations, certificate expirations, rotations, etc. is like planned obsolescence on steroids.
This is a subtle but important distinction…
There are so many layers now between a user and the application code. What if due to some misconfiguration or new image push or ___ the web server or load balancer or PaaS router or CDN or Cloudflare or whatever starts serving some default placeholder, or error message, or its own content up on my URL?
That’s why I’d argue for a non-200 status code for the default “hello” page.
And in production monitoring I’d use something like https://heiioncall.com/blog/enhanced-api-monitoring-with-exp... to verify the presence of some special header set only by your application, so you know that your desired code is actually being called. (In addition to asserting the HTTP status code.)
[1] https://heiioncall.com/blog/enhanced-api-monitoring-with-exp...
These problems resonate with me for sure, and after tackling them at other companies, they were part of our team’s inspiration for building Heii On-Call [1] as a lightweight, minimal monitoring platform for apps / APIs / Cron Jobs / etc.
Instead of “ingest all the metrics/logs” we have guides showing, for example, how to set up the most minimal possible Prometheus / Alertmanager, with the smallest footprint of CPU/memory/disk requirements. [2]
And for many of our users they don’t even use any of our Prometheus or Datadog integrations, and just stick with our dead simple out-of-the-box HTTP monitoring. [3]
Note that these address the “now” problem of monitoring: is it working now, or not? While most of the article talks about the “past” problem of monitoring (long term storage of logs, metrics; pretty charts).
The “past” part is the part that the author is pointing to as being rarely useful and the source of the cost and complexity. I think there’s probably still value in dumping whatever logs and metrics you want into S3 just in case you need to go digging one day, but for the most part, this is pretty low value. The relatively higher value and lower implementation complexity is in the “now”.
[2] https://heiioncall.com/guides/minimalistic-monitoring-and-al...