HNHacker News
TopNewBestAskShowJobs

heinrichhartman

2,496 karma · joined September 28, 2015

Blog: HeinrichHartmann.com Keybase: https://keybase.io/hartmann Twitter: @HeinrichHartman http://twitter.com/Heinrichhartman GitHub: https://github.com/HeinrichHartmann/
submissionscomments
heinrichhartman··on Show HN: Testing HN titles against a neural network
> Paul Graham hates LISP

Bad: 0.0579 - Good: 0.9427

heinrichhartman··on File systems unfit as distributed storage back ends: lessons from Ceph evolution
ACM subscription helps, since their "Digital Library" has pretty good coverage of the CS literature. This is something that your employer might want to sponsor.

A strategy to find interesting papers that are worth reading as a novice in the field, is to pick any current paper that is interesting to you and look for heavily cited references. The following websites have citation data for papers:

[1] SemanticScholar.com

[2] GoogleScholar.com

[3] citeseerx.ist.psu.edu

heinrichhartman··on The world is sitting on a $400T financial time bomb (2017)
"terrible" is the wrong word here. I far prefer absolute values, to arbitrary "corrected" values that are harder to interpret.

I was looking at exactly that 6T figure, and was like: This is an AWEFUL LOT for a country with less than 20 Million people.

heinrichhartman··on OpenBSD 6.6
Very cool! What kind of software do you use on your machines?

- Are you doing docker in a Linux VM?

- Any graphical Applications other than Firefox and emacs?

heinrichhartman··on Let's Write an LLVM Specializer for Python (2015)
> In this talk, I live-code a simple stack machine and turn it into an interpreter capable of running Web Assembly. I then use that to play a game written in Rust.

Wow. This is mind blowing. Thanks for the pointer.

heinrichhartman··on British holidaymakers 'traumatised' after arrest at US border
I see this a lot in the US: The "Good Guy" vs. "Bad guy" theme is very commonly used, and portrayed news, politics, film, etc.

I find this to be a dangerous oversimplification. I have never met a single "bad person".

All humans are driven by desires, fears within the constraints of their environment. Some of decisions turn for the better, others for the worse. All criminal activity I have seen comes from people who have manoeuvred themselves into bad situations, from which they can't entangle. So they resort to violence or other kinds of criminal activity in order to bring themselves to a seemingly better place. I don't want to make excuses for anyone, but the idea that someone is a "bad" or "good" person, just does not make any sense.

heinrichhartman··on Statistics for Engineers: Applying statistical techniques to operations (2016)
PS: At Circonus we have built technology that is explicitly addressing many of the statistical pitfalls that are explained in the post.

If you are looking for a monitoring vendor, who deeply cares about getting the the statistics right (especially around aggregating and analysing latency data), have a look at https://circonus.com / https://lps.circonus.com/statistics-for-engineers/ and/or reach out to me.

heinrichhartman··on Statistics for Engineers: Applying statistical techniques to operations (2016)
Most certainly yes. I am planning on proposing it again for next years conference season. In particular SRECon EMEA 2020 in Amsterdam comes to mind.

PM me your email, and I'll put you on the mailing list.

heinrichhartman··on Statistics for Engineers: Applying statistical techniques to operations (2016)
Interesting pointer. I had looked at the Control Theory literature. We used the CUSUM "control chart" as an internal bit in Anomaly Detection method, we built a while ago. This had limited success. Usually the data is just too noisy. Industrial sensors tend to be much better behaved than stuff tha you get from the Linux kernel.

Are you aware of people in the IT-Ops domain who use control charts?

heinrichhartman··on Statistics for Engineers: Applying statistical techniques to operations (2016)
Author here. Glad to see this on HN!

This article is quite dated. I ran the Statistics for Engineers class at various conferences over the years, and updated the material. I literally just did a session at SRECon EMEA today [1]!

The course material is here: https://github.com/HeinrichHartmann/Statistics-for-Engineers...

Todays version includes new material about:

- How averaging percentiles breaks down

- How sub-sampling affects percentile calculations

- Comparison of "mergeable aggregation methods" like HDR Histograms, t-digest, etc.

If you liked the article, make sure to check out the github course material. It's much broader and more up-to-date.

[1] https://www.usenix.org/conference/srecon19emea/presentation/...

heinrichhartman··on Launch HN: Quilt (YC W16) – A versioned data portal for S3
One use-case is this one: https://github.com/HeinrichHartmann/arxiv_meta
heinrichhartman··on Launch HN: Quilt (YC W16) – A versioned data portal for S3
Excited to see this being re-launched. "git for data" ranks pretty high on my all time list of tech I want to see succeed.

I find the business model very interesting: A kind of "middle layer" SAAS, where you provide a new front-end for an existing service. Not seen that very often. Certainly helps with the data privacy issues. Rapid on-boarding is another immediate benefit.

heinrichhartman··on Where Theory Meets Chalk, Dust Flies
Can we add a tag [paywall] to these posts, so I don't waste my time clicking on them?
heinrichhartman··on E-books at local libraries: a fight between publishers and librarians
I feel exactly the opposite.

I hate reading stuff on screens.

- It hurts my eyes when I do this for hours.

- It comes with lots of distractions.

- Digital Annotations are a PITA

- Zapping through a book works much better in hard-copy

If I am supposed to spent 1h+ reading your content. You better make sure I can buy a printed copy.

heinrichhartman··on Is there a regular expression to detect a valid regular expression?
I have been a happy user of PEG grammars for a while:

https://en.wikipedia.org/wiki/Parsing_expression_grammar

In particular this lua implementation:

http://www.inf.puc-rio.br/~roberto/lpeg/re.html

It let's you write stuff like this:

      list <- (name s)*
      name <- [a-z][a-z]*
      s <- %s*
Features:

- Looks like EBNF

- Regular expressions are a subset

- Fast parser.

heinrichhartman··on Reference Capabilities for Safe Parallel Array Programming
This is a very nice Journal! https://programming-journal.org/

> Diamond open access, free-of-charge to authors. The articles are accessible to everyone, forever, and the authors don’t need to pay any fee.

> All articles are published under a Creative Commons license.

> The journal is an overlay on arXiv, meaning that we upload all articles to arXiv as a long-term storage system.

Just as it should be. If I write a relevant publication, this will be on the top of my list. At this point, publishing with folks like Elsivier should be considered hostile and anti-science e.g. [1].

[1] http://heinrichhartmann.com/blog/2013/01/17/Why-Openness-ben...

heinrichhartman··on Dropbox no longer follows symlinks to items outside of your Dropbox account
It's actually not an anti-pattern, to symlic files to their install locations. It's quite common to install executables to custom locations like `/opt/$company/bin` or `/opt/$appname/bin` and only symlink to /local/bin, ~/bin, etc.

GNU Stow (already mentioned here) and NIX are examples.

heinrichhartman··on DDSketch: A fast, fully-mergeable quantile sketch with relative-error guarantees
> number of bins in terms of memory

Less than that, with the appropriate compression scheme (sparse encoding).

> But then I got completely lost in the math to show why this guarantees anything.

You get guarantees on _relative_ error:

- 100 to 110 is a 10% error

- 1000 to 1010 is a 1% error

Depending on your logarithmic base, you can detect errors up to a certain relative %-tage.

heinrichhartman··on DDSketch: A fast, fully-mergeable quantile sketch with relative-error guarantees
Sorry, but I don't follow this argument:

(1) HDR-Histogram merges are 100% accurate and very fast (few microseconds)

(2) The range of HDR-Histogram is bounded in the same way that floating point numbers are bounded. Hence the name "High Definition Range":

> For example, a Histogram could be configured to track the counts of observed integer values between 0 and 3,600,000,000,000 while maintaining a value precision of 3 significant digits across that range.

http://hdrhistogram.github.io/HdrHistogram/

Circllhist offers a default range of 10^-128 .. 10^+127, which has been more than ample for all use-cases I have seen.

https://github.com/circonus-labs/libcircllhist/blob/master/s...

heinrichhartman··on DDSketch: A fast, fully-mergeable quantile sketch with relative-error guarantees
Nice work! Averaging percentiles is well-known to give terrible results. Glad to see more people, taking this problem serious, and providing viable alternatives!

A note on Accuracy: At Circonus, we have been using a version of HDR-Histograms [1] for many years to aggregate latency distributions, and calculate accurate aggregated percentiles. Accuracy was never a problem (worst-case error <5%, usually _much_ better).

If I read your evaluation results correctly, you also found HRD-Histograms to be as-accurate or more-accurate, than DDSketches, correct?

The differentiator to HDR Histograms seems to be merging speed and size, where DDSketches seem to have an edge.

One thing that is not immediately clear to me from reading the paper is, how much of the distribution function can be reconstructed from the sketch? E.g. for SLO calculations one is often interested in latency bands: "How many requests were faster than 100ms?" [2].

Is it possible to approximate CDF values ("lower counts") from the sketch with low/bounded error?

[1] https://github.com/circonus-labs/libcircllhist

[2] http://heinrichhartmann.com/pdf/Heinrich%20Hartmann%20-%20La...

heinrichhartman··on WTF – A personal information dashboard for your terminal
yes, exactly. Why do musicians, who have a job (e.g. as a teacher) charge for concerts they give in their private time?

Sure, they invested time and money into into it, but they have a JOB which should pay their rent. Pretty rude for them to ask the audience to contribute back.

#sarcasm

heinrichhartman··on Curl exercises
This one is handy, if you want to create your payloads dynamically:

    cat payload.json | curl $url --data -@-
You can use this to re-play access logs, like so:

    sed -n 23p accesslog.json | jq '.args' | curl $endpoint --data -@-
heinrichhartman··on Things I Learnt from a Senior Software Engineer
> I would want that function to be multiple smaller functions with docstrings and such [...]

I have tried that style in the past, and have reverted to writing longer functions with intermittent comments. I now consider "single caller functions" a smell.

If logic is serial (do A, then B, then C) it's fine to have serial code:

   main() {
     ... A ...

     ... B ...

     ... C ...
   }
The issue with splitting out the intermediate steps into functions is, that the serial flow is broken. Instead of the logic now looks something like

    A(...){ ... }
    C(...){ ... }
    main() { 
       A(...)
       ... B ...
       C(...)
    }
Where B might have been a tiny task, that is not worth splitting out. In effect:

- The ordering of the logic is broken.

- Passing arguments can be a pain, if a lot of pieces are touched.

And what have you gained? Not sure there are many upsides besides debatable aesthetics of avoiding inline code comments.

heinrichhartman··on Things I Learnt from a Senior Software Engineer
> Extremely high-performance database code is not what most of us are doing, however.

That's true. But look at the overall coding style in that file:

Code is structured into cohesive blocks, which are prefixed by a comment which explains in english language the intention, pitfalls, limitations of the following code.

I find this quite elegant and readable. An ideal to aspire to.

I am not advocating nonsensical JavaDoc:

    // Get's foo from bar
    private Foo get_foo(Bar bar);
If you are able to just use names and types to document your intentions, pitfalls etc ... then by all means, do just this. E.g.

     for( person in address_book ) {
          body = template.render( name => person.name, ... )
          mail = Mail.new(person.email, body)
          mailgun.queue(mail)
     }

However, if stuff get's more nuanced don't be afraid to use english language to explain what you are doing. You future self will thank you : )
heinrichhartman··on Things I Learnt from a Senior Software Engineer
> When refactoring and preventing huge-ass PRs: “If I’d have changed all the tests first then I would have seen I had 52 files to change and that was obviously gonna be too big but I was messing with the code first and not the tests.” Is breaking it up worth it?

My 2 cents: There are two things to consider:

1. reviewability

2. deployment risk

If it takes a colleague 3 days to review your code, your PR is too big. If you panic, on the thought of deploying this ever, your PR is too big.

On the other side:

- Huge PRs that only change formatting are fine. Easy review. Low risk if properly automated/tested.

- Largs PRs that are feature neutral are acceptable as long as they are reviewable.

- PRs that refactor 2000LOC, fix 2 bugs and add 3 features are not a good idea.

heinrichhartman··on Things I Learnt from a Senior Software Engineer
I have heard this so many times:

> Comments: I find comments, outside of docstrings, a smell.

But is this really the case? I find comments like these invaluable:

     /*
      * Mark walreceiver as running in shared memory.
      *
      * Do this as early as possible, so that if we fail later on, we'll set
      * state to STOPPED. If we die before this, the startup process will keep
      * waiting for us to start up, until it times out.
      */
     SpinLockAcquire(&walrcv->mutex);

https://github.com/postgres/postgres/blob/master/src/backend...

Part of the job of a developer is to communicate

- domain model (Concepts, Relations between them)

- intention behind the code (Why is this code here?)

- possible pitfalls (I tried this, it does not work because ...)

- limitations (TODO: This special case does not really work, yet)

for your colleagues and future heirs.

How would you ever cramp information like this into variable names and types? (... without introducing so much abstractions that the code becomes unreadable).

heinrichhartman··on I Visited 49 Sites. Hundreds of Trackers Followed Me
So ironic that the NYTimes, where this article is published, is itself one of the worst offenders. The article clearly states this as well:

> Among all the sites I visited, news sites, including The New York Times and The Washington Post, had the most tracking resources.

So hat-tip for the self-awareness. Now how about "sweeping your doorstep" ?

heinrichhartman··on America’s social-media addiction is getting worse
> but I don't really support the government saving us from ourselves.

I don't understand that argument. By the same reasoning all drugs should be legal:

Everyone know's it's bad. People get trapped. It has severe consequences for personal health and the society as a whole. But, hey, "it's not impossible to break free from it" by yourself. No need for Government action.

The key metric for me is impact to society. You can do all bad things to you if you want to, and they only impact yourself, but if things have large impact on:

- Social fabric (stability of families and communities)

- Political Culture (erosion of discourse culture)

- Education (lower attention span)

- Healthcare (burnout!)

- Working productivity

It might be a good idea to change the rules to prevent these effects.

heinrichhartman··on Show HN: A time tracker that asks you every 10 min what you're doing
I like the app, but I am a bit confused by the data usage.

* As a starter: The source code is only a JS snippet. No HTML in there.

* Right away I "fullstory.com" appear in the JS. So I have reason to believe that all user interactions (mouse, keyboard, etc.) are uploaded to fullstory.com. Not sure this is actually the case. Certainly not in favour of it.

* In the network tab, I see requests to heapanalytics.io.

Setting aside the question if such data use is necessary, appropriate, or done elsewhere. I wonder if this is actually legal. I certainly did not consciously agree to my data being collected in this way and shared with third parties.

When you scroll down, you eventually see a Privacy Policy and TOS links in grey:

> By using this site you agree to the [Terms and Conditions] and the [Privacy Policy].

- Is a text like this actually effective?

- Am I violating EU law, if I use the same patterns in my Apps? (I am EU based.)

heinrichhartman··on GitHub is down
> Same applies to master/release branches.

Yes. This was the case I had in mind. Path to madness.

← PreviousPage 9 of 15Next →