Bad: 0.0579 - Good: 0.9427
2,496 karma · joined September 28, 2015
Bad: 0.0579 - Good: 0.9427
A strategy to find interesting papers that are worth reading as a novice in the field, is to pick any current paper that is interesting to you and look for heavily cited references. The following websites have citation data for papers:
[1] SemanticScholar.com
[2] GoogleScholar.com
[3] citeseerx.ist.psu.edu
I was looking at exactly that 6T figure, and was like: This is an AWEFUL LOT for a country with less than 20 Million people.
- Are you doing docker in a Linux VM?
- Any graphical Applications other than Firefox and emacs?
Wow. This is mind blowing. Thanks for the pointer.
I find this to be a dangerous oversimplification. I have never met a single "bad person".
All humans are driven by desires, fears within the constraints of their environment. Some of decisions turn for the better, others for the worse. All criminal activity I have seen comes from people who have manoeuvred themselves into bad situations, from which they can't entangle. So they resort to violence or other kinds of criminal activity in order to bring themselves to a seemingly better place. I don't want to make excuses for anyone, but the idea that someone is a "bad" or "good" person, just does not make any sense.
If you are looking for a monitoring vendor, who deeply cares about getting the the statistics right (especially around aggregating and analysing latency data), have a look at https://circonus.com / https://lps.circonus.com/statistics-for-engineers/ and/or reach out to me.
PM me your email, and I'll put you on the mailing list.
Are you aware of people in the IT-Ops domain who use control charts?
This article is quite dated. I ran the Statistics for Engineers class at various conferences over the years, and updated the material. I literally just did a session at SRECon EMEA today [1]!
The course material is here: https://github.com/HeinrichHartmann/Statistics-for-Engineers...
Todays version includes new material about:
- How averaging percentiles breaks down
- How sub-sampling affects percentile calculations
- Comparison of "mergeable aggregation methods" like HDR Histograms, t-digest, etc.
If you liked the article, make sure to check out the github course material. It's much broader and more up-to-date.
[1] https://www.usenix.org/conference/srecon19emea/presentation/...
I find the business model very interesting: A kind of "middle layer" SAAS, where you provide a new front-end for an existing service. Not seen that very often. Certainly helps with the data privacy issues. Rapid on-boarding is another immediate benefit.
I hate reading stuff on screens.
- It hurts my eyes when I do this for hours.
- It comes with lots of distractions.
- Digital Annotations are a PITA
- Zapping through a book works much better in hard-copy
If I am supposed to spent 1h+ reading your content. You better make sure I can buy a printed copy.
https://en.wikipedia.org/wiki/Parsing_expression_grammar
In particular this lua implementation:
http://www.inf.puc-rio.br/~roberto/lpeg/re.html
It let's you write stuff like this:
list <- (name s)*
name <- [a-z][a-z]*
s <- %s*
Features:- Looks like EBNF
- Regular expressions are a subset
- Fast parser.
> Diamond open access, free-of-charge to authors. The articles are accessible to everyone, forever, and the authors don’t need to pay any fee.
> All articles are published under a Creative Commons license.
> The journal is an overlay on arXiv, meaning that we upload all articles to arXiv as a long-term storage system.
Just as it should be. If I write a relevant publication, this will be on the top of my list. At this point, publishing with folks like Elsivier should be considered hostile and anti-science e.g. [1].
[1] http://heinrichhartmann.com/blog/2013/01/17/Why-Openness-ben...
GNU Stow (already mentioned here) and NIX are examples.
Less than that, with the appropriate compression scheme (sparse encoding).
> But then I got completely lost in the math to show why this guarantees anything.
You get guarantees on _relative_ error:
- 100 to 110 is a 10% error
- 1000 to 1010 is a 1% error
Depending on your logarithmic base, you can detect errors up to a certain relative %-tage.
(1) HDR-Histogram merges are 100% accurate and very fast (few microseconds)
(2) The range of HDR-Histogram is bounded in the same way that floating point numbers are bounded. Hence the name "High Definition Range":
> For example, a Histogram could be configured to track the counts of observed integer values between 0 and 3,600,000,000,000 while maintaining a value precision of 3 significant digits across that range.
http://hdrhistogram.github.io/HdrHistogram/
Circllhist offers a default range of 10^-128 .. 10^+127, which has been more than ample for all use-cases I have seen.
https://github.com/circonus-labs/libcircllhist/blob/master/s...
A note on Accuracy: At Circonus, we have been using a version of HDR-Histograms [1] for many years to aggregate latency distributions, and calculate accurate aggregated percentiles. Accuracy was never a problem (worst-case error <5%, usually _much_ better).
If I read your evaluation results correctly, you also found HRD-Histograms to be as-accurate or more-accurate, than DDSketches, correct?
The differentiator to HDR Histograms seems to be merging speed and size, where DDSketches seem to have an edge.
One thing that is not immediately clear to me from reading the paper is, how much of the distribution function can be reconstructed from the sketch? E.g. for SLO calculations one is often interested in latency bands: "How many requests were faster than 100ms?" [2].
Is it possible to approximate CDF values ("lower counts") from the sketch with low/bounded error?
[1] https://github.com/circonus-labs/libcircllhist
[2] http://heinrichhartmann.com/pdf/Heinrich%20Hartmann%20-%20La...
Sure, they invested time and money into into it, but they have a JOB which should pay their rent. Pretty rude for them to ask the audience to contribute back.
#sarcasm
cat payload.json | curl $url --data -@-
You can use this to re-play access logs, like so: sed -n 23p accesslog.json | jq '.args' | curl $endpoint --data -@-I have tried that style in the past, and have reverted to writing longer functions with intermittent comments. I now consider "single caller functions" a smell.
If logic is serial (do A, then B, then C) it's fine to have serial code:
main() {
... A ...
... B ...
... C ...
}
The issue with splitting out the intermediate steps into functions is, that the serial flow is broken.
Instead of the logic now looks something like A(...){ ... }
C(...){ ... }
main() {
A(...)
... B ...
C(...)
}
Where B might have been a tiny task, that is not worth splitting out.
In effect:- The ordering of the logic is broken.
- Passing arguments can be a pain, if a lot of pieces are touched.
And what have you gained? Not sure there are many upsides besides debatable aesthetics of avoiding inline code comments.
That's true. But look at the overall coding style in that file:
Code is structured into cohesive blocks, which are prefixed by a comment which explains in english language the intention, pitfalls, limitations of the following code.
I find this quite elegant and readable. An ideal to aspire to.
I am not advocating nonsensical JavaDoc:
// Get's foo from bar
private Foo get_foo(Bar bar);
If you are able to just use names and types to document your intentions, pitfalls etc ... then by all means, do just this.
E.g. for( person in address_book ) {
body = template.render( name => person.name, ... )
mail = Mail.new(person.email, body)
mailgun.queue(mail)
}
However, if stuff get's more nuanced don't be afraid to use english language to explain what you are doing.
You future self will thank you : )My 2 cents: There are two things to consider:
1. reviewability
2. deployment risk
If it takes a colleague 3 days to review your code, your PR is too big. If you panic, on the thought of deploying this ever, your PR is too big.
On the other side:
- Huge PRs that only change formatting are fine. Easy review. Low risk if properly automated/tested.
- Largs PRs that are feature neutral are acceptable as long as they are reviewable.
- PRs that refactor 2000LOC, fix 2 bugs and add 3 features are not a good idea.
> Comments: I find comments, outside of docstrings, a smell.
But is this really the case? I find comments like these invaluable:
/*
* Mark walreceiver as running in shared memory.
*
* Do this as early as possible, so that if we fail later on, we'll set
* state to STOPPED. If we die before this, the startup process will keep
* waiting for us to start up, until it times out.
*/
SpinLockAcquire(&walrcv->mutex);
https://github.com/postgres/postgres/blob/master/src/backend...Part of the job of a developer is to communicate
- domain model (Concepts, Relations between them)
- intention behind the code (Why is this code here?)
- possible pitfalls (I tried this, it does not work because ...)
- limitations (TODO: This special case does not really work, yet)
for your colleagues and future heirs.
How would you ever cramp information like this into variable names and types? (... without introducing so much abstractions that the code becomes unreadable).
> Among all the sites I visited, news sites, including The New York Times and The Washington Post, had the most tracking resources.
So hat-tip for the self-awareness. Now how about "sweeping your doorstep" ?
I don't understand that argument. By the same reasoning all drugs should be legal:
Everyone know's it's bad. People get trapped. It has severe consequences for personal health and the society as a whole. But, hey, "it's not impossible to break free from it" by yourself. No need for Government action.
The key metric for me is impact to society. You can do all bad things to you if you want to, and they only impact yourself, but if things have large impact on:
- Social fabric (stability of families and communities)
- Political Culture (erosion of discourse culture)
- Education (lower attention span)
- Healthcare (burnout!)
- Working productivity
It might be a good idea to change the rules to prevent these effects.
* As a starter: The source code is only a JS snippet. No HTML in there.
* Right away I "fullstory.com" appear in the JS. So I have reason to believe that all user interactions (mouse, keyboard, etc.) are uploaded to fullstory.com. Not sure this is actually the case. Certainly not in favour of it.
* In the network tab, I see requests to heapanalytics.io.
Setting aside the question if such data use is necessary, appropriate, or done elsewhere. I wonder if this is actually legal. I certainly did not consciously agree to my data being collected in this way and shared with third parties.
When you scroll down, you eventually see a Privacy Policy and TOS links in grey:
> By using this site you agree to the [Terms and Conditions] and the [Privacy Policy].
- Is a text like this actually effective?
- Am I violating EU law, if I use the same patterns in my Apps? (I am EU based.)
Yes. This was the case I had in mind. Path to madness.