HNHacker News
TopNewBestAskShowJobs

derf_

2,803 karma · joined July 22, 2012

submissionscomments
derf_··on Why does MPEG Transport Stream still exist?
> I have had no problems with measuring NTP drift.

Yeah, their claim is just weird. RTP does not impose an accuracy requirement on its timestamps (despite the name "NTP timestamp" in the Sender Reports, they are not actually expected to be synchronized with an NTP source), but I am skeptical such requirements would be met in practice if they did exist. The author only talks about video, but audio is a much bigger problem: if you do not send the right number of samples to the sound card at the right rate, you are going to get annoying clicks and pops, and "dropping or duplicating" audio frames will only cause more problems. You do not need to send media for a day to have these issues. Oscillators are bad enough they show up in minutes, and WebRTC obviously has strategies for dealing with them.

> If you want all your tracks to be synced you need to mark them as one MediaStream.

More specifically, the underlying mechanism of giving tracks that need to be synchronized the same CNAME has existed in RTP since the original RFC 1889 from the year 1996. The timestamp wrapping does require some care to get right, but is basically a non-issue.

That said, a lot of WebRTC applications will not ask for synchronization because it necessarily introduces latency, and for interactive use cases you are often better served by sacrificing exact sync for lower latency (as long as latency is low enough, sync is never going to get too bad, anyway). But that is very different from saying the standards do not support it if you want it.

derf_··on Pro-cash movement warns that people could be losing more than they bargained for
In the 1974 political thriller The Parallax View, one of the characters boards a plane at LAX, and then purchases a ticket from the stewardess. It is the most jarring scene in the entire film.
derf_··on Why the brain’s connections to the body are crisscrossed
> I guess the circulatory system is like that too...

I have some news for you: https://en.wikipedia.org/wiki/Circle_of_Willis

derf_··on The Twitter API is now effectively unmaintained
With one memo, Stephen Elop [1] drove Nokia's market cap from its peak of over $57b in March of 2010 to a trough of less than $7b in July of 2012 [2].

[1] https://communities-dominate.blogs.com/brands/2011/08/coinin...

[2] https://www.macrotrends.net/stocks/charts/NOK/nokia/market-c...

derf_··on Qualcomm has open sourced its aptX and aptX HD encoders
https://hydrogenaud.io/index.php/topic,114566.msg1023511.htm...
derf_··on SVB depositors, investors tried to pull $42B Thursday
They relaxed the need for a capital conservation buffer, too. These actions were part of the Fed's response to the market panic at the onset of the pandemic, and you can read a bit about their reasoning in their own press release:

https://www.federalreserve.gov/newsevents/pressreleases/mone...

Capital requirements are complicated. There are different types of equity and assets are risk-weighted, but treasuries and the types of bonds SVB bought are generally given the lowest risk-weightings.

derf_··on The Inverse Jim Cramer ETF
I think you have misunderstood Buffett theory. Most (i.e., more than 50%) of actively managed funds underperform, but in aggregate the amount of underperformance across all funds is roughly equivalent to the management fees they charge. Imagine a bell curve centered around the overall market's returns, but shifted over by a few percent to account for the fees.

An inverse of all major actively managed funds will get you the same average performance before the fees (the same as the overall market's), but it won't dip into those managers' bank accounts and transfer their fees to you. It will charge its own fees. So it will underperform, too.

The management fee for this ETF, for example, is 1.2% per year.

derf_··on What Happens Now to All the Laid Off Tech Workers? [audio]
https://academic.oup.com/qje/article/135/4/1965/5858010
derf_··on ARM Assembly: ∞ Ways to Return (2017)
In 2010, I tried to use callgrind to profile a project on Arm, after having used it to great effect on x86, and discovered that because of the variety of ways to return (and call!) functions on Arm, callgrind was unable to reliably identify function call and return sites. It created cycles in the call graph and even failed to record a function's self measurements correctly (because it could not tell when you left that function).

The problem boiled down to the valgrind frontend code that splits things up into basic blocks being incapable of having an instruction be both a conditional jump and a function call / return at the same time. That never happens on x86, but of course this is possible (and totally normal) on 32-bit Arm. Sadly, I ran out of time to try to re-architect this code and had to move on to other projects.

Over 12 years later, it looks like it never did get fixed: https://bugs.kde.org/show_bug.cgi?id=252091

derf_··on Image Compression with Singular Value Decomposition
> Is there a smart way to store the basis vectors?

Consider, instead, compressing small blocks of the image instead of a whole image plane at once. Let's say 8x8.

Over such a small region, pixels are usually very highly correlated. You could model them statistically as an order-1 autoregressive process with a high correlation coefficient (e.g., x[i + 1] = x[i]*rho + noise, with rho >= 0.95 usually, separately in both the horizontal and vertical directions).

Then, computing the eigendecomposition of the 8x8 cross-correlation matrix C_{ij} = rho^|i - j| produces a set of common eigenvectors that you can use for all blocks (recall that the left and right singular vectors U and V of a matrix M are just the eigenvectors of M.M^T and M^T.M, respectively). So now instead of several very large vectors, you only need a single 8x8 matrix of basis vectors (because it's square and orthonormal, the inverse is just the transpose).

But wait. Let's take the limit as rho -> 1. It just so happens that the resulting eigenvectors converge to the Type 2 Discrete Cosine Transform [0]. So, in fact, you don't need to transmit the basis at all.

You can store the output of the transform as fixed point numbers with some arbitrary base (the choice controls the amount of compression: anything too small just gets rounded to zero). Maybe use run-length encoding as your generic compression, because you expect to have a lot of zeroes (visit them in a zig-zag order, to get the big values first and maximize the length of the runs of zeroes), with some Huffman coding of the actual symbols. Add a simple predictor for the (0,0) coefficient of each block (it winds up being the average of the pixel values, so it should be pretty similar from block to block).

Congratulations, you just invented JPEG.

[0] http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1672...

derf_··on Evidence that Adani Group has engaged in stock manipulation and accounting fraud
No, you still owe the stock, but if trading is halted or the stock's registration is revoked, you cannot close the position. That means that (a) you still owe (daily!) stock borrow fees, and (b) you are still required to keep collateral posted against the shares with your broker (because the stock is not trading, the last posted price can be considerably above zero, so this can tie up a non-trivial amount of money).

In recent memory, this has happened to Longfin Corp. [0], Cynk Technologies [1], China Biotics [2], and surely others. If it does happen before you can close out a short position, that's bad. You can be stuck that way for years [3].

[0] https://www.sec.gov/enforce/information-longfin-investors

[1] https://www.sec.gov/litigation/suspensions/2014/34-72594.pdf

[2] https://www.sec.gov/litigation/opinions/2013/34-70800.pdf

[3] https://www.barrons.com/articles/getting-caught-short-152306...

derf_··on Bitcoin falls under $16,000
> 1. Can you point any company in recent history that went through bankruptcy/auction to pay is retail investor their money back...

Hertz: https://www.cnn.com/2021/07/01/business/hertz-bankruptcy-car...

derf_··on Better CPU selection for Linux timer expiration
There is latency associated with coming out of deeper C-states (from hundreds of microseconds to milliseconds, depending on the transition, which may include L1 and L2 cache flushes, etc.), so it isn't just the power. It's faster, too.

It might reduce throughput of a single-threaded process, but perhaps not even that if it avoids bouncing cache lines around to different cores.

Remember, though, that almost all timers never reach expiration, anyway, so the real savings here is the 25% reduction in the time to set or cancel a timer. That work, if it ever gets done, is now deferred to when a CPU is about to go idle, which (by definition) means it has nothing better to do.

derf_··on The case for JPEG XL
AV1 does not use rANS in any form. The quoted sentence is simply incorrect. However, any necessary patent claims owned by Microsoft, an AOM member, would be freely licensed to everyone for use in AV1 implementations if it actually did use rANS. Which it doesn't.
derf_··on The case for JPEG XL
We created avenues for open source / community participation in AOM that did not require people to pay any money. Specifically, Mozilla sponsored the membership of VideoLAN, which in theory anyone could join, and a number of people did and do participate through that organization.
derf_··on J40: Independent, self-contained JPEG XL decoder
That is a common misunderstanding of first-to-file. Under first-to-file, prior art is absolutely still a defense, and much easier to prove than under first-to-invent. If you publicly disclose something before someone who invented, but did not disclose, the same thing, your publication counts as prior art. It is much easier (read: cheaper) to demonstrate that "X document was published on Y website on Z date" than it is to start doing discovery to compare private lab notebooks or whatever to decide who really invented the thing first.

First-to-file is mostly about simplifying the complicated litigation edge cases where two people claim to have invented, but not disclosed, the same thing during the (US-specific) one-year grace period they have to file a patent on it.

Prior art may still not invalidate the patent, though. Usually the patent holder will argue that their patent differs from the prior art in some respect (you can often find these arguments in the patent's file wrapper at http://patentcenter.uspto.gov/). But that narrows the scope of the patent, and being your own prior art is a pretty good defense against infringement.

derf_··on RFC 9293: Transmission Control Protocol (TCP)
In the IETF's case, this is because, by policy, RFCs are immutable once published. You can update an RFC by publishing a new RFC that specifies the changes, or a new one that obsoletes the original entirely (this document does both to various RFCs). There is also an errata process for simple mistakes.

That means that minor changes tend to accumulate in other documents over the years. You could imagine tooling that automatically incorporates these changes into the presented document, like you suggest, but someone would have to build that tooling. That is a bit challenging, as the format of an RFC that updates another is not designed for such automated processing (e.g., go read this document and tell me what changes should be made to the text of RFC 5961, which it updates), and because the format of a published RFC is not semantic (e.g., adding a new section will not automatically renumber the remaining sections, because it's just a .txt file).

You could also design a different policy, but immutability has its advantages, too, and the current policy has been in use for decades, so the case for switching would have to be pretty compelling.

So for now the "Updated by:" header at the top of each document is what you get.

derf_··on So you have decided to start a free software consultancy
> I've had more than one customer argue with me over the years that they would rather pay hourly than a fixed price (however that fixed price was calculated)

Hello, I have been one of these people.

If I agree to a fixed price, then the goal of the contractor becomes "do the minimal amount of work possible to deliver something that can be argued to meet the letter of the contract (and no more)."

If I agree to an hourly rate, then the goal becomes "bill as many hours as possible while demonstrating at least some plausible incremental value for those hours."

Unless I am _really_ good at specifying a precise deliverable (and who among us in the software world is?), the second set of incentives looks much healthier, both for me and for the contractor. I think this is particularly true for open source deliverables. Maybe it costs more, but bounding that precisely is easy ("cannot charge more than X hours in Y period").

I don't want to micromanage your hours. I don't have time for that. Mostly the fact that when you send me an invoice you might expect the question of "What did you actually do?" to come up is sufficient to make sure you really did something (and if not, pretty easy to resolve).

Frequently in the fixed-price contracts I have done, even with reputable people with whom I've had a previous relationship, that "argue" bit above is not hyperbole. Then it falls on me to demonstrate why what you did does not meet the requirements, or is flawed in some other way, and I don't have time for that either. That was why I was paying you. Now the relationship is adversarial and asymmetric: it is easy to not understand a problem, and hard to both understand it and convince another party who is not invested in understanding it to understand.

Whereas if you are billing hourly, you are more than happy to be told about problems that you can then charge me to fix. I get something I can actually use that way, too.

derf_··on Apple, Google, Facebook's AV1 standards group under EU antitrust investigation
> Also, I think the intention is that the license you have to grant only covers implementations of AOMedia standards; if a VVC implementation or whatever also used your invention, that would probably need a separate license you could charge for.

That is correct. Source: I helped draft the license.

derf_··on Buffett, Jones and Hamm: An Oil Wisdom Trifecta
> ... it seems like Buffett is betting that won't happen ...

At the end of 2021, Berkshire owned about $4.5 billion of Chevron shares. In that same year, Berkshire Hathaway Energy, which has made huge investments in wind and solar, had $4 billion in earnings [0]. BHE is investing $18 billion (starting in 2006 and continuing through 2030) to modernize the electrical grid to improve transmission from the remote areas that are the best sources of wind and solar energy (as opposed to near coal plants). BNSF, Berkshire's railroad, which is a far more carbon-efficient way to move freight than roads and trucking, invested $41 billion in fixed assets between 2010 and 2020 [1].

It is possible for someone's approach to energy investing to be multi-faceted.

[0] https://www.berkshirehathaway.com/letters/2021ltr.pdf [1] https://www.berkshirehathaway.com/letters/2020ltr.pdf

derf_··on W3C to become a public-interest non-profit organization
> Time to take standardization to ISO or another "real" standardization body.

If you want to see how the web could be worse, do this.

The ISO process is openly hostile to the development of royalty-free standards. You can direct a lot of criticism towards the W3C, but they got the patent policy right.

derf_··on Meta Scrutinizing Sheryl Sandberg’s Use of Facebook Resources over Several Years
> ...and this could be for a number of reasons that have nothing to do with anything you did.

My father used to captain ships, and once explained to me that if the ship runs aground, it doesn't matter that the captain was asleep in his bunk at the time. He was responsible and would be court martialled for it. That meant he made damn sure his XO standing the night watch was competent and sober.

It doesn't matter if you didn't do anything. You should have done something. That's the job. You're responsible.

Yearly pay for an O-6 is between $87,984 and $155,754, by the way: https://www.federalpay.org/military/coast-guard/captain

derf_··on I'm Too Risk-Averse for Index Investing
Maybe from 10,000 feet. Buffett only invests in certain kinds of companies (those within his "circle of competence"), so you only get so much diversity.

Also, because Berkshire is a conglomerate, businesses which generate large amounts of cash (e.g., insurance float, or businesses that would pay dividends if they were a stand-alone companies) can be used to fund capital-intensive businesses (railroads, energy companies, etc.) without paying taxes to move the money from business A to business B, because they're all owned by the same legal entity. Don't underestimate the power of this effect. It is not possible to achieve with an index fund.

derf_··on uBlock, I exfiltrate: exploiting ad blockers with CSS
https://datatracker.ietf.org/doc/html/draft-iab-protocol-mai...
derf_··on The decline of unfettered research (1995)
As someone who sat on a committee handing out (small) research grants in the past (usually as no-strings-attached donations), the concern was never really that you would not deliver exactly what was promised. The concern was that you would not do anything at all (or very little). I know it might not even occur to an honest person that this would be a problem, but you would be surprised.

If you no longer believe what you proposed is a useful thing to do... send an e-mail. This may be easier to resolve than you think.

derf_··on “AI promised to revolutionize radiology but so far its failing”
I don't know anything about seismology, and I am going to put aside the money and focus on the math.

> The features you're interested in are almost never what's clearly imaged -- instead, you're predicting what's in that unimaged, "mushy" area over there through fundamental laws of physics like conservation of mass and understanding of the larger regional context. Those are deeply difficult to incorporate in machine learning in practice.

I was part of a university research lab over 15 years ago that was doing exactly this [1], with just regular old statistics (no AI/ML required). By modeling the variability of the stuff that you could see easily, you could produce priors strong enough to eek out the little bit of signal from the mush (which is basically what the actual radiologists do, which we know because they told us). It isn't a turn-key, black box solution like deep learning pretends to be. It takes a long time, it is highly dependent on getting good data sets, and years of labor goes into a basically bespoke solution for a single radiology problem, but the results agree with a human as closely as humans agree with each other. You also get the added bonus of understanding the relationships you are modeling when you are done.

From university lab to clinically proven diagnosis tool is of course a longer road, and I have not been involved in these projects for a long time, but my point is that the math problem on its own is tractable.

[1] http://midag.cs.unc.edu/

derf_··on Eliminating Data Races in Firefox
You are right to be concerned. The primary culprit is the use of C99 variable-length arrays (or alloca when those are not available). I can see why ASAN would want to pad those.

Fortunately, the allocation sizes of these arrays are tightly constrained by the limits of the Opus format (this is not an accident: I helped design the format). I performed a careful analysis (by hand) to ensure the maximum possible stack usage (without ASAN) was not anywhere close to the size allocated for the stack. It is of course possible that I made mistakes, but I tried very hard not to.

The broader point is that this is an entire class of memory access error that ASAN cannot properly validate due to its design. Because it changes the system under test in ways that differ from a real deployment, I cannot use it to mechanically validate my hand analysis.

It is worth noting that valgrind does not have this problem (because it instruments the original binary, instead of making compile-time changes), and is also used by Firefox in testing.

derf_··on Eliminating Data Races in Firefox
I have seen false positives in ASAN.

In Firefox, media threads use a reduced stack size (128 kB), due to, historically, 32-bit address-space concerns with lots of media elements on a page. Opus, in its default configuration, requires a fair amount of working stack memory (a few dozen kB in some cases). "A few dozen" is much smaller than 128 kB, but apparently enabling ASAN increases the stack usage, quite a bit. So we got crashes under ASAN when it overflowed the stack that would not have been possible without ASAN enabled.

https://bugzilla.mozilla.org/show_bug.cgi?id=750231

derf_··on OpenSSL Security Advisory
OpenSSL's API has a lot of surface area.

As someone who has used it on a project, once you start trying to do "advanced" things like optimize round trips in conjunction with your application layer on top of it, you have to start poking around some obscure parts of that API, where the only documentation is "read the source code". Even some basic stuff like validating that the certificate corresponds to the domain you connected to (like, pretty important!) was historically not done by default and required interfacing with a bunch of low-level stuff. The things you used to have to do to use the Windows system certificate store were also pretty hideous. I believe these things are handled better in more recent versions, but of course much software was written in the past, including mine, and if you wanted it to work (and continue to work, with older versions), you had to do the hideous things.

So a drop-in replacement is really a tall order, and would also require repeating many of OpenSSL's mistakes to achieve true compatibility. Ironically OpenSSL 1.2 itself broke a lot of these APIs in ways that affected my project (and were not, in my opinion, always strictly better).

It's a mess.

derf_··on CFTC Orders Coinbase Inc. To Pay $6.5M Penalty
You say that like this stuff does not happen in other regulated contexts: https://www.fca.org.uk/news/press-releases/fca-fines-and-pro...

Without trying to defend Coinbase in particular, isn't the fact that this was caught and punished evidence of the system working as intended? Like, what you would not want to see is a "regulated" market where the regulators never find anything bad going on.

← PreviousPage 7 of 17Next →