HNHacker News
TopNewBestAskShowJobs

vii

745 karma · joined May 26, 2009

http://john.freml.in john@fremlin.org
submissionscomments
vii··on Terry Tao on some desirable properties of mathematical notation
Enumerating what we want from notation helps us understand how far we are from the ideal. The whimsical introduction of Notation to talk about notation makes it practical. Given a domain in mathematics, adding notation (e.g. modulo arithmetic) can make complex notions pretty to express and quick to prove. I used to really enjoy this and tried to redefine notation for each exposition. It's shorter and prettier, but just pushes complexity into the notation :) and teaching people new notations is expensive, actually unless repeatedly used, more expensive than laying out details in a less concise notation.

Programming languages are notations within this framework - and domain specific languages, while much more efficient are unpopular as the costs of changing notation, in terms of training people, are too high.

The cost of communicating the notation is captured in a few of the desiderata (e.g. 1,7) but practically it is most important. If we want to be easily understood we should speak a common language!

vii··on Java Feature Spotlight: Text Blocks
Agreed that the whitespace elision is overly complex and too prescriptive for a small issue. Either whitespace is important or not; if not it doesn't matter so there should be no special handling, and if it does - the special handling is very confusing.

Scala has stripMargin, integrated into IntelliJ to explicitly indicate how to dedent the text block. Kotlin has trimIndent.

Anything that takes this long to explain should not be a core language feature.

vii··on Intel Tock-Ticks Chipsets Back to 22nm
Intel decided a couple of years ago to ramp down spending on new processes

https://www.technologyreview.com/s/601102/intel-puts-the-bra...

Moore's law is not a law but a target improvement rate: companies aim to resource sufficient individually risky projects enough to improve the process to achieve Moore's law. Intel was far ahead of competitors on process and decided to ramp down their spending. This and the resurgence of the AMD platform are consequences.

Was it the wrong strategy? Not clear at all - it would have been super expensive to keep going. They can choose to invest their huge cash pile and regain leadership

vii··on American tech giants are making life tough for startups
The big platform companies see engagement and growth metrics for all startups, not just the numbers they are pitched. The public didn't expect Instagram to be a success - Facebook had more information.

This information asymmetry is a big issue. It means big platforms can detect competitors early and know when to offer generous acquisition deals to neutralise them. The people working in these competitors don't have this comparative basis to tell how well they're doing.

As a small player, it's very hard to see how successful your competitors are. For example, Google and Apple only provide vague statistics about number of downloads in their appstores, rounded very loosely. But they internally can track detailed metrics. Facebook and Twitter are used as distribution mechanisms, for login, and as sources of contact info. They can glean insights from that.

Facebook has even better insights through the Onavo VPN, which as it tunnels traffic from other apps, can even show which features are doing well https://techcrunch.com/2018/02/12/facebook-starts-pushing-it...

vii··on GDAX will restore the balance of margin called users with company funds
During the crash, if the market were functioning normally, people could have bought cheap ETH in the ETH-USD market and sold it at nearly the normal price via the ETH-USD and BTC-USD markets all without leaving GDAX. Their outage page has the grace to mention that there were slowdowns.

A few seconds is normally long enough for liquidity to transfer between markets and this time it wasn't. That's the issue

vii··on GDAX will restore the balance of margin called users with company funds
It seems that the price plummet only occurred on the GDAX ETH-USD market. Even the GDAX BTC-ETH market hasn't the spike. Normally there are many bots ferrying liquidity between these markets so this suggests that the ETH-USD market was not giving willing traders a chance to place orders. Their first blog post indicated it was working perfectly, in which case a design rethink is in order.

Coinbase regularly goes down when there is heavy trading. Very decent of them to publicly refund people for this incident, but given they have regular Ethereum trading issues, they should be quicker to suspend their market and more humble about the technical issues they're facing

vii··on Five years of IPv6: whither the next five?
DJ Bernstein of qmail and crypto fame etc. wrote about this in detail, ten years ago: ``The IPv6 mess''[0]

Designing around an incremental upgrade strategy is hard. It's a very important design constraint for an upgrade though!

The extra, wastefully spent bits in IPv6 make it super expensive to implement routing tables. Compare that approach to the 2^48 effective limit in the 2^64 bit address space in AMD64. Like IPv6, AMD64 is an upgrade from a 32-bit system - but AMD64 included support for the previous 32-bit version and won its war long ago, while IPv6 is still reluctantly limping along.

Upgrades of huge interconnected systems need to be incremental. As a human, invested in the process, it's hard to accept that: the point of the upgrade is to replace the old system :)

[0] https://cr.yp.to/djbdns/ipv6mess.html

vii··on Ask HN: How do you become productive in a new project as a senior developer?
I've had several (previously) senior developers join projects I've led or worked on. It's really hard to get them up to speed and contributing effectively - some never succeed :(

Firstly, accept you're not going to be productive immediately as you don't have the project and business context. This is fine. Resist the urge to redesign everything, bring in new technologies and practices until you understand what is going on. Your instant reactions and conclusions will not be attuned to the needs of the project - so practice humility. Socially you may be used to knowing more than anybody else about the system you work on. Don't forget how much hard work this was to achieve the first time and get ready to do that again - but better :)

Secondly, understanding you won't be contributing at a senior level to the project immediately, focus on learning the stack. One senior developer we had whose task was to do an integration of an open source program, got bogged down for months due to disliking the scripting language most of the backend was in. To learn a stack, make your own throwaway end-to-end projects. If you don't understand the stack, you won't contribute at a senior level.

Thirdly, make sure to regularly commit code to the project. Of course, you have to talk to surrounding teams, customers, etc., and learn to cut releases, and so on, but most important is regularly contributing so as to get a feeling for who on the teams understands the system and what the real development process is like. Easy ways to make quick code changes beyond bugfixes: improve logging and metrics (also great for getting an overview of how things work). To support your suggestions for new practices and stacks: gather and document data about the real issues (regular causes of incidents, etc.) and build relationships. In time, the value of an experience transfer is huge - but the value of scattershot rewrites without context is hugely negative. Good luck!

vii··on Google’s not-so-secret new OS
It's awesome that Google is doing this and in public too https://fuchsia.googlesource.com/

Unfortunately, the hard part of an operating system isn't in a cool API and a rendering demo. It's in integrating the fickle whims of myriad hardware devices with amazingly high expectations of reliability and performance consistency under diverse workloads. People don't like dropped frames when they plug in USB :) Writing device drivers for demanding hardware is much harder than saving registers and switching process context. The Linux kernel has an incredible agglomeration of years of effort and experience behind it - and the social ability to scale to support diverse contributors with different agendas.

Microsoft, with its dominant position on the desktop, famously changed the 'preferred' APIs for UI development on a regular cadence. Only Microsoft applications kept up and looked up to date. Now Google has such a commanding share of the phone market - Android is over 80% and growing http://www.idc.com/promo/smartphone-market-share/os - they have a huge temptation to follow suit. Each time that Microsoft introduced a new technology (e.g. https://en.wikipedia.org/wiki/Windows_Presentation_Foundatio... WPF) they had to skirt a fine line between making it simple and making sure that it would be hard for competitors to produce emulation layers for. Otherwise, you could run those apps on your Mac :)

There are many things to improve (and simplify) in the Android APIs. It would be delightful to add first class support for C++ and Python, etc. A project this large will be a monster to ship so hopefully we'll soon (a few years) see the main bits integrated into more mainstream platforms like Android/Linux - hopefully without too much ecosystem churn

vii··on RethinkDB Postmortem
Slava (coffeemug) is very brave to introspect publicly and whenever I interacted with him, behaved with greatest integrity and kindness. Unfortunately, his hypothesis here that devtools are a tough market does not ring true. Making a business is always tough, making one that does something new is even tougher. There are many tools and database companies doing extraordinarily well, from Atlassian to Cloudera to Docker to Elastic. The difficulties that RethinkDB faced, as it pivoted from closed source MySQL DB backend, to open source independent database with a very cool update mechanism, were in my analysis much more mundane. Simply, RethinkDB was not easy. Here is a flavor of their thinking, when they rejected a REST HTTP interface: "The RethinkDB query language is so simple, expressive, and pleasant, that trying to make it available over HTTP would be quite difficult and a disservice to users."

Developers have a big advantage when making devtools in that they are their own target market. They are, but only at the start of the project before they become embroiled in it. Ironically this feeling of making it for oneself leads to a mistaken urge to thrust complexity onto the target. As an infra engineer myself, this is something I always try to keep in mind -- I worked on a modelling language at a large company and I credit its success to its absolutely straightforward and accessible syntax, lack of dependencies and straightforward integration into workflows. The temptation is always to make sure users are aware of the choices, force them out of bad habits and make them consider the tradeoffs (this Strange Loop talk from Peter Alvaro addresses the social treatment of users https://www.youtube.com/watch?v=R2Aa4PivG0g ). For RethinkDB, it was wrong to try to force people to learn a new language to use a DB, when there are just tons of interfaces already, and the new language has incremental benefits over say SQL.

However elegant the design of software, the deployment that is the largest or most demanding for it will always find problems. There's no point seriously trying to claim great reliability or quality for a nascent project. It's unbelievable. Instead you need to focus on getting it used in some friendly project that is willing to pay the cost of these teething problems for some particular feature. Once battle tested, reasonable tech leads can choose to use it, comfortable that they won't be the biggest deployment (e.g. git bootstrapped by its use in the Linux kernel). Alternatively, and less scrupulously, you can try to bootstrap by providing a very convenient experience at the start of a simple project for junior engineers - there are many NoSQL DBs that promise the world and are incredibly easy to set up (see the https://aphyr.com/tags/jepsen Jepsen blog for many sad examples, RethinkDB proudly did well in this test).

RethinkDB refused to make the market fit compromises necessary. Market fit is about timing. Don't read the Economist. Ship early and keep your users on board through whatever engineering compromise necessary.

vii··on Why do traders in investment banks feel their jobs are immune from AI, etc?
The question is a logical fallacy. I worked at an IB and even in 2011 traders were acutely aware of the benefits of technology and eager to invest in it and embrace it. Many of the answers here are regurgitations of techno-utopian talking points that do not take into account three issues

1. Traders generally exert an advisory/supervisory role on the set of prices that the bank offers, prices which are based on a formula or other fairly automatic means, with an added human adjustment. They already extensively use technology.

2. Traders are therefore most involved where profit can be made but simple algorithms don't work. For example, pricing big deals in illiquid markets, like when a company issues a large complex bond. As this contract is by definition not traded yet and not the same as others, there is necessarily limited applicable training data, so that there is no way to learn by example - i.e., use deep learning techniques (what I assume the question is asking about). In this case, trust and relationships are extremely important as both sides of the deal have limited information.

3. Markets change dynamics, often very rapidly. Traders have to react intelligently to events: like interest rates hitting the zero lower bound, wars breaking out or industrial accidents. They need to anticipate the actual consequence to future cash flows and also to sense the appetite of the market after the event. Publicly announced AI techniques are very far away from this kind of complex general reasoning.

The days of manual trading are long gone: of open outcry traders, yelling in bullpits and making handsignals, when banks would hire big imposing ex-football players. The question is a "why do you feel you can get away with beating your wife"?

vii··on R vs. Python for Data Science: Summary of Modern Advances
Both R and Python have gained libraries for SQL syntax queries on their datasets. These are powered by sqllite. As much of the data that gets into R or Python is obtained from a SQL query (or SparkQL etc.) it's awesome to be able to query, aggregate and even join internal datasets in the same way! https://github.com/ggrothendieck/sqldf is game changing.

The wealth of statistical packages available for R is astounding. Packages like GLMNet are made available by the top researchers in the field. In comparison, Python libraries tend to be relatively unsophisticated and less robust.

vii··on Startup Technical Diligence Is a Waste of Time
These quotes really show the paradox: within the weird definition used in the article of "amazing engineers" which is not defined but seems to imply those who can talk a big game about architecture or score well on the HackerRank, the "quality" of the engineer has no relation to business success. That's because the engineer's quality is being assessed for being able to bullshit about architecture or score points for writing "clean code."

The idea that "even great engineers" might take on technical debt is insane. In a startup situation, great engineers take on technical debt with the greatest of glee and at every opportunity, as the primary goal is to find product market fit. Bullshitting about scalability, maintainability and frameworks is counterproductive and maturely, one understands that any software will be almost totally rewritten whatever happens: take this example by John Carmack to start a prototype environment for rapid iteration, explicitly laying out ways in which it should be rewritten if successful https://groups.google.com/forum/#!msg/racket-users/RFlh0o6l3...

Many good engineers stay in their comfort zone of some part of the software lifecycle. Seems like the author was trying to assess people imagining their situation was that of big company software project going in for a third generation rewrite with little uncertainty about what it should do. This is not the situation startups are generally in, and great engineers in this situation adapt; they pivot without getting stuck on old conceits, and have a knack of understanding which signals to heed to get to that glorious market fit where suddenly reliability is valuable. Definitely they can be assessed on ability.

vii··on Clean Links – Converts obfuscated or nested links to genuine clean links
Referrer hiding is important to Google itself: https://googleblog.blogspot.com/2011/10/making-search-more-s... Previously, some (generally scammy) websites used these keywords forwarded from Google to actually change the content.

By concealing the keywords people were searching for to land on your website, they force content hosters and ad networks to depend on the Google analytics + ads ecosystem. With these keywords, you could gather information about your Google rank for queries.

Whether it's net beneficial for you as a browser owner to pass referrer information - probably it is if you like the people hosting the content you read, as it helps them understand their traffic and improve their service. If you don't like them (probably because they are spam) then this just helps them be more effective spammers.

vii··on Ask HN: Any advice for an early startup employee who feels screwed and lonely?
Rationally, employees can get at most at their replacement cost, more or less. As you have built up experience in your present role and made yourself a 'key element', you should be compensated more highly than other companies will offer you for the same sort of work. If that's not the case, then you are right that you are underpaid - just accept one of those offers, or negotiate again on that basis. There are plenty of roles in the Bay Area where technical people can make more than 165k with salary being just a part of total compensation - and you can decide how any opportunities you can find line up with your current levels. If you can't get more elsewhere then you should feel better about it :)

You could try to demonstrate that you were a key value generator by doing a new startup without the other people who have a slice of this one :)

As to whether you deserve it or not, that's a tough one.

vii··on A Way to Detect Bias
This assumes that the populations of A and B are of the same size. A larger sample will tend to have a lower minimum under many real world distributions - a sample of one will have its minimum equal to its maximum.

Another reason the use of mins here is not helpful, is that adding one equally awful accepted candidate to group A and B would then remove whatever bias there was according to the test, which is not what we want the test to indicate.

The idea by PG is a rough rule of thumb and breaks down trivially - suppose VC fund X were to accept all candidates, but group A was worse than B, the test would falsely imply that the fund was biased.

It's unfortunate the idea was dressed up in statistical persiflage because it isn't rigorous -- it's a rough guideline. To make it rigorous wold be very hard: either the abilities of the candidate populations would have to be measured very closely (unrealistic), or a more scientific experiment conducted (A-B test where candidates from each group are included or excluded opposite to the prior decision, which would need big groups).

vii··on Apple’s Share of Smartphone Industry’s Profits Soars to 92%
The comparison is extraordinarily apt.

Apple invests only a small proportion of its profit into R&D. http://appleinsider.com/articles/15/01/28/apples-rd-spending...

Samsung invests more absolutely and more proportionally. http://www.strategyand.pwc.com/global/home/what-we-think/glo...

Of course, much of an iPhone is also made by Samsung. Not sure why creaming off profits is considered a good thing or sustainable . . .

vii··on How to Evaluate Machine Learning Models: Hyperparameter Tuning
In a happy situation the sensitivity of the optimzation to hyperparameter changes is low. That's why the 'random' approach provides reasonable results. If the optimization quality were heavily dependent on the hyperparameter, for an exaggerated example, only providing good results for exactly one value of the hyperparameter, then guessing 60 times and getting within 5% of the best value of the hyperparameter would not guarantee a good model optimization.

The main difficulty with hyperparameters is that one often does not actually know a priori a reasonable range to search in. Suppose you have a regularisation constant C - without some calculation based on your data how can you pick that constant? By picking the range of the hyperparameter, the problem is just punted to a hyperhyperparameter.

More interesting than blindly guessing values, is measuring the sensitivity of recall, precision and cross validation performance to changes in the hyperparameters. Make sure that the sensitivity is low!

vii··on Borderlands Books is Closing
This is a deliberate attempt by the owners of the bookstore to make press about the minimum wage causing them to close down. As apparently Alan and Judy are owner operators, doing very little business so hopefully not requiring much external labour, this is a slightly contorted argument. Who would complain that they are not paying themselves the minimum wage and couldn't they structure it so they were not employees of their own partnership?

The unfortunate fact is that the owners have so far been able to exploit this location poorly. Valencia St around that block is now very highly sought after with numerous new glitzy businesses opening up in the last few years. If they were lucky enough to own the property or have a transferable lease they would now have an opportunity to make a huge profit on selling it to someone willing to cater to the yuppie crowd. The fact that this is not referred to in the blog post is quite disingenuous.

Another bookshop that closed down in SF was much more honest about the reasons without a weird political agenda: http://missionlocal.org/2015/01/a-heaven-for-books-moves-on/

If one takes a right wing point of view then the fact that the space will now be used more efficiently is definitely beneficial, so if the minimum wage were to blame then must be good ;)

From the point of view of the left, the question might be whether the bookstore should be subsidized to provide a living wage to its employees and owners. This is rather difficult to support as it is unclear that there is any benefit accrued to anybody beyond a sort of nostalgic effect of having a rather empty bookshop on a busy street, and if that space should be subsidized it should be made more widely useful.

vii··on CSS changes that increased Reddit account creation by 200K yearly users
Exactly.

An account registration is basically like a hit metric.

The creation of an account does not imply that it will be used or the person who created it has not created many more either accidentally or for example because they could not log in.

vii··on CSS changes that increased Reddit account creation by 200K yearly users
This is a typically poor example of ignoring the complexities inherent in analysing new user acquisition.

Suppose you implement a change that causes the registration form to crash for some subset of people? I am sorry to say for a while I was responsible for causing such issues at a very large service ;) Depending where the crash occurs and where you are counting signups you may actually get a massive boost in your metrics, as the victims retry with every attempt met by failure.

Suppose you correct an issue making it easier for people to login? Then lower registrations - the key here is to make sure you have a good metric to look at (e.g. accounts created that are interacting well with the site after as long a window as possible).

vii··on A Question
socioeconomic status

The rise of wealth inequality has been substantially documented, for example http://www.csmonitor.com/USA/USA-Update/2014/1110/Economic-i...

vii··on A brand new Foursquare, with a brand new logo and look
Foursquare had great usage all over the world. I've traveled widely (more than ninety countries) and all over Latin America and Eastern Europe Foursquare is regularly used. For some reason, it got a lot of hate from the Bay Area tech crowd who felt its heyday passed -- despite the fact that SF airport was always swarming.

The pressure to keep on growing and have viral growth and turn into a destination app means that app like Foursquare with less mainstream gamifications sadly don't get the respect they maybe deserve, and are forced to try to go viral, etc. which is not a natural path for every app, and there should be a way to fund and let these apps achieve their natural success.

Foursquare location data was great (used by Uber) and it's sad they're alienating the users who provided them this value.

vii··on A Closer Look at Android RunTime (ART) in Android L
This architecture is insane from an overall system point of view.

Why ship DEX proprietary bytecode to be compiled by the same compiler in the same way on hundreds of millions of devices to produce the same output?

It would seem far more logical to compile all the versions needed, one for each device architecture, just once. As other aspects of the apk need to be customised for each device class already, there is not really new overhead.

Going native is a great idea, doing it this way is plain bonkers.

vii··on The Two Cultures of Mathematics (2000) [pdf]
I feel the parallel is more sophisticated. In the paper Gowers bemoans that the theoreticians dismiss the depth of combinatorics. Similarly, in modern distributed systems, there is a huge gulf between the theoretically 'interesting' problems of optimality and the difficult problems solved by the practitioners (deployment, fault mitigation and control mechanisms) and similarly a lack of vocabulary to articulate a grand sounding theory from real problems.
vii··on Ask HN: How do you start a website that requires user contribution?
Having worked on a growth team for a few years and where I incremental millions more active users above trend, the underlying techniques are surprisingly simple.

- implement many different ways you can display the content you have and try to make it relevant

- don't display empty pages, try to show the most relevant content you have even if it's not great

- if you want people to perform an action (here, adding content), ask them to do it and make it easy :)

- send out email or other notifications, giving social context of other people performing the action you want them to perform and with a clear call to action, once again

vii··on Refactoring the Mirrortocracy
Great post! I think there does need to be some very introspective reevaluation of commonly accepted technology hiring processes.

Obtaining negative signal solely from firing decisions disregards the many other sources of information that can indicate that an employee was not a good fit in a role. In addition, in a team, bad performance of a small number of members can often be masked provided the rest of the team is very good.

Personally, I feel we should reconsider the appropriateness of a 'cultural fit' interview. As a concrete example of how this restricts the pool of applicants: asking about remuneration is a red flag, which discriminates against applicants who are not from well-off backgrounds, because for them immediate economic outcomes are very important.

vii··on Mary Meeker's 2014 Internet Trends
Once again, awesome analysis. Highlights for me

- great breakdown of timespent/adspend per medium showing huge potential for ads on mobile and overspend on print

- the tablet/phone boundary is being stretched by Korea with very low 'tablet' use but high phablet use

- clear age division on transition to online on-demand TV

- music switching to subscription with collapse of pay per track

- video consumption grows and grows on mobile

vii··on It's Different for Girls
Articles like this just reinforce stereotypes (women as victims) without recognising the reality that men can also be victims (of male or female abusers). This makes it even harder for men who are victimized to come forward.

There are plenty of studies which show that men are very unlikely to report sexual assault -- for example, recently http://www.slate.com/articles/double_x/doublex/2014/04/male_...

Sexual harassment is an important issue; but it affects both genders. Starting out with examples of unwelcome sexual touching in the workplace and saying "it's different for girls" drags the debate back twenty years.

I'm a man and I've faced unwelcome touching in the workplace and I'm sure I'm not exceptional.

vii··on Shadow DOM
It's really sad that this isn't clear.

CSS was intended to be simpler than DSSSL but ended up being more complicated, harder(?) to implement and immensely less powerful. One thing that could work well would be to implement layout in JavaScript, i.e. making some layout properties like x,y,z co-ordinates available to JS and then letting it lay out the text boxes in a defined way with extensible hooks.

At least that would mean web developers would only have to learn one language, rather than two!

← PreviousPage 3 of 5Next →