HNHacker News
TopNewBestAskShowJobs

lhnz

5,754 karma · joined June 16, 2010

www: https://sebinsua.com

github: https://github.com/sebinsua

gist: https://gist.github.com/sebinsua

twitter: https://twitter.com/sebinsua

[ my public key: https://keybase.io/sebinsua; my proof: https://keybase.io/sebinsua/sigs/h5SINjJKRvoKp0ggBhJuVn61d2Zfb3sbIp7ChK3OlTY ]

submissionscomments
lhnz··on Privatisation has been a costly failure in Britain
The UK is increasingly resembling the rest of the world's rentpig. The UK government has made Britain into one massive investment vehicle for foreign capital -- our utilities, leading firms, football teams, landmarks and property all serve to enrich sovereign wealth funds in the Middle East and Asia. These foreign investors have all received lavish incentives and grants, and in return we have been allowed to live way beyond our means for decades.

It's alarming to note the extent to which our resources have been sold off. The Conservative party, for instance, facilitated the sale of nearly 10% of Thames Water to China, and a further 10% to the Abu Dhabi Investment Authority.

Effectively, the country is being turned into a dividend machine for foreign capital, while our government debt continues to soar, now standing at over £110 billion and threatening to surpass our entire education budget [0]. This rampant exploitation of our national resources and the escalating debt crisis are alarming indicators of our economic vulnerability.

The country's economic situation is gravely concerning. Our mounting debt issues are additionally exacerbated by the looming crisis in our public services, which are showing signs of failure and are predicted to become significantly more costly. For example, we currently allocate 38p of every pound spent on public services to the NHS. However, projections show the number of Brits aged 85 and older doubling in the next 25 years [1]. Despite these alarming statistics, nobody in government or the media wants to have an honest conversation with the public about what this means.

There are also limits to how much more redistribution can be done. As per 2019/20 data, the top 1% of income taxpayers, earning above £180,000, received 13% of all income but contributed 29% of income tax receipts. The burden is even more evident when considering that the top 10% of income taxpayers were responsible for approximately 60% of all income tax receipts. In contrast, the lower half of income taxpayers, those earning less than £26,000, accounted for only 10% of income tax receipts, suggesting a significant tax skew towards higher-income earners.

It's really not looking good.

[0] https://i.imgur.com/dxK3pUB.png

[1] https://i.imgur.com/SV909oB.jpg

lhnz··on Ask HN: Could you share your personal blog here?
http://sebinsua.com

I started to do short blogs on algorithmic problems and also neural networks for a beginner-but-tenacious audience.

lhnz··on UK competition regulator finds Adobe’s purchase of Figma could reduce innovation
I don't want to get into an argument with you and didn't explain the argument very well, however, it's not as if I'm just coming up with weird opinions on my own here.

The basis of my argument comes from the director of competition policy at the International Center for Law & Economics, who explains it all in depth here: https://www.ft.com/content/b7fdbb35-e104-47bf-9e03-ef5d8b324...

  > Moreover, start-ups depend on acquisitions. Along with
  > an initial public offering, being bought is the main 
  > way entrepreneurs and venture capital investors can 
  > “exit” the firms they have built. The harder it is to 
  > sell your company, the harder it is to make a 
  > return. Fifty per cent of US start-up executives said
  > that being acquired was a long-term goal, and 90 per
  > cent of US start-up exits in 2008-18 happened thanks
  > to acquisitions. 
  >
  > Empirical evidence suggests investment in start-ups is 
  > sensitive to rules on acquisitions. One paper found 
  > venture capital activity grows by about 40-50 per cent 
  > in countries that enact pro-takeover laws, and US states
  > that introduced anti-takeover laws saw a 27 per cent 
  > decline in VC investment deals compared with those that 
  > did not.
lhnz··on UK competition regulator finds Adobe’s purchase of Figma could reduce innovation
On the contrary, the ability for startups to be purchased by larger companies is the majority of the financial incentive towards creating startups. If competition regulators prohibit these transactions, it will harm the startup ecosystem and reduce innovation.
lhnz··on Code the shortest path first
I think perhaps conservative means something different to you that it does to me.

You can write code conservatively by avoiding higher-level abstractions and preferring boring technologies. If you look at the engineering indulgences discussed within the article, they are all things that you can conservatively avoid without any prior understanding of the problem.

The intuition behind this approach is described within Sandi Metz's "The Wrong Abstraction" talk (https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction).

I also disagree that zero mental stamina should be spent on anyone that isn't between the chair and the keyboard. If you're a software engineer working within a company, the approach should generally be understandable by your team at least.

lhnz··on Code the shortest path first
Yes, we value understandability if an application is succcessful.

My point was that even if logic isn't valuable understandability helps the person reading it decide whether to delete it or replace it. Code that can't be understood and sits within vast applications providing no value is often very difficult to remove and ends up being a long-term cost to a business.

lhnz··on Code the shortest path first

  > If you're unfamiliar with the problem space, just bang away at it.
  > Write that god object, that 500 line function, hardcode all the things.
  > You wouldn't be able to come up with useful abstractions so don't
  > bother trying.
I take your point about this applying to the context of prototyping. But speaking universally, I'm trying to explain a third way in which you don't try to pick abstractions or apply hyped patterns or find code that you can make DRY, but instead try to write your code conservatively as if it were to be featured in a programming tutorial for newbie engineers. You try to make it easy to understand but you don't make choices about the solution that would require more understanding than you currently have.

In this case, it's not about applying "object-oriented design & algorithms & design patterns & frameworks & abstractions & higher-order functions & monoids & whatever else you found on Hacker News". It could be writing a god object or 500 line function, but only if these are easy to understand.

Basically, I think we should write programs as communication to ourselves and others and as a form of "theory-building". I think our artifacts should fit into this objective and communicate understanding. (I am heavily Naur-pilled, e.g. https://pages.cs.wisc.edu/~remzi/Naur.pdf)

lhnz··on Code the shortest path first
Maybe I'm not senior enough, but I think this misses the point. If I'm not prototyping, I don't try to "code the shortest path first", I try to code using the most popular libraries, and the most concrete, broad-strokes, fundamental abstractions, towards a goal of improving understandability.

At the end of the day, most code is deleted anyway. But even then people still need to understand it. I am attempting to write in the "lingua franca" and to make it clear to myself and others what I understand about the problem.

lhnz··on Bridging the gap between neural networks and functions
Thanks, I've fixed this now. I assume you were on mobile before.
lhnz··on Douglas Crockford on JavaScript
I agree with you that JavaScript's async abstractions are nice, but I think a lot of people will misunderstand the example you've given, as ensuring that the asynchronous value for `b` and `c` will be resolved prior to the asynchronous value of `a`.

Promises begin to resolve as soon as they are created. Therefore, assuming that the three executions of `promise()` in that code take 3 seconds, 10 seconds and 2 seconds respectively, we should expect `aPromise` to be resolved during `Promise.all([promise(), promise()])` immediately prior to `b` and `c`. The only thing is that we do not unwrap `aPromise` with `await` and point `a` towards it until 10 seconds after `await Promise.all([promise(), promise()])` was called.

lhnz··on Douglas Crockford on JavaScript
Not going to comment about Go here but JavaScript code like what is written above confuses many people. `Promise`s begin to resolve immediately on creation whether or not we've attempted to unwrap their value using `await`.

Therefore, it's quite possible that the first call to `promise()` might resolve after 3 seconds, while the next two calls could take 7 seconds and 10 seconds respectively. In this case, the `Promise.all` would produce a result after 10 seconds, and then the next line `const a = await aPromise` would unwrap the value within `aPromise` immediately as it would already have been fulfilled 3 seconds into execution of the `await Promise.all([promise(), promise()])` line.

lhnz··on Douglas Crockford on JavaScript
Not going to comment about Go here but JavaScript code like what is written above confuses many people. `Promise`s begin to resolve immediately on creation whether or not we've attempted to unwrap their value using `await`.

Therefore, it's quite possible that the first call to `promise()` might resolve after 3 seconds, while the next two calls could take 7 seconds and 10 seconds respectively. In this case, the `Promise.all` would produce a result after 10 seconds, and then the next line `const a = await aPromise` would unwrap the value within `aPromise` immediately as it would already have been fulfilled 3 seconds into the `await Promise.all([promise(), promise()])`.

lhnz··on Douglas Crockford on JavaScript
Not going to comment about Go here, but the JavaScript code above is confusing for a lot of people. Promises begin to resolve immediately on creation whether or not we've attempted to unwrap their value using `await`.

Therefore, it's quite possible that the first call to `promise()` might resolve after 3 seconds, while the next two calls could take 7 seconds and 10 seconds respectively. In this case, the `Promise.all` would produce a result after 10 seconds, and then the next line `const a = await aPromise` would unwrap the value within `aPromise` immediately, since it would already have been fulfilled.

lhnz··on Ask HN: Freelancer? Seeking freelancer? (June 2023)
SEEKING WORK | REMOTE WORLDWIDE, LONDON, UK

Take a read of my blog if you wish to quickly gauge my skillset: https://sebinsua.com

I'm a full-stack product engineer. I usually program in TypeScript, Python and Rust but have hundreds of hours of experience using a wide range of languages and technologies. I am an expert within the Node.js and React ecosystems, where I have 12 and 9 years of experience respectively.

  Technologies: TypeScript, JavaScript, Python, Node.js, Rust, React,
                SQL, PostgreSQL, HTML, CSS, Docker, Git, Kubernetes,
                Linux, Jupyter, FastAPI, Pandas, NumPy,
                Machine Learning, Large Language Models (LLMs), etc.
Résumé/CV: https://sebinsua.com/resume.pdf

LinkedIn: https://linkedin.com/in/sebinsua/

Email: me@sebinsua.com

lhnz··on Ask HN: Who wants to be hired? (June 2023)
Take a read of my blog if you wish to quickly gauge my skillset: https://sebinsua.com

To summarise, I'm a full-stack product engineer. I usually program in TypeScript, Python and Rust but have hundreds of hours of experience using a wide range of languages and technologies. I am an expert within the Node.js and React ecosystems, where I have 12 and 9 years of experience respectively.

  Location: London, UK
  Remote: Yes
  Willing to relocate: Yes
  Technologies: TypeScript, JavaScript, Python, Node.js, Rust, React,
                SQL, PostgreSQL, HTML, CSS, Docker, Git, Kubernetes,
                Linux, Jupyter, FastAPI, Pandas, NumPy,
                Machine Learning, Large Language Models (LLMs), etc.
  Résumé/CV: https://sebinsua.com/resume.pdf
  Email: me@sebinsua.com
lhnz··on Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?
I've heard that nowadays subword/token embeddings are learned during the training phase, and that they are useful for reconstructing the embeddings of words that contain them, and in fact allow the model to handle typos like "aple" (instead of "apple").
lhnz··on Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?
While it's true that we don't differentiate the input samples, we do differentiate the loss function's output with respect to each of the network weights. We use the chain rule to calculate each of these "gradients" and that process is known as backpropagation.

(You might have intended to say this, in which cases I'm just trying to add clarity.)

lhnz··on Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper?
I think it applies because lookups can be done by multiplying one-hot vectors (or masks) with matrices; this is roughly analogous to what happens when we multiply Q with K^T in a self-attention head.

Read this: https://e2eml.school/transformers.html#table_lookup

And then read: https://e2eml.school/transformers.html#attention

Matrix multiplication is differentiable as it is continuous. Therefore you can calculate partial derivatives of these operations. The ability to do that is what allows gradient descent optimization via the chain rule.

  > Differentiable relative to what?
  > What is (x) in the d(hashtable)/d(x) equation?
I think the derivative we actually care about is `∂L/∂(hashtable-lookup)` but here the `hashtable-lookup` is actually the matrix multiplication mentioned above. We want to know how much the "loss" (or error) changes with respect to each of our "lookups". Knowing how each of our "lookups" causes our model to err from the output we expect, allows us to optimise it.

Note: it's not really a "lookup" in the normal sense of the word. Hashtable lookups are discontinuous since we either have a value for a particular key or we do not. Our matrix multiplication is sort of like a fuzzy, probabilistic, continuous form of lookup in which nearby keys get partially activated based on their similarity to the query, and in which a small change in this query causes continous changes to the keys produced.

As far as my understanding of the self-attention equation (e.g. `softmax(QK^T / sqrt(d_k))V`) goes, its actually quite important that we get this fuzzy output in which lots of keys get partially activated for a particular query. If it only picked the maximum similarity and ignored the rest, there would be less information propagating through the network and it'd be harder for the network to learn relationships/interactions between inputs. This is why we scale `QK^T` by `sqrt(d_k)` in order to pass a tighter range of values into the `softmax()` function (which importantly generates probabilities that sum to 1, but contains exponentials which give it a tendency to over-emphasize the maximum value and ignore other values if they are too far apart).

lhnz··on Hugging Face Releases Agents
The argument that this is necessary isn't close to being convincing enough for governments to consider following through with such a drastic cause of action.

And, the "AI-might-end-up-killing-everyone" community doesn't seem to be able to see this through other people's eyes in order to make an argument for this without belittling the other perspective.

If other people change their minds, it probably won't be through persuasion but from catastrophe.

lhnz··on Tech jobs remain a good bet despite mass layoffs in Silicon Valley
No, that's not really what has happened. If you ask a question, it tends to mean you lack knowledge about something, and therefore you don't have enough context to produce a question that perfectly teases out the correct answer from GPT. Instead what happens is that you have to carefully engage in conversation and iterate towards improving GPTs context and your understanding of what it is saying well enough to ask it more incisive questions or to correct obvious inconsistencies in what it has communicated.
lhnz··on Tech jobs remain a good bet despite mass layoffs in Silicon Valley
Why are you convinced of this?

From what I can see, GPT-4 is incredibly useful but you have to be very careful to lead it in the correct direction otherwise it produces bad outputs. I say this as somebody that probably uses it every 10 minutes. Do you think that it will stop making errors of judgment and be able to behave as an agent soon?

I've not encountered anything yet that gives me faith in that, other than hype about it increasing in ability exponentially which seems unlikely.

lhnz··on A non-technical explanation of deep learning
I have a few funny analogies that I think kind of work.

1. "gradient descent" is like tuning a guitar by ear and listening to the beat frequencies ("loss") and then decreasing these by tuning a string up or down.

2. the best I can come up with for "backpropagation" is to imagine a clever device that can tirelessly optimize a Rube Goldberg machine for you but as a science, not an art.

lhnz··on A non-technical explanation of deep learning
Hm, I don't think that's quite it. I went through my own process of learning how neural networks work recently and wrote this based on my learning: https://sebinsua.com/bridging-the-gap

As far as my understanding goes, you can represent practically any function as layers of linear transformations followed by non-linear functions (e.g. `ReLU(x) = max(0, x)`). It's this sprinkling of non-linearity that allows the networks to be able to model complex functions.

However, from my perspective, the secret sauce is (1) composability and (2) differentiability. These enable the backpropagation process (which is just "the chain rule" from calculus) and this is what allows these massive mathematical expressions to learn parameters (weights and biases) that perform well.

lhnz··on A non-technical explanation of deep learning
Hm, watching Neel Nanda videos recently and I do get the feeling that there are lots of unknowns in ML and also in what trained networks have learnt.
lhnz··on The UK will spend £100M to develop its own 'sovereign' AI
It'd be more sensible to invest this money directly into Google DeepMind, which already exists and is based in London.
lhnz··on Microsoft is using illegal bribes in the Middle East and Africa
This is actually particularly bad. They followed company policy and pointed out the compliance issues but Microsoft did not reprimand the employee involved and instead promoted them into a position that they could more easily evade accountability and also punish the person that had drawn attention to compliance issues.

It gets worse as they are put on a performance improvement plan (PIP), even though they have 18 years of experience and presumably had no significant performance issues up until this point.

If the information about corruption within the post is true, Microsoft undoubtedly deserve to pay a significant fine, but unfortunately:

  > What is a shock: This time around, the SEC and DOJ
  > have both declined to investigate Microsoft over 
  > the same types of bribes in the Middle East and 
  > Africa. They acknowledged my evidence (which I 
  > submitted three times) yet did not take up the 
  > case, claiming that the current pandemic has 
  > prevented them from gathering more evidence from 
  > abroad—even though I have already provided 
  > documentation that I believe shows Microsoft is in 
  > breach of the 2019 agreement and is still 
  > participating in corrupt business practices in 
  > direct violation of U.S. law.
I think that by not punishing this the SEC and DOJ are sending a signal to whistleblowers in large corporations that they do not take compliance seriously and that they will not protect employees that try to draw attention to wrongdoing. At the very least it suggests to me that when whistleblowing, you must avoid alerting anybody in your organisation that you disagree with what you see, and do everything through external parties instead of internal policies.
lhnz··on Show HN: Live coaching app for remote SWE interviews, uses Whisper and GPT-4

  > and remember not to use the loopback device as input for the video chat app.
I laughed out loud.
lhnz··on Revert for jart’s llama.cpp MMAP miracles
From what I can see, @jart had spent a considerable amount of time on this problem and had posted an interesting-but-not-production hack to it (https://github.com/ggerganov/llama.cpp/commit/5b8023d9354010...) on March 17th, which they had also excitedly posted about on Twitter.

This was 2 weeks prior to @slaren's contribution (https://github.com/slaren/llama.cpp/commit/fc685122f95f212d1...) on March 29th, so in a sense, it's quite possible that what you've just shown is that @slaren saw that @jart was working on mmap support, worked out a cleaner solution and then wasn't happy with only being a co-author -- for their contribution, they believed that they must be the only person mentioned on the PR: although this is weird, since I don't think they even have a public profile, so maybe instead the truth is that they weren't comfortable with working with somebody that hypes up any changes they've worked on for popularity?

I don't think saying "my changes" on Twitter and other social media means what you suggest it does as is it is just informal speech to refer to things you've worked on with "my", and particularly when you see the times this was expanded (e.g. "yesterday my changes to the LLaMA C++ file format were approved") it seems more reasonable than it does without this context.

lhnz··on Revert for jart’s llama.cpp MMAP miracles
I believe you meant to respond to: https://news.ycombinator.com/item?id=35430052

However, I won't respond to you here, since (1) it should be quite clear that I think @slaren wasn't given enough recognition for their work from my prior comments and that there is a more positive approach you could have taken to helping to give them this, and (2) the rest of what you said about ethics is subjective, and I think wrong in magnitude -- for example, I'm not sure it's correct to call it "plagiarism" when @jart's PR mentioned the collaboration with @slaren, used co-authored commits and linked to their PR.

lhnz··on Revert for jart’s llama.cpp MMAP miracles
The major point I make is that the posts online that purport to come from Slaren do not show that they had "distanced themselves from [the] situation".

  > No one claiming to be that user said anything derogatory or ever called
  > for drama. You're making hollow accusations, basically: "Your messages 
  > were intermingled among bad comments"
I did not make that accusation. My accusation is that it is ill-advised to enter an anonymous imageboard where people use words like "troon" and often show mob-like behaviour, and to decide there to mouth-off about how someone took all credit for your code, removed backwards compatibility, and to add that their original attempt was an "abomination".

This is not "removing yourself from [the] situation" as Slaren asserts.

  > And I'd certainly consider it distancing yourself from the conversation
  > when you close your PR and remain silent when someone essentially steals
  > your code [...] I'm the one "responsible" for noticing this and raising
  > the flag that something isn't right.
Yes, I can see that you were one of the key people that created drama, by asking "I'm wondering how much was written by you and how much by jart?" [0] and then when they publicly said they didn't want to start drama, trying to get private comment from them by saying "My contact info is in my profile if there's more to say."

  > To the extent that drama was caused by this, it's wholly the fault of the
  > person who created this situation with her unethical behavior and 
  > intentionally misleading statements.
I think somebody could choose to believe this, but somebody that reads the GitHub and desuarchive.org threads might also feel that @InconsolableCellist and @slaren had a part to play, too.

  > Of course no one can verify if that user is the same one in the 4chan threads.
Yes, it's technically possible that somebody pretending to be Slaren investigated the GitHub and was able to correctly infer exactly what happened chronologically including that they had collaborated on jart's Discord. However, Occam's razor suggests it was Slaren themselves and not a very clever troll.

I'm really not muddying the waters here. What you're trying to argue is difficult for me to believe, and whether or not you disagree with the level of recognition given by jart, your comment that "it's wholly the fault of the person who created this situation with her unethical behavior" is ugly. It pins all the blame on jart when it's quite clear from both Slaren and your comments that you were trying to cause drama (anonymously and publicly).

I'd just like to add that if yourself and @anzz1 had wanted to give @slaren the recognition that they deserved in a positive way, you'd have linked to https://github.com/ggerganov/llama.cpp/issues/91#issuecommen... and signal-boosted that as the key insight that enabled the PR to land, rather than taking the approach you took.

[0] https://github.com/ggerganov/llama.cpp/pull/586#issuecomment...

← PreviousPage 4 of 34Next →