HNHacker News
TopNewBestAskShowJobs

tzhenghao

1,905 karma · joined June 17, 2016

founder / CEO of Cellulose https://www.cellulose.ai

Website: https://zhenghaotan.com Email: hi at zhenghaotan dot com Twitter: @zhenghaooo

YC Badge: 0x4405dd9f2de9c10f45ab4013fc56c461728a4795

submissionscomments
tzhenghao··on The tiny corp raised $5.1M
Not surprising lol. This was also the experience I had while experimenting with MLIR approximately 3 years ago. You'd need to git checkout a very specific commit and then even change some flags in code to have a successful build. I'm sure things are better now but I haven't messed with it since then.
tzhenghao··on The tiny corp raised $5.1M
I think AMD is going down a different path, ie. ROCm then partnering with ML frameworks further up the stack for first class support.

https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-a...

tzhenghao··on The tiny corp raised $5.1M
> I think the only way to start an AI chip company is to start with the software. The computing in ML is not general purpose computing. 95% of models in use today (including LLMs and image generation) have all their compute and memory accesses statically computable.

Agree with this insight. One thing Nvidia got right was a focus on software. They introduced CUDA [1] back in 2007 when the full set of use cases for it didn't seem very obvious. Then their GPUs had Tensor cores, and more complementary software like TensorRT to take full advantage of them post deep learning boom.

Right as Nvidia reported insane earnings beat too [2]. Would love more players in this space for sure.

[1] - https://en.wikipedia.org/wiki/CUDA [2] - https://www.cnbc.com/2023/05/24/nvidia-nvda-earnings-report-...

tzhenghao··on I didn’t learn Unix by reading all the manpages (2022)
I've used tldr (https://github.com/tldr-pages/tldr) if I'm a beginner to a specific tool, then read up the manpages and/or SO if I need more flags.
tzhenghao··on Ask HN: I have 176 logins/accounts. How many do you have?
This. And password managers can do more by constantly checking latest account breaches like on https://haveibeenpwned.com then flagging it to their customers to rotate their credentials.
tzhenghao··on Ask HN: I have 176 logins/accounts. How many do you have?
I have over 300 on 1Password, some with more than 5 different accounts from the same site like Google and Slack. I also lean heavily on the 1Password extensions to help on the retrieval part.

It's incredibly insane but I manage it by adding a namespace postfix so it's easier to identify which when autocompleting login forms. For example, "Google (Personal), "Google (UMich)" etc.

tzhenghao··on PyPI new user and new project registrations temporarily suspended
Tragedy of the commons - only need a few bad actors to ruin it all for us. Almost all distributors face this problem, from Docker Hub to PyPI. This also reminded me of official Postgres Docker image running a cryptominer in the background [1]

[1] - https://github.com/docker-library/postgres/issues/770

tzhenghao··on PyTorch for WebGPU
Just a little push back here, I think you strike on the right theme where a programming language could fill this gap. However, I wonder if new domain specific languages will eventually be the more elegant solution. Think Modular's Mojo [1] or Meta's KNYFE [2] mentioned earlier this week.

[1] - https://www.modular.com/mojo [2] - https://ai.facebook.com/blog/meta-training-inference-acceler...

tzhenghao··on PyTorch for WebGPU
This. Just to riff off an example, a lot of the APIs in common DL frameworks like PyTorch revolve around numpy or pickle formats. These are Python first semantics.
tzhenghao··on AI’s compute fragmentation: what matrix multiplication teaches us
Good points, but I'd push back just a little.

> Sure, the actual implementations can be tricky, but a single engineer (trained in it) can get that done for a specific hardware in a couple months.

I want to agree with you on this, but in practice, it's...

1. Hard to hire that engineer with deep expertise in handwritten kernels. CUDA engineers are still hard to come by and doesn't scale with productionized AI engineering demand.

2. "A few months" is a tough pill to swallow from an engineering roadmap POV, especially when models are deployed on a monthly basis. Most of the hand tuning efforts aren't scalable and will have to be done again on most iterations. This is especially true in reinforcement learning and robotics.

> But, in retrospect, the actual value add from a system that does all this for you is quite low. It's a realization I've been struggling to accept.

Yeah I remain neutral on this. On one hand, I can see that especially having to invest significant engineering effort (see point 2 above). On the other hand, you won't really know until you start benchmarking these models (and as you should).

tzhenghao··on The Federal Reserve is expected to increase interest rates by a quarter point
Oh yes, for sure! Speaking specifically about the US: Every POTUS (from both parties) during my lifetime has favored lower interest rates, cause jobs, "economy is strong" arguments - all means to look good during their term.

The Fed is now stuck in a really hard place - raise too much too fast and you get many more SVBs going belly up, posing a larger systemic risk, assuming we even "make it out of the woods". Pause / raise slowly, and you risk inflation running red hot, and people who are technically employed, live paycheck to paycheck start to rely on food banks to cover the rapid raise in cost of living - a tax on everybody.

The Fed is also fighting other dynamics like supply consequences of the Ukraine war, or energy crises, some of which are transitory, some are not. Still lots of uncertainty.

All this on top of a tug of (policy) war between an administration that favors handing out more packages, and the Fed's duty in achieving pricing stability. Can't see this ending well.

tzhenghao··on OpenXLA Is Available Now
Good catch and good point. What I was thinking was NVVM dialect. You're right on TensorRT being mostly a runtime.
tzhenghao··on OpenXLA Is Available Now
Maybe! I'd like to think of ONNX being the first standardization wave. That said, there are lots of technical limitations, such as: 1) It being a protobuf with 2GB file size hard limit. Makes it really hard and painful for large ML models. 2) Graph rewriting on these protobuf messages are extremely painful - takes significant engineering effort to productionize a ML model

Lots of innovation here. Time for a proper DL compiler.

tzhenghao··on OpenXLA Is Available Now
They're solving the same "high level problem", but with very different approaches.

TensorRT is proprietary to Nvidia and Nvidia hardware. You'd take a {PyTorch, Tensorflow, <insert some other ML framework>} model and "export / convert" it into essentially a binary. Assuming all goes well (and in practice rarely does, at least on first try - more on this later), you now automatically leverage other Nvidia card features such as Tensor cores and can serve a model that runs significantly faster.

The problem is TensorRT being exclusive to Nvidia. The APIs for doing more advanced ML techniques like deep learning optimization requires significant lock-in to their APIs, if they are even available in the first place. And all these assuming they work as documented.

OpenXLA (and other players in the ecosystem like TVM) aim to "democratize" this so there are more support both upstream (# of supported ML frameworks) and downstream (# of hardware accelerators other than Nvidia). It's yet another layer or two that ML compiler engineers will need to stitch together, but once implemented, they in theory can do a lot of optimization techniques largely independent of the hardware targets underneath.

Note that further down in the article they mention other compiler frameworks like MLIR. You can then hypothetically lower (compiler terminology) it to a TensorRT MLIR dialect that then in turn runs on the Nvidia GPU.

tzhenghao··on How to hire engineering talent without the BS
I think it has to do with headcount requirements once you're Google scale too. Leetcode style interviews become yet another "process to avoid disaster at a big company". Disaster here means hiring a bad engineer and then having to fire them for performance reasons or whatever. Google's whole concept of hiring committees like it was graduate school also reinforces this. They probably do this to minimize nepotism at the hiring stage.

All these processes become items on a hiring checklist, but to your point, it doesn't guarantee hiring a great engineer. Just one of those tragedies embedded alongside other big co bureaucracy processes.

tzhenghao··on Google Search Is Dying (2022)
> The long answer is that most of the web has become too inauthentic to trust.

Reddit was also partly successful in that regard because it catered to niche communities who ask the most interesting questions and, trolls aside, responses from the community that's generally trustworthy. There are literally subreddits with discussions on solutions that are deemed better than what you get from customer support. Reddit also doesn't have "promoted" content / responses, which is largely the issue with Google ads outlined by author.

tzhenghao··on I'm a very slow thinker (2016)
I can relate with author. It takes me a while to actually synthesize questions thrown at me, and sometimes I look like a fool staring back silently but I'm enumerating through a bunch of responses in my head. Ultimately, it's like a chess game where I don't instinctively make a move just yet.
tzhenghao··on Chick-fil-A confirms customer accounts hacked in months-long cyberattack
This is very unfortunate. Are there any better solutions to (consumer side) credential stuffing? The only ones I can think of are, and of varying effectiveness: 1. Don't share passwords between services - encourage use of password managers. 2. Encourage use of MFA - still low penetration rate
tzhenghao··on Use GNU Emacs
I'm more of a vim guy, but "I get it". I started using these "serious" editors back in college and eventually settled with vim over emacs. I don't wanna get into the whole editor wars - it's like picking a Pokemon at the start of the game.

I went through the whole phase of adding a bunch of plugins, custom macros and have a huge .vimrc file that, once in a while, would break if I was using the wrong vim variant {vi, vim, nvim}. For the sake of this argument, emacs users face some form of the same too.

It got to a point where all my hardcore configurations made my editor experience diverge even further away from the promised "out of the box" experience, and not on par with autocomplete functionalities available in modern IDEs like VSCode.

So now I'm on VSCode. A M1 MacBook can handle it, so I told myself to quit being a masochist, and that itself saves me hours especially navigating a big and complex codebase. But I remember how to use vim, and that comes in handy if I have to SSH into some remote machine and make some tweaks to scripts.

tzhenghao··on M1 MBP down to 3.5hr battery life when writing code
VsCode extensions contribute significantly to battery drain as well. The more extensions you have running, coupled with a larger codebase, the more compute needed to reindex for autocompletion, warnings etc.
tzhenghao··on Making macOS Apps Uninstallable
What bothers me even more is the fact that nowadays newly installed apps even need to "ask user for permission to access the {Documents, Downloads, Desktop, ...} folders". This is especially true when using a new terminal emulator like iTerm or Alacritty
tzhenghao··on GitHub Is Down
It seems intermittent for me when browsing repositories, both public and internal/private ones. Normally works every 4/5th refresh, but I might be partially responsible for making it worse.
tzhenghao··on Shells are two things
I've used click [1] a lot to build Python tooling scripts the past few years. Click usage is "sort of" similar to the author's proposed solution. There's also a small section here [2] that describes some of the issues covered in the article (in context of argparse).

[1] - https://github.com/pallets/click [2] - https://click.palletsprojects.com/en/8.1.x/why/#why-not-argp...

tzhenghao··on Avoid exception throwing in performance-sensitive code
This! I'd like to think exceptions are reasonable as a terminal state for a particular user workflow, which sometimes even end up having the program terminated. Examples include "unexpected I/O, corrupt files etc."

Exceptions should never be used if the control flow has means to recovery. For example, if-else statements with more different, downstream instructions.

tzhenghao··on Data Version Control
+1. git-lfs is sufficient for tracking binaries, including a ML model, at that cadence.

Thinking more abstractly, there is benefit for code and data to live "next" to each other, if possible. Atomically committed to a codebase and the latter loaded / used by the former without connecting to yet another workflow.

tzhenghao··on Hard Startups
Might be a non-answer, but I think this really depends on how much leverage you have in pulling engineering + product culture levers.

In my experience, you need to first define a small set of north star metrics that lead to the promised land, which could be years, if not a decade away. Ideally, this should not be more than 3 so there's focus in the long term. If you define this well, everybody will understand the significance of it and be aligned to hit those objectives. Any increase, exponential or not, is progress the team should feel proud of. This is 80% of the "selling" for long time scale projects.

Once you've done that, all other initiatives should focus on improving only these metrics. Generally speaking, very aggressive milestones and keeping that momentum going, while not punishing teams for missing targets will move the needle faster than the other way round.

tzhenghao··on Ask HN: What were the things you did that made the biggest impact at your work?
Encouraging others to ask questions. Forcing myself to ask in meetings as an example even though the questions look "stupid". It's uncomfortable but we've mitigated several implementation mistakes over the years. This sounds like truism but still rarely happen in practice.
tzhenghao··on Clang Format Tanks Performance
I’d agree with you, given the chance to do so in a codebase that has vanilla dependencies. If you work in robotics or do a lot of parallel computing for example, the libraries you use are almost always based on C++, so there’s not an easy way to code against them. Even if there is such a solution, a gap between the languages can lead to non-trivial behaviors and that in itself is more costly to fix than to write in the first class language C++ anyway.
tzhenghao··on What happens to Asian-American overachievers when the test-taking ends? (2011)
This. I think I'm better off today having raised by tiger parents than not. I remember the days when I feared coming home for canings if I had "median" high school scores. It wasn't fun. In hindsight, this weird combination of fear and desperation to achieve excellence I still carry on till this day. The only difference now is me competing against yesterday's self, and this has helped me significantly in my career.
tzhenghao··on How Slack Harms Projects
I've learned to dislike Slack as my career takes me to larger teams. It's great for discovery (think internal Stack Overflow for eng roles), but terrible at maintaining context on a specific "project thread". The argument against this is "you're using Slack wrongly", but at that point, how much more value add do you get from traditional communication methods aka email? Oh, and the "X is/are typing..." is a hostile feature. It's like I'm obligated to sit there with lots of anxiety, waiting for you to deliver the message.
← PreviousPage 2 of 6Next →