HNHacker News
TopNewBestAskShowJobs

lalitmaganti

765 karma · joined February 1, 2021

Senior Staff Software Engineer at Google working on Perfetto (docs.perfetto.dev). Passionate about open source, performance, and the intersection of the two. Writing about all of these at lalitm.com.
submissionscomments
lalitmaganti··on One Number I Trust: Plain-Text Accounting for a Multi-Currency Household
Interesting! You might want to try Tabula in that case.

For that type of "obfuscated" PDFs I've come across, it does well, it's just a lot slower to run than pdf2text.

lalitmaganti··on 10 years of personal finances in plain text files
The core software itself is completely agnostic to currency or country. Otoh the importer ecosystem is somewhat US focused but a) you can easily write importers yourself b) I just published importers for a bunch of UK institutions (e.g. HSBC, Barclays, AJ Bell) - see the other post hanging around the HN frontpage :)
lalitmaganti··on One Number I Trust: Plain-Text Accounting for a Multi-Currency Household
Happy to answer :)

> What impact has having such clear visibility into your accounts had on your relationship with your wife?

That's a great question. Thankfully when it comes to finances we are very aligned in our habits and goals. So we find it very natural to be open because we know that we're both going to be aligned.

Where we differ heavily though is how much we are willing to really getting onto the nitty gritty details. She really likes knowing how our money works but she also has no interest in spending so much time and effort on it.

But that works great because I love this stuff. So every month or so we have a "finances session" where I sit down with her, take her through the books and make sure we're both happy with everything.

Obviously this very much depends on the couple whether this works but it has for us so far!

lalitmaganti··on One Number I Trust: Plain-Text Accounting for a Multi-Currency Household
> How do you think it compares time-wise to using existing accounting software?

Author here. I tried various consumer budgeting apps before I ended up building my own (and then going to Beancount). The main problem with every one of the apps I tried is that they don't handle investments well. 99% of my money is invested and having net worth figures which are wildly wrong because the app is only tracking bank accounts really annoyed me. That was the reason I built my own thing in the first place.

> Was the time investment worth it to get the control and visibility you now have?

Absolutely yes. I think it helps me really understand where my money is going, how I can make it work harder etc. Even though the RE part of FIRE doesn't appeal to me, the FI part does and knowing where I stand at all times has been very motivating.

lalitmaganti··on I ignore the spotlight as a staff engineer
> Yeah sorry if my assessment was a bit callous and overly critical, and I hope you don't take it personally

Not at all. Thanks for reading and taking the time to comment!

> and multiple times throughout my career experienced a rug pull of my own mental model of my value as a software engineer

Sorry to hear you've had that experience. Unfortunately, over the past year we've once again entered the "let's question software engineer's value" space with AI so you're very right to warn on the risks of not overselling your importance as a SWE. It pays to always be watchful and introspective on what changes are happening around you and how to best adapt to them.

lalitmaganti··on I ignore the spotlight as a staff engineer
Firstly, thanks for the thoughtful comments across the post. Really happy you found it insightful!

> But it's best to build a strong portfolio of going fast (where needed), going slow (where high leverage), and if everyone agrees your "going slow" led to huge returns you get the best of all worlds.

I guess maybe I oversold this a little in the post but I definitely think that product orgs value speed more than infra orgs: there's often an mismatch when I speak to product engineers on how fast they expect us to build things for them vs how fast we can actually go while considering all our other customers and not breaking other usecases.

> my protip for prospective staff engineers is to _never_ say you only care about [speed, stability]

Fully agreed, both are important for engineers to have. As above, I think the relative composition of them varies though depending on the area (just pulling numbers from the top of my head, maybe it's 70/30 in favor of speed in product orgs, 30/70 in favor of stability in infra orgs).

> surely we can agree impressing customer stakeholders is even more important in (healthy) product orgs

You're absolutely right for what I say in the post; reflecting I think I maybe did not go into this topic in enough detail for the nuance it deserves (but the post already felt long enough!).

Let's split the discussion into 3 different areas (as IMO they each work slightly different): infra/devex, B2C and B2B.

* Infra/devex: as I say in the post, it's critical to impress your other team's managers as that's how you prove impact.

* B2C: your customers are consumers so it's all MAUs, revenue etc. There is no "customer stakeholder" who will give you direct feedback for your promo packet

* B2B: here's where it gets interesting. So if your management chain is directly able to talk to customers (and especially senior managers) to solicit feedback without middle layers (parter manager, account managers, PMs) then yes you're absolutely correct. But often this is not the case: the middle layers act as a filtering point so you get only a fuzzy sense of how stakeholders in the other company about a specific technical thing you worked on. So again it's sort of at the whim of how your manager feels the other company's managers feels.

The basic point I was trying to make is that if you're working on external facing product, from my understanding, even if you impress your external stakeholders, it's not like in your promo packet you can attribute concrete quotes to the customers. quotes like "we couldn't have done X in our company without the work of Y engineer in your org who worked on Z". Whereas this sort of quote is extremely common to see in infra promo packets.

Hope this made sense, I'm not sure I communicated this last point well!

lalitmaganti··on I ignore the spotlight as a staff engineer
I would say it's worked out pretty well for me at least given my career trajectory! Feel free to draw your own conclusions from my resume (it's on my about page).

I think you are conflating "exec attention" with "important projects": these are very much not the same thing.

lalitmaganti··on I ignore the spotlight as a staff engineer
> But he can only do so because he is a staff software engineer.

I don't agree with this at all. This is how I've worked for my entire time at Google, all the way from new grad L3 joining the company till today. Ignoring the spotlight does not mean "don't get attention from other people" but "don't chase the project execs are focusing on".

Whenever I've work on a project, I make a very active effort to make sure engineers are aware of it, especially if I think they would find it useful. But that's different than going to my execs and asking "what's the highest priority at the moment" and working on that.

lalitmaganti··on I ignore the spotlight as a staff engineer
I'm very well aware of the privilege afforded to me by my company and the time I'm working in. I tried to emphasize in the post several times that this is not a "universal guide to success" and that in other companies or teams, a different approach and strategy might be needed.

The whole reason I wrote this post at all was, with the success of Sean's work on HN recently, I felt people were leaning too far into the direction of "you need to constantly move around and go where the exec attention is". I just wanted to show that, from my singular experience, it is possible to carve out a different path in some positions while still being ambitious and "successful" (for some definition of success).

lalitmaganti··on I ignore the spotlight as a staff engineer
> if you ever have the inclination to write about it

I definitely plan on writing a lot more about this in the coming months :) After seeing Sean's own posts and the fact this post resonated, it feels like there are people out there who might be interested in this sort of thing :)

> books, people, experiences, professional lessons learned

Books not so much but one thing i've been very fortunate to have is very good mentors I can learn off. I've had the same manager from when I first joined Google and honestly I've learned so much just from watching him work and interact with people. Also a couple of senior directors/engineers in other teams as well who I always make a habit to catch up with.

If you're interested, stay tuned to the blog :)

lalitmaganti··on I ignore the spotlight as a staff engineer
> Would a PM be responsible for this sort of broader thinking in a more typical team?

Good PMs do exactly this in product teams yes. But unfortunately PMs are not immune to shifting priorities and moving around either, just like I describe for engineers. So it's very hard to make it work but the best PMs I know somehow manage anyway!

lalitmaganti··on I ignore the spotlight as a staff engineer
My goal with this post was not to claim this is a universal template to success for everyone but simply sharing an approach that worked for me.

I tried to point out several times that, yes there are places where a "move fast with leadership" approach works better. And yes this only works in the biggest companies capable of sustaining an infra team for a long period of time.

lalitmaganti··on I ignore the spotlight as a staff engineer
> The risk comes from being wrong.

It's definitely a high risk high reward strategy but if you have the context from being the space for years and you've done your due diligence by speaking to your customers before you build things, you reduce the risk significantly.

Of course the risk can never be zero though and luck definitely played a role in past successes.

lalitmaganti··on I ignore the spotlight as a staff engineer
It's not that I want to avoid the career metagame (I would argue I haven't so far) but that the career metagame is different depending on your environment.
lalitmaganti··on How good engineers write bad code at big companies
My personal feeling is that there's a way to both agree with the underlying issues Sean writes about while being more optimistic and providing better alternatives. Something I feel I should start writing about more.
lalitmaganti··on We stopped roadmap work for a week and fixed bugs
Since the fixit just finished on Friday, I don't have hard numbers from this one I'm afraid :)

Historically though, I would guess maybe 5-10% end up needing some followup fix which is itself usually smaller than the original (maybe a typo in some documentation or some edge case we spot when it hits prod etc).

The smaller the original fixes, the less likely you are to need followups so another reason to prefer working mainly on them!

lalitmaganti··on Three Years from GPT-3 to Gemini 3
> But it suggests that “human in the loop” is evolving from “human who fixes AI mistakes” to “human who directs AI work.” And that may be the biggest change since the release of ChatGPT.

I feel like I've been hearing this for at least 1.5 years at this point (since the launch of GPT 4/Claude 3). I certainly agree we've been heading in this direction but when will this become unambiguously true rather than a phrase people say?

lalitmaganti··on We stopped roadmap work for a week and fixed bugs
> this is not care when the priority system is so broken that it requires a full suspension

I'm not sure I understand this line. The whole point of the fixit is to address the bugs which are considered "low priority" because they only appear in a edge case or are not quite 100% perfectly polished but still matter over the long tail of people using the product.

Or do you propose that every issue like this needs to be fixed before doing anything else?

lalitmaganti··on We stopped roadmap work for a week and fixed bugs
Author here! Really glad to have sparked a lively discussion in the comments. Since there is so many threads since I last looked at this post, making one top level comment to provide some thoughts:

1) I agree that estimating a bug's complexity upfront is an error prone process. This is exactly why I say in the post that we encourage everyone to "feel out" non trivial issues and if it feels like the scope is expanding too much (after a few hours of investigation), to just pick something else after writing up their findings on the bug.

2) I use the word "bug" to refer to more traditional bugs ("X is wrong in product") but also feature requests ("I wish X feature worked differently"). This is just a companyism that maybe I should have called out in the post!

3) There's definitely a risk the fixit week turns into just "let's wait to fix bugs until that week". This is why our fixits are especially for small bugs which won't be fixed otherwise - it's not a replacement for technical hygiene (i.e. refactoring code, removing dead code, improving abstractions) nor a replacement for fixing big/important issues in a timely manner.

lalitmaganti··on We stopped roadmap work for a week and fixed bugs
As I pointed out in the "criticisms" section, I don't see fixit weeks as a replacement for good technical hygiene.

To be clear, engineers have a lot of autonomy in my team to do what they want. People can and do fix things as they come up and are encouraged to refactor and pay down technical debt as part of their day to day work.

It's more that even with this autonomy fixits bugs are underappreciated by everyone, even engineers. Having a week where we can address the balance does wonders.

lalitmaganti··on Gemini 3
> Gemini app surpasses 650 million users per month

Unless these numbers are just lies, I'm not sure how this is "pushing their AI into everything they can". Especially on iOS where every user is someone who went to App Store and downloaded it. Admittedly on Android, Gemini is preinstalled these days but it's still a choice that users are making to go there rather than being an existing product they happen to user otherwise.

Now OTOH "AI overviews now have two billion users" can definitely be criticised in the way you suggest.

lalitmaganti··on WebAssembly (WASM) arch support for the Linux kernel
Tried running:

  ~ # du -h 
  (...)
  [Runner sh (2390656)]: Wasm crash: RuntimeError: operation does not support unaligned accesses
  [Main]: Stopping CPU 0
  [Main]: Stopping CPU 1
  [Main]: Stopping CPU 2
  Kernel panic - not syncing: Aiee, killing interrupt handler!
  [Runner sh (2390656)]: Kernel panic: Aiee, killing interrupt handle
> Due to a bug in LLVM's build system, building LLVM a second time fails when building runtimes (complaining that clang fails to build a simple test program). A workaround is to build it yet again (it works each other time, i.e. the 1st, 3rd, 5th etc. time).

I'm incredibly curious what this bug might be!

lalitmaganti··on Perfetto: Swiss army knife for Linux client tracing
> Yes I know, I used to work on Android ;)

Ahhhh good to know :D

> If it would help, I am of course happy to file bugs and also make simple fixes :)

Please yes do file bugs! It's so useful to have these things tracked: I think you've already given several great pieces of feedback in your comment, it would be really nice to have this tracked.

Doubly so if you have screenshots/videos/descriptions of other profilers (e.g. instruments) and how they do things so we can learn from their UX as well!

And of course, if you're willing to contribute, we would love to have them! We have a decent contributor base today but it's something I want to continue growing and investing in over time!

> can do something like get an "activity view" (like the stack that shows how many tasks are active) across all tracks

I think you're asking for basically the same as what https://github.com/google/perfetto/issues/3463 is asking for if I'm not mistaken? That will do it per-group (i.e. per process for the JSON format). Or do you actually want something global across all processes? I think we could do that, maybe hidden behind a command?

> across all tracks, or quickly select the region between two spans

I think you're asking for https://github.com/google/perfetto/issues/154!

> pin a whole process (rather than just a thread)

Makes sense, please file a bug!

> especially since I can't really right-click anything which is what I typically do when looking for features

There are some philosophical differences in the Perfetto team about "right clicking" :) I'm pro having it as an additional discovery mechanism but others feel it's a sign of bad UI/UX. Some commentary from users might help break ties!

lalitmaganti··on Perfetto: Swiss army knife for Linux client tracing
Thanks for the detailed comment!

> overwhelmingly it gets used because of their integration with the exceptionally simple JSON tracing format

So in the open source community maybe, but it's use in the Android organization in Google is ubiquitous. Obviously, people outside the company don't see this, but we do. And I think even outside, there are a fair number of people using it for Android/Linux related things.

> UI should just get split off into its own thing

By split off, do you mean from a "branding" perspective? Or like separate as its own technical entity? If the former, maybe a reasonable idea but I think rebranding now would lose a lot of the "name recognition" which comes with the Perfetto name.

If the latter, there are two problems as I see it: 1) The UI uses the trace processor as its "backend". And the trace processor is deeply integrated into the rest of Perfetto. 2) If it's split off, who maintains it?

> it's just very strange to me that the UI will always be showing things like "syscall trace" or "connect a device" when I am looking at GPU kernels or network spans

Only in the recording page right? At least from the timeline perspective, everything should be only related with your trace. And in practice, we've been working on the last year or so on "pluginizing" the UI - basically turning the UI into a modular set of components and separating the "core" of the UI (which doesn't know anything about Android or Chrome and reasons purely in terms of "tracks", "slices", "counters" etc).

So it's actually possible to just totally remove all the parts of the UI you don't care about by going around and turning off things on the "Plugins" page. Obviously, most people are not going to do that but maybe we can offer them a "one click" solution when they first open the UI to do so?

> because I get the impression that they don't even really particularly like supporting what everyone else is using it for.

This is not true, we actually really do like supporting it for other people. It's just that sometimes what we want to do for open source conflicts with what Google wants from us as a team in prioritisation. I actually talked about this in detail recently: https://lalitm.com/perfetto-oss-company-prio/

> Also, I repeat this a lot, but I think a lot of profiling tools could stand to learn things from Apple's Instruments.

At least from the Android system profiling level, we do look at Apple Instruments! And yes there are things we can learn from them! But I don't think the gap is as large as you suggest (see below).

> It's even weirder and more proprietary in the ways I mentioned above while also not having any sort of import feature

So this is one reason Apple Instruments is so good at what it does: it's a focused profilier helping you debug problems on Apple devices and so they can deeply bake that intelligence in the tool. With Perfetto, I think if you record a real Perfetto Android trace (instead of generating a JSON one) you do actually get a much more "intelligence" (even though it's not as much as Apple).

For a synthetic trace, it's very hard for us to offer the experience because we don't know the same "happy debugging paths": we have no idea how you want to look at the data. How are is the UI meant to know on a random trace what sort of "problems" the user is looking for?

Would love to hear your thoughts on this because it's not like it's a problem we're not aware of, it's more, we think it's a super hard problem with no easy solution.

> but it is exceptionally good at common tasks like "zoom into this region"

At least for this example, Perfetto has this feature: when zoomed out, click on a slice in an area you want to zoom to and then press "f". It wil snap zoom into that section

> or "let me quickly jump to the part of this call stack that is a problem"

I think we also have this but it's a few more clicks than I think we should have. Definitely something we should close the gap on!

lalitmaganti··on Perfetto: Swiss army knife for Linux client tracing
> How much overhead does the recording have?

Very much a symptom of what you are trying to record! See below

> I have tried ftrace before but it's making everything 100x slower...

If ftrace is making things 100x slower, I'm not sure that Perfetto is going to help you very much: fundamentally, for kernel stuff, it uses ftrace under the hood!

lalitmaganti··on Perfetto: Swiss army knife for Linux client tracing
> I don't suppose there is some supported way to inject macros or "ui automation" [1] into the iframe?

This is exactly what https://github.com/google/perfetto/issues/1342 is about :) It's not the highest priority issue but it is something I want to implement sooner than later.

Oh and separately you're right we should add support for also passing macros over the postMessage API. Will file a bug for this.

lalitmaganti··on Perfetto: Swiss army knife for Linux client tracing
Perfetto does support embedding though I concede it's not as seamless as it was with chrome://tracing.

As with Chrome://tracing, you can put the UI inside an iframe and pass the trace to the UI via postMessage (see https://perfetto.dev/docs/visualization/deep-linking-to-perf...). There are quite a few people that do this: the big ones I'm aware of is the Rust language CI for performance (https://github.com/rust-lang/rustc-perf/issues/1857) and Flutter Devtools (https://github.com/flutter/devtools/pull/4386/files).

There's also an effort via contributions from some external folk to add more "proper" support for embedding (i.e. compiling Perfetto directly into a larger application, more like a library). That discussion is happening at https://github.com/google/perfetto/pull/2267 and the PRs spawned off it. We're hoping to have some more public documentation on this once that effort finishes up.

lalitmaganti··on Perfetto: Swiss army knife for Linux client tracing
Please file bugs against us at https://github.com/google/perfetto/issues! We're always very interested in seeing traces which are hitting performance issues. We take them seriously and are responsive at fixing them (or at least explaining what makes it hard to fix).

As I note in the post, we optimize for the needs of Android/Chrome developers so we may not necessarily know about whatever condition you are hitting with your trace!

lalitmaganti··on Perfetto: Swiss army knife for Linux client tracing
Fun fact: Perfetto also gained support for the pprof format within the last month :)

It opens a special "aggregate flame graph" view of the profile since pprof does not preserve time information. But it works! We use it for visualizing aggregates across thousands of profiles in production!

lalitmaganti··on Perfetto: Swiss army knife for Linux client tracing
This talk was meant to be a comprehensive look into the tool for an audience familiar with performance and tracing (but not necessarily Perfetto and how it can be used on Linux).

If you're more looking for a short intro, I'd maybe suggest our docs website, specifically the page https://perfetto.dev/docs/getting-started/start-using-perfet...

For example:

* if you work in the Linux kernel, try https://perfetto.dev/docs/getting-started/start-using-perfet...

* If you're generally curious about performance or tracing try https://perfetto.dev/docs/getting-started/start-using-perfet...

There's also our quick starts which take you through "I just want to see the tool in action". Find the system tracing one at https://perfetto.dev/docs/getting-started/system-tracing

← PreviousPage 2 of 3Next →