1,516 karma · joined October 18, 2011
https://parallelthoughts.xyz
https://twitter.com/mgill25
Thanks all for the suggestions! :)
Does anyone have recommendations of how to learn to think recursively? Sure, using the language more and more does it, but what about a more general recursion as a problem solving paradigm pattern? I'm enjoying doing recursive code exercises and got into Backtracking problems. Creating things like Sudoku solvers, Knight's Tour grid etc was quite fun. I still can't develop recurrences easily though.
Any books/exercises that people here found helpful?
Also, if this is open source, I will definitely be interested in checking it out/contributing etc.
- I have a huge Zettelkasten note archive (I manage it through Sublime-Zk, which is getting a bit slow to operate it). Can I somehow directly import all my Zettel notes?
- I tried to import my Anki cards, but it seems like while I could go through them, I couldn't assign them to any particular deck/reclassify them. They show up as part of a "Not Found" deck and I don't see any option/drag-and-drop method to move them anywhere else.
I'll definitely be giving Mochi a try. Thanks a lot for creating it!
It's not like he was getting paid to work on this, was it? And people do have a life beyond open source. People could have forked and worked on the issues themselves, but that's asking too much. Why do the hard work when you can just write a comment/tweet blaming someone else, right?
Your comment is precisely what entitlement looks like.
Remote: Yes
Willing to relocate: Yes
Technologies: Python, SQL, Java, Go, Lua, Rust (learning), PostgreSQL, Kafka, BigQuery, Spanner, Beam/Dataflow
Résumé/CV: https://parallelthoughts.xyz/resume.pdf
Email: contact@manishgill.com
I'm a Data Engineer, who has built teams from scratch, and delivered end to end customer-centric data platforms. Looking for an interesting challenge in a high scale environment.
I was actually hoping to do some sort of part-time Data Science/ML Masters along with my work, but looks like I'll have to get back on the interview grind pretty soon. Not looking forward to getting back on the Interview grind - take home assignments which last a week or so, or intense algorithms training. Next company will definitely be a stable/big tech one. But for that, I'll need to leetcode like crazy.
Unsure of my future.
https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_use...
the ::text[] addition had no impact on perf. I actually constructed this post from my notes at the time so might missed have a few things here and there.
The bit about query planner recognizing things is exactly what the challenge is to be honest. Like I said at the end, the good mental model of SQL helps a lot, and mine is decent-ish I hope but can still learn a lot. :)
- The server mode compatibility with the way Mac applications are set up. I'd ideally want an emacs server running in the background, and a client app open/close seamlessly whenever I'd like. What actually ends up happening is that I close the client windows but the app never "quits". So it's a zombie icon always hanging around my cmd+tab list.
- "Windows" (or Frames?) have generally not worked seamlessly for me. I've to always put in hacks in my config to ensure the emacs windows work like other apps do for mac. They're either too small and then size up when config is applied (which has some latency so it's noticeable when it happens), or they don't work at all.
- Font rendering. There is some minute rendering difference in the way the same fonts that renders on Vim/iTerm/Macvim/Sublime/VSCode/IntelliJ, but it's the worse by far on my emacs. :(
- All of these complaints are for the GUI mode tbh, which I prefer to use since I like to keep my terminal separate from my editor. Terminal-only emacs is something I tried a few years ago and I remember getting annoyed by keybinding issues etc.
Maybe all of the above are old issues that are no longer present, or maybe I was doing things the wrong way. It just never felt like it was a seamlessly integrated application on the Mac. I hope that has changed or will change. :)
The only real disadvantage is the lack of a good library ecosystem within lua and openresty. Over the years, as we moved some more logic to edge because $reasons, we ended up rolling our own libs quite a few times. Documentation can be made much easier as well.
That, and the occasional limitations of Lua as a language.
Overall though, quite an awesome project.
Is this really true? Also, how do you just mention offers? And will the programmer interviewing me really care? Should this only be done at the HR stage?
Like, what's a good way to let the other person know that they're not the only one.
I always ask a basic SQL question (involving sorting/grouping and one simple join) in my interviews for backend and/or data. IMO if you are calling yourself a Data Engineer and do not know SQL, then you haven't really worked with data, you've mostly just worked on data infrastructure.
Once you wrap your head around basics such as GROUP BYs, JOINs, CASE statements etc, you move on to advance concepts such as Window functions and there suddenly a new world of possibilities opens up to you for analytics! I've dabbled a bit in PL/pgSQL, but the syntax is way too arcane.
SQL can also be counter-intuitive sometimes. I rewrote a particular PG query and reduced time from ~30minutes down to 2 seconds by adding a subquery. :)
And no, ORMs can't really do what SQL can do in an equivalent manner.
I agree with almost everything you said except this line. Usually, people with production access are (or should be) sensible enough not to touch the prod db with anything untested that might mutate things.
If they're not sensible enough for that, they shouldn't have had access in the first place, and instead of placing blame and firing people, I'd start thinking about /why/ that happened. Points to a more fundamental flaw in the Engineering. Firing won't fix anything.
Also note that the statement in question talks about an old methodology.
As for the article, good stuff. I enjoyed reading it.
Programming Languages: We use heavy Javascript for our core libraries, VueJS and Angular for frontend (different projects), PHP/MySQL backend (legacy) with our own home-baked framework. We also have our own edge nodes with lots of OpenResty/Lua, and our data pipelines are in Python, Java and Scala.
Language: English predominantly, but discussions and meetings can be a mix of English and Hindi (we're based out of North India). I switch to English exclusively if there's a teammate who doesn't understand Hindi very well.
"Is Agile used": I think the question is broadly about engineering practices and yes, we do use Agile/Jira, doing 2-week or more Sprints in cross functional teams. It's not perfect and everyone has their own complaints with this model, but oh well. We do heavy code reviews and have a dedicated QA team - for both automated and manual testing, extensively E2E.
Culturally, we're pretty open and transparent. Everyone /broadly/ knows what everyone else is working on. It's not startup culture, but as far as mid-sized companies go, it's pretty good. Not a whole lot of politics, a "get the job done" attitude. We occasionally open source some projects via Github. Work from home/Vacation policies are decent. I'm currently on Christmas holidays actually ^_^
LOTS of SQL. Writing sophisticated queries, building a mental model of what a DB will try to do, doing perf evaluation by looking at query plans, Analytic/Window functions.
DB optimizations. Especially BigQuery and Postgres.
Non-trivial Java 8.
Holidays are for Rust, advanced Streaming systems, and Docker/K8 :-)
Will also be looking to get deep into the Python AI/ML ecosystem in 2019. I'm fairly adept in Python and thankfully the ecosystem for Data Science/ML is quite mature.
Clickhouse would also be a good option for doing aggregating queries that TSDBs are mostly used for.
One of my wishlist items in the data space is a Managed Clickhouse offering. :-)
I've experimented with Parquet data on S3 for a work POC, and the latency to fetch the data/create tables/run the Spark-SQL query (running on EMR cluster) was quite noticeable. I was advised that EMR-FS would make it run quicker, but never got around to playing with that. But I guess the creating of in-memory tables using raw data snapshots would still remain true? Or maybe I missed something.
Also, I take it if 24 hrs is the latency requirements for ingestion to availability of this data, obviously this isn't the data platform that is powering the real time booking/sharing of Uber rides. I'd be curious to see what is the data pipeline that powers that for Uber.
1. Vietnam series by Ken Burns 2. Icarus 3. Barca Dreams
I like Lua quite a bit and it is of course, extremely fast, but I wish there were good "batteries" along with other stuff that I've come to expect from Python/Ruby et all. Of course, I've coded up some of those libraries myself as well. :)
Just wondering what people's general thoughts on this are.
Literally said those words.
Eventually figured it out but I came away with a bad impression of the company. Not the kind of stuff I judge people on.
As someone who is concerned about user and data privacy, I like these developments. What's anti-colonial about it other than just a catchy headline?