HNHacker News
TopNewBestAskShowJobs

iblaine

716 karma · joined May 6, 2012

Data Engineer doing Data Engineer things www.iblaine.com
submissionscomments
iblaine··on Cadence: Uber's Workflow Orchestration Engine
Workflow engine = can support hundreds of parallel workflows. This includes airflow, luigi, dagster, appworx, are used to manage data and are typically processes that run in minutes to hours. Orchestration engine = can support millions of parallel workflows. This includes Uber Cadence & Netflix Conductor, are used to manage services and are typically processes that run in microseconds to minutes.
iblaine··on Ask HN: How do I improve our data infrastructure?
Hiring a data engineer would be your best option. The second best option would be to outsource the infra to a company like datacoral.com or pachyderm.io.

Be aware that a single database or tool is probably not a solution. You may get more rope to hang yourself with.

Any solution is going to be a function of data engineering, systems design and project management. If you lack one of those abilities then you'll need more of the other two to make up for it.

(Feel free to reach out to me on https://www.linkedin.com/in/iblaine/ if you like. I'm not selling anything...I have been in the DE industry for a while...)

iblaine··on The Indictment of Julian Assange Is a Threat to Journalism
IANAL but he’s awaiting trial for hacking, not being a journalist. Whether or not the charges are true will be one for the courts to figure out.
iblaine··on Google makes the power of BigQuery available in Sheets
IMHO, this is the biggest news to come out of Google Next. Putting large volumes of data in the hands of analysts, in a safe way, is an impressive feat.
iblaine··on Startup Stock Options – Why a Good Deal Has Gone Bad
IPOs are increasingly turning into an exercise of the Greater Fool Theory [1]. I'm not sure if it's investors or employees who are the bigger fool. Investors can fall back on ratchets [2]. Lyft may sue a bank for helping pre-IPO investors short sell [3]. What does seem to be clear is the widening gap between smart money (investors) and dumb money (employees).

[1] https://en.wikipedia.org/wiki/Greater_fool_theory

[2] https://www.investopedia.com/terms/f/fullratchet.asp

[3] https://www.cnbc.com/2019/04/06/lyft-is-threatening-litigati...

iblaine··on Microsoft CTO on the future of software engineering [audio]
Sanjay said one of the reasons he hired Kevin is because he wanted another hard core engineer in upper management at Microsoft. MSFT has historically allowed business decisions to trump technical decisions and Keven adds balance.

/former LinkedIn employee

iblaine··on Amundsen – Lyft’s data discovery and metadata engine
Metacat is worth checking out as well.
iblaine··on Why Do Buses Bunch?
This happens daily on the Muni 30X in San Francisco. What this algorithm needs is a comfort level such that a persons decision to get on a bus is a function of their expected comfort level and expected arrival time.
iblaine··on Ask HN: Best questions for an engineer to ask a CEO during a job interview?
I curated a list of 'winning' questions from this thread. Good luck and please ignore everything in this thread.

* Does the company have contingency plans for down markets?

* what is the current strike price and how many shares are outstanding

* Will you share slides from the last board meeting?

* Why the heck are you still working?

* Have you had any regrets?

* Does the company take any money in, or is it simply running off funding?

iblaine··on ArangoDB Receives $10M Series A Funding
What are the closest equivalents / competitors to ArangoDB?
iblaine··on 'Big Data' Has Come to Mean 'Small Sampled Data'
"Perhaps the biggest issue is that we need to stop treating “big data” as a marketing gimmick." says the guy who did a TED talk titled 'What Can Big Data Do For You'.

Also, comparing IBM PS/2 disk transfer rates to a 10PB cluster in a cloud makes absolutely no sense.

iblaine··on Lyft Files S-1
As a former engineer at Lyft, looks like my RSUs would be worth ~2x my salary per year. Typical RSU grants are 25% of your salary per year, so those Lyft RSUs would have been a good return. But that's at a $18-25B valuation. I think $15B is more realistic given the losses and most recent round of funding. Lyft is in a tough industry. Kudos to Logan Green for getting this far. Good to see a UCSB alumn do well.
iblaine··on How to Run SQL on PDF Files
This is a confusing feature. Converting PDFs to text is already a trivial technical challenge, so it's not a product differentiator. What is a product differentiator is converging indexing(index and store all the things) used by Rockset. This PDF feature seems like a distraction.

Also, I uploaded my PG&E bill to Rockset and got an empty result set...maybe I'm using it incorrectly.

iblaine··on Elizabeth Holmes' Final Months at Theranos
NPR has a great podcast on Theranos called The Dropout. There seems to be a growing trend that it's ok to fake it till you make it...consider Theranos, The Fyre Festival, Trump.
iblaine··on SQL: One of the most valuable skills
I recently had a candidate that had very good Scala skills and very poor SQL skills. Despite Scala and spark being powerful, you can over complicate a solution in those languages that could otherwise be easily solved in SQL. That’s interesting to me because I see SQL as a fundamental tool that you build on top of and some people these days seems to skip it.
iblaine··on 38% of bugs at Airbnb could have been prevented by using types
And this is ok. A shift away from strongly typed languages encourages laziness. A shift towards resilient micro services accommodates laziness. It’s good enough to get by and allow large companies like airbnb to grow. Clean code vs time to market is a debate that’ll never end.
iblaine··on Life of a Sql Query
This is clearly a primer on SQL and meant for those looking for an introduction. If you're one of those people, then you would also be interested in this deck on the evolution of SQL. https://www.slideshare.net/MarkusWinand/modern-sql
iblaine··on When hiring senior engineers, you’re not buying, you’re selling
This is clickbait to sell a book. "Having trouble doing the thing? The problem doing the thing is you. Buy my book for $45, then you can do the thing."
iblaine··on HeidiSQL – GUI client for MariaDB, MySQL, Microsoft SQL Server and PostgreSQL
I prefer lucidchart for being easy to integrate to confluence and share with teams.
iblaine··on Ask HN: What would your ideal issue tracker look like?
Not pivotal
iblaine··on You Weren't Meant to Have a Boss (2008)
TL;DR; Successful guy visits Africa, sees animals in the wild instead of Zoo's, creates clickbait article to imply capitalism is a zoo and people do not belong in zoo's. The premise of this argument is that people do not belong in large groups. Ironically that is what we all try to do. Over time, people organize so they can accomplish greater things as a whole rather than individuals. The end of the article goes on to say, "Founders arriving at Y Combinator often have the downtrodden air of refugees. Three months later they're transformed: they have so much more confidence that they seem as if they've grown several inches taller." I can't decide if this article attempting to be advice or a sales pitch for Y Combinator.
iblaine··on Chariot is shutting down
This is generally how the CEO of Lyft got the idea to create Lyft.
iblaine··on Ethereum: A number of hard forks turn out to be scams
Crypt has too many opportunists. It really is tragic because the technology is interesting. I do not see it the problem being fixed.
iblaine··on Show HN: Gaia – Build pipelines in any programming language
How does this compare to airflow? Is it free or how does it generate revenue?
iblaine··on In 1991, Congress authorized $650M to make driverless cars a reality (2013)
I heard the same story about the jogger being hit by the autonomous Humvee at CMU. I ended up going somewhere other than CMU, but the onsite interview process included a tour, and that tour mentioned this jogger being 'hit'.
iblaine··on JIRA is an antipattern
Just because a tool gives you enough rope to hang yourself with, doesn't mean it's a bad tool.
iblaine··on What Happens When a Founder Is Fully Vested?
Of the founders I have met that are fully vest, most stick around. I presume because they have a comfortable lifestyle or their shares will be worth more by their direct participation in the company.

I firmly believe that true altruism does not exist and find it insulting when I'm told otherwise. There's always a trade off.

iblaine··on Lyft Files for IPO
If we learned anything from SNAP, it's that investors are still greedy. APPF is a good example of a recent IPO that has done very well since its IPO date. IMHO, hyped up companies like Lyft have all the value sucked out before the IPO.
iblaine··on A history of the distributed transactional database
TiDB should be in here but trying to keep up with the latest trends is overwhelming.
iblaine··on Amazon Quantum Ledger Database
> Assuming no bad actors

Blockchain gives you security against bad actors. Here's a similar use case for a marijuana supply chain: https://www.nasdaq.com/article/leveraging-blockchains-to-tra...

← PreviousPage 6 of 14Next →