HNHacker News
TopNewBestAskShowJobs

iblaine

716 karma · joined May 6, 2012

Data Engineer doing Data Engineer things www.iblaine.com
submissionscomments
iblaine··on The evolution of the data engineer role
A full history of DE should include some original low code tools (Cognos, Informatica, SSIS). To some extent, the failure of these tools to adopt to the evolution of the DE role has lead to our modern data stack.
iblaine··on LG 28-inch 16:18 DualUp Monitor
In my experience, 27" monitors on top of each other will hurt your neck and/or eyes, and side by side is more comfortable.
iblaine··on GitHub Copilot is generally available
I used github copilot for a week, got some good laughs, then never used it again. Working at a publicly traded healthcare company, it worries me that my IDE has the technical ability to snoop on my code. More than anything else, github copilot is a cool parlor trick, in its current form. Surely it'll improve over time.
iblaine··on Ask HN: Share your personal site
http://www.iblaine.com

* hosted on github

* Uses a Jekyll template

* minimal content, a few blog posts, projects, links to social media

I've always wanted to host a unix terminal as my personal site but every gadget, feature, piece of content is something that may need maintenance.

iblaine··on USB-C hubs and my slow descent into madness (2021)
I had 4 monitors going w/Plugable products. I've since gone down to 2 and I would recommend Plugable to anyone looking for dual monitors on a mac.
iblaine··on USB-C hubs and my slow descent into madness (2021)
I also have a Plugable Thunderbolt 3 Dock, currently powering two displays
iblaine··on Ask HN: Who is hiring? (February 2022)
One Medical | Remote or San Francisco, CA or Austin, TX | Full-time | https://www.onemedical.com/careers/

At One Medical, we are passionate about revolutionizing the healthcare industry by offering a new approach to primary care. We combine people-centered design, technology, and a team of talented healthcare providers to give our members an amazing experience (in-house developed apps, infrastructure focused on healthcare, medical records, and virtual care to name a few).

Engineering and Corporate Roles: https://www.onemedical.com/careers/all-departments/ - Staff Software Engineer, Identity (Remote) - Senior Software Engineer (Full-Stack, Remote) - Senior Software Engineer, Data Engineering (Remote/SF) - Software Engineer Manager (Remote)

Security at One Medical: https://www.onemedical.com/careers/security/ - Senior Enterprise Security Manager - (Remote/SF/Austin)

Our focus on technology and preventative healthcare has allowed us to help address aspects of the current covid crisis. We've been doing telemedicine for years, setting up outdoor covid stations across the nation, and as of last month we now offer antibody testing, outside of all of the other services we provide.

For a better understanding of One Medical, please check out our Instagram (https://www.instagram.com/onemedical), reach out to our Recruiting team (https://www.linkedin.com/in/sbunker) or me

iblaine··on PRQL – A proposal for a better SQL
The syntax seems similar to Apache Pig. Both are declarative and primarily built to be a procedural form of SQL.
iblaine··on The Modern Metadata Platform: What, Why, and How?
I'm losing count of all the metadata management systems being open sourced. First there was Wherehows out of LinkedIn [1]. Then there was Amundsen out of Lyft, which is now stemma.io [2]. Then there was Marquez out of WeWork [3]. Now we have DataHub out of LI. It's a little crazy how underserved this problem was several years ago and how over served it's becoming.

I use Amundsen and am pretty happy with it.

[1] https://engineering.linkedin.com/blog/2016/03/open-sourcing-...

[2] https://eng.lyft.com/amundsen-lyfts-data-discovery-metadata-...

[3] https://marquezproject.github.io/marquez/

iblaine··on The Next Big Challenge for Data Is Organizational
Organizing data is a problem everyone wants to solve until they figure out the cost. It's much easier for organizations to throw data over the fence and let data engineers fend for themselves than it is to get buy-in from the top to organize data closer to the source.

There seems to be a lot of hype and hope to organize domains of data much like you would products of a business. I just don't see it happening today, largely because people in engineering/product, upsteam of DE teams, are unwilling to put in the effort.

iblaine··on Facebook Renames to Meta
Reminds me of MySpace renaming itself to My_____.
iblaine··on OpenSea product chief accused of flipping NFTs with insider information
Reminds me of a situation where I worked at an affiliate network that tracked ads and commissions. Adsense was relatively new back then and I figured out how to match impressions to clicks to AdWords keywords to sales. You could approximate the ROI/click for keywords on Google and to some extent SEO. The reaction from upper management was excitement then fear that the tool would get out, which it eventually did. Surely this will be a learning experience for OpenSea. It does happen. Here's another example. [1]

[1] https://www.theatlantic.com/business/archive/2015/10/draftki...

iblaine··on The SEC has told us it wants to sue us over Lend. We don’t know why
>we’re seeking to allow eligible customers to earn interest on select assets on Coinbase, starting with 4% APY on USD Coin (USDC)

My uneducated view of the Howie test is an investment qualifies as a security if you invest, do nothing, and expect a return. That’s my dumbed down definition that makes sense to me, a non lawyer. I do think Coinbase has painted a target on their back, the govt sees crypto as a threat to stability, and this lawsuit is an attempt to create stability.

iblaine··on For programmers, remote working is becoming the norm
Valid point, that was some hand wavy analysis on my part. Here's a new graph [1] showing the frequency with a denominator. "remote" seems to show up in every post these days, could be something like "no remote" throws this off...

[1] https://github.com/iblaine/hn-whoshiring-analysis/blob/main/...

iblaine··on For programmers, remote working is becoming the norm
Here's a related graph [1] supporting this conclusion, that remote work has recently increased. This graph shows the frequency of "remote" in HN Who's Hiring posts over time.

[1] https://iblaine.github.io/assets/images/projects/graph-remot...

iblaine··on Show HN: Visualizer for High Level System Design
I can see this being useful for virtual sys design interviews. Having to rely on Lucicharts, Miro or Google Drawing for virtual system design interviews is usually a challenge. People have to search for icons, name them, draw associations and on and on. An interface that does much of that busywork for you would be nice.
iblaine··on What the Heck is a Data Mesh?
> the writing of said logic becomes the responsibility of the team who owns that particular domain

Thanks, that much makes sense. I'm still not buying into the idea of a Data Mesh, mostly because pushing costly requirements upstream is a hard idea to sell.

iblaine··on What the Heck is a Data Mesh?
My understanding of a Data Mesh is it's an approach to turn data into a product, much like you would create an API to interface with a service. A Data Mesh is additional business logic to make data easier to understand, at the cost of implementing that business logic.

A Data Mesh sounds eerily similar to Kimball. It's an up front investment to simplify the data. Kimball is frowned upon these days because dimensional modeling is another hurdle to your data. It makes sense that a Data Mesh would get the same treatment.

The fact that "Data Mesh" is pushed by a consulting company has me suspicious as well.

iblaine··on The Rise and Fall of the OLAP Cube (2020)
We evolved our guidelines for data modeling to something that closely resembles Kimball, but it's not Kimball.

Some of our ETLs were created by people that were not familiar with Kimball. We went about defining new data modeling guidelines, taking the good parts out of this work, eliminating the bad parts, and many of the good parts share a lot in common with Kimball.

iblaine··on The Rise and Fall of the OLAP Cube (2020)
I'm a bit opinionated on data modelling, so a couple points.

1. "OLAP Cubes" arguably belong to Microsoft and refer to SQL Server cubes that require MDX queries. It's a solution given to us by Microsoft that comes with well understood features. "OLAP" itself is a general term used to describe any database used for analytical processing, so OLAP has a wide range of uses.

2. OLAP Cubes (as defined above) started to decrease in population in 2015 (I'd argue).

3. Any solution to fixing "OLAP" that comes from a commercial vendor is suspicious. As painful as Kimbal & Inmon are, they are ideas that don't require vendor lock in.

4. At my current company, we recently wrapped up a process where we set out to define the future of our DW. Our DW was encumbered by various design patterns and contractors that came and went over the past decade. We analyzed the various tables & ETLs to come up with a clear set of principles. The end result looks eerily close to Kimball but with our own naming conventions. Our new set of principles clearly solve our business needs. Point being you don't need Kimball, Inmon, Looker, etc to solve data modelling problems.

5. Columnar databases are so pervasive these days that you should no longer need to worry about a lack of options when choosing the right storage engine for your OLAP db.

6. More and more data marts are being defined by BI Teams w/little modeling/CS skills. To that end, I think it's important to educate your BI teams as a means to minimize the fallout, and be accommodating in your DW as means to work efficiently with your BI teams. This is to say settle on a set of data modelling principles that work for you, but may not work for someone else.

iblaine··on Ask HN: Who is hiring? (July 2021)
One Medical | Remote or San Francisco, CA or Austin, TX | Full-time | https://www.onemedical.com/careers/

At One Medical, we are passionate about revolutionizing the healthcare industry by offering a new approach to primary care. We combine people-centered design, technology, and a team of talented healthcare providers to give our members an amazing experience (in-house developed apps, infrastructure focused on healthcare, medical records, and virtual care to name a few).

Security at One Medical: https://www.onemedical.com/careers/security/ - Senior Security Engineer, Security Detection & Response Engineer - (Remote/SF/Austin)

Engineering and Corporate Roles: https://www.onemedical.com/careers/all-departments/ - Software Engineer, Health Record (Remote/SF) - Senior Software Engineer, Get Care (Remote/SF) - Software Engineer, Data Engineering (Remote/SF)

Our focus on technology and preventative healthcare has allowed us to help address aspects of the current covid crisis. We've been doing telemedicine for years, setting up outdoor covid stations across the nation, and as of last month we now offer antibody testing, outside of all of the other services we provide.

For a better understanding of One Medical, please check out our Instagram (https://www.instagram.com/onemedical), reach out to our Recruiting team (https://www.linkedin.com/in/sbunker) or me. https://news.ycombinator.com/item?id=27032191

iblaine··on It appears that Reddit is forcing me to use their app to see deep linked content
Reddit seems destined to follow in Digg's footsteps. The negative feedback on the reddit redesign and the reddit app has been building for years, and they continue to push it. The motivation is puzzling. Are executives knowingly pushing the need for advertising at the expense of users? Do Product managers have too much free time?

For anyone else annoying by the reddit redesign, you can use the Redirector chrome extension to fix it w/the following...

    redirect: https://www.reddit.com(.*)
    to: https://old.reddit.com$1
iblaine··on On Comments in Code
> Every time I come back to those parts of the program, I am happy I made the effort to add a comment – they have been very helpful!

Exactly. Rarely hear someone say there are too many comments in this code! And feel free to not stop at the docstrings. Nested loops, obfuscated 1 liners, business logic that required a cross functional meeting to understand, are all valid reasons to be generous with comments.

iblaine··on Cloudera taken private for $5.3B, acquires Datacoral and Cazena
I was a Cloudera client in 2015. I've since moved to company where we did on prem (thousands of nodes) then another AWS shop. Each case was an example of a superior solution to Cloudera. Cloudera's market seems to be shrinking & they need to change.

On Datacoral, it seems like a very good product, simplifying the overhead of data pipeline development but still staying modern. Datacoral has many cool features, such as treating ETLs as code with an option to develop via GUI. The industry is moving that way, has been for some time.

Good for Datacoral to get acquired and get exposure to so many new customers. But Cloudera has their work cut out for them to stay relevant in a market that is extremely competitive.

iblaine··on Ask HN: Who is hiring? (June 2021)
One Medical | Remote or San Francisco, CA or Austin, TX | Full-time | https://www.onemedical.com/careers/

At One Medical, we are passionate about revolutionizing the healthcare industry by offering a new approach to primary care. We combine people-centered design, technology, and a team of talented healthcare providers to give our members an amazing experience (in-house developed apps, infrastructure focused on healthcare, medical records, and virtual care to name a few).

Security at One Medical: https://www.onemedical.com/careers/security/ - Senior Security Engineer, Logging and Automation - (Remote/SF/Austin)

Engineering and Corporate Roles: https://www.onemedical.com/careers/all-departments/ - Software Engineer, Billing Optimization (Remote/SF) - Senior Software Engineer, Get Care (Remote/SF) - Software Engineer, Data Engineering (Remote/SF)

Our focus on technology and preventative healthcare has allowed us to help address aspects of the current covid crisis. We've been doing telemedicine for years, setting up outdoor covid stations across the nation, and as of last month we now offer antibody testing, outside of all of the other services we provide.

For a better understanding of One Medical, please check out our Instagram (https://www.instagram.com/onemedical), reach out to our Recruiting team (https://www.linkedin.com/in/sbunker) or me.

iblaine··on Shameless Work Influencers Are the Bane of LinkedIn
LinkedIn influences are an upgrade over LinkedIn Groups, such that groups are overrun with spam from recruiters. Which is to say LinkedIn Influences are not as bad as some alternatives.
iblaine··on Ask HN: Who is hiring? (May 2021)
One Medical | Remote or San Francisco, CA or Austin, TX | Full-time | https://www.onemedical.com/careers/

At One Medical, we are passionate about revolutionizing the healthcare industry by offering a new approach to primary care. We combine people-centered design, technology, and a team of talented healthcare providers to give our members an amazing experience (in-house developed apps, infrastructure focused on healthcare, medical records, and virtual care to name a few).

Security at One Medical: https://www.onemedical.com/careers/security/ - Senior Security Engineer, Logging and Automation - (Remote/SF/Austin)

Engineering and Corporate Roles: https://www.onemedical.com/careers/all-departments/ - Software Engineer, Billing Optimization (Remote/SF) - Senior Software Engineer, Get Care (Remote/SF)

Our focus on technology and preventative healthcare has allowed us to help address aspects of the current covid crisis. We've been doing telemedicine for years, setting up outdoor covid stations across the nation, and as of last month we now offer antibody testing, outside of all of the other services we provide.

For a better understanding of One Medical, please check out our Instagram (https://www.instagram.com/onemedical), reach out to our Recruiting team (https://www.linkedin.com/in/sbunker) or me.

iblaine··on Diving Deep on S3 Consistency
Anyone else still seeing consistency problems w/S3 & EMR? The latest AWS re:Invent made it sound like this would be fixed but as of yesterday I was still using emrfs to correct S3 consistency problems.
iblaine··on Best practices for writing SQL queries
"Best practices for writing SQL queries in metabase" should be the title here.

10 or so years ago when SQL Server, Oracle & MySQL dominated the industry, you could talk about SQL optimization with the expectation that all advice was good advice. There are too many flavors of databases to do that today.

iblaine··on Ask HN: Who is hiring? (April 2021)
One Medical | San Francisco, CA or Austin, TX | Full-time | https://www.onemedical.com/careers/

At One Medical, we are passionate about revolutionizing the healthcare industry by offering a new approach to primary care. We combine people-centered design, technology, and a team of talented healthcare providers to give our members an amazing experience (in-house developed apps, infrastructure focused on healthcare, medical records, and virtual care to name a few).

Security at One Medical: https://www.onemedical.com/careers/security/ - Senior Security Engineer, Logging and Automation - (SF or Austin)

Engineering and Corporate Roles: https://www.onemedical.com/careers/all-departments/ - Software Engineer, Billing Optimization (SF) - Staff Software Engineer, Health Record (SF) - Senior Software Engineer, Get Care (SF)

Our focus on technology and preventative healthcare has allowed us to help address aspects of the current covid crisis. We've been doing telemedicine for years, setting up outdoor covid stations across the nation, and as of last month we now offer antibody testing, outside of all of the other services we provide.

For a better understanding of One Medical, please check out our Instagram (https://www.instagram.com/onemedical), reach out to our Recruiting team (https://www.linkedin.com/in/sbunker) or me.

← PreviousPage 2 of 14Next →