574 karma · joined April 22, 2015
The first things to understand is if you are hiring for a DS role to support product development/understanding, growth or marketing, advanced BI, core product features (eg ML in your offering) etc. What VP your DS would report to in a 500 person version of your org chart makes a huuuge difference in understanding what skills and aptitudes you are looking for.
Design the position and responsibilities now and in 2 yrs for the ideal candidate, then design your hiring strategy.
Looks like it was the vendor ‘Legalinc’.
Or https://iceberg.apache.org/
Which both keep track of data versioning and management of file based datasets on object stores.
Most people today have very ad how approaches to handling data versioning and lineage on Hadoop datasets.
Delta here is adding features more closely associated with RDBMS or MPP data warehouses to the Spark data pipelines with parquet data on object stores big data world.
https://github.com/apache/incubator-iceberg
https://github.com/apache/incubator-hudi
Happy to see Delta go open source.
Facebook is embarrassed by what many orgs would consider routine corporate communications practices. This is positive that they care about their brand and reputation at a higher level than your local closely held global conglomerate.
It is a bit off-putting (and just wrong) to read the author talk about white male privilege being a comparable head start to her half a million dollars of family money she’s received. (And 28,000/yr still coming!) For an essay centered on owning up to personal privilege it still has some serious short comings in recognizing how extreme her wealth privilege is versus almost all intrinsic characteristic in society privileges.
For instance if you are interested in soup to nuts sensors in the field to predictive maintenance software in the cloud, is very different set of companies than if you are just interested in data collection layer, signal transmission layer, storage layer, business logic or analytics layers.
You will find start ups that claim to be "full stack" but you might want to have some caution and understand what they've actually built and where it is in "in production" not in pilot or PoC phases.
The other interesting thing about IIoT start ups is that many are focused on specific verticals/application areas.
Companies involved with tracking trucking fleets versus operating drone fleets versus monitoring oil fields seem unlikely to all be relevant to you.
We aren't impossibly far off from this, but not that close yet either.
Very exciting.