HNHacker News
TopNewBestAskShowJobs

cottonseed

2,959 karma · joined August 22, 2011

email: <username first word> dot <username second word> at gmail
submissionscomments
cottonseed··on Yann LeCun quits Twitter amid acrimonious exchanges on AI bias
Not sure why you're being downvoted. I haven't been following this and I appreciated the summary.
cottonseed··on Who should own the copyright of AI-generated art?
The algorithm, obviously.
cottonseed··on Tell HN: Interviewed with Triplebyte? Your profile is about to become public
Thanks for the links. I registered but never set up a profile, and if you click the link in the email or [0] above, it forced you to set up a profile before you can configure your privacy settings.

Also, after opting out of personal data sharing:

> We're processing your request and should be done within 30 days.

Same for deleting your account.

cottonseed··on Ask HN: What's your quarantine side project?
Nice project! Not a quarantine project, but we've been building data frame abstractions in Python for genetics [1] [2]. We spent a lot of time studying the existing abstractions (pandas, R/dplyr, pyspark, etc.) Desinging a data frame in Python is an interesting and challenging problem. Our design is far from perfect, but I think we've found an interesting design point. Here's your example in Hail:

  >>> vehicles = hl.import_table('vehicles.csv', impute=True, delimiter=',', quote='"')
  >>> t = vehicles.filter(vehicles.make == "Saab")
  >>> t = t.order_by(t.year)
  >>> t.show(3)
  +-------+--------+-------+-------+----------------+-------------------+---------------------+-------+----------+-----------+-------+-------+
  |    id | make   | model |  year | class          | trans             | drive               |   cyl |    displ | fuel      |   hwy |   cty |
  +-------+--------+-------+-------+----------------+-------------------+---------------------+-------+----------+-----------+-------+-------+
  | int32 | str    | str   | int32 | str            | str               | str                 | int32 |  float64 | str       | int32 | int32 |
  +-------+--------+-------+-------+----------------+-------------------+---------------------+-------+----------+-----------+-------+-------+
  |   380 | "Saab" | "900" |  1985 | "Compact Cars" | "Automatic 3-spd" | "Front-Wheel Drive" |     4 | 2.00e+00 | "Regular" |    19 |    16 |
  |   381 | "Saab" | "900" |  1985 | "Compact Cars" | "Automatic 3-spd" | "Front-Wheel Drive" |     4 | 2.00e+00 | "Regular" |    21 |    16 |
  |   382 | "Saab" | "900" |  1985 | "Compact Cars" | "Manual 5-spd"    | "Front-Wheel Drive" |     4 | 2.00e+00 | "Regular" |    23 |    17 |
  +-------+--------+-------+-------+----------------+-------------------+---------------------+-------+----------+-----------+-------+-------+
  showing top 3 rows
Hail's tables are functional. Operations like `filter` and `order_by` return new tables. That means it would be an error to use `vehicles.year` in the `order_by`, since the input and the sort expression refer to different tables. Unfortunately, this means you can't use `.` chaining.

A little more background on the project: Hail's raison d'etre is a 3-dimensional generalization data frames we use for genetic data called a MatrixTable [3]. Conceptually, it is matrix-of-dicts rather than lists-of-dicts.

Genetic data is massive, so all of this is lazy and works on out of core data. The Python front end constructs an IR representing the query, it's fed through a query optimizer (written in Scala) and executed by a backend. We're working on multiple backends, but our primary backend right now is Spark.

[1] https://hail.is/docs/0.2/index.html

[2] https://hail.is/docs/0.2/hail.Table.html

[3] https://hail.is/docs/0.2/hail.MatrixTable.html

cottonseed··on Why do we fall into the rewrite trap?
Obligatory comment I have to repost every time rewrites come up: https://news.ycombinator.com/item?id=11554288
cottonseed··on Calico Labs Publications
Here's the first ranking Google turned up: the nature biomedical sciences index: https://www.natureindex.com/supplements/nature-index-2019-bi...

Harvard, the top biomedical institution, gets a score of 2,312.65. Alphabet (Calico isn't broken out), the #18th corporate institution, gets a 13.98, basically below all the major pharma companies. It wouldn't make the list of top 200 institutions overall. I didn't look closely at methodology, but it looks like it's counting publications in nature journals but doesn't weight for impact, which would surely up weight more prestigious institutions. That said, Alphabet/Calico/Verily's primary goal isn't necessarily to publish.

disclosure: I work at The Broad Institute (#66 institution, ha)

cottonseed··on The Difference Between CI and CD
3m seems aggressive to do builds and spin up infrastructure for anything non-trivial.

Reading a bit closer, I see the author describes CI as a sanity check, "ensur[ing] the bare minimum" and doesn't consider deploying on every commit. Maybe 3-7m is more realistic then.

However, I'm slightly surprised by this definition of CI. According to Fowler [0], "Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time. ... The key test is that a business sponsor could request that the current development version of the software can be deployed into production at a moment's notice." So having CI gates on the development version that are weaker than the release tests would not seem to be continuous delivery according to his definition.

We're currently releasing on every commit and our CI build (which implements continuous delivery) takes about 15m.

[0] https://martinfowler.com/bliki/ContinuousDelivery.html

cottonseed··on The Difference Between CI and CD
> Keep it short. 3-7 minutes should be max.

Who has a 3-7m CI build here?

cottonseed··on Ask HN: Are there books for mathematics like Feynman's lectures on physics?
I asked this on stackoverflow: https://math.stackexchange.com/questions/62190/mathematical-...
cottonseed··on Ask HN: Where can I work as a software engineer to combat large global problems?
Come work at the Broad Institute and work on open science to understand and improve human health: https://www.broadinstitute.org/about-us. My group isn't hiring right now, but others are: https://www.broadinstitute.org/careers/software-engineering. edit: My email is in my profile. Reach out if you have any questions.
cottonseed··on Ask HN: What are considered “Best in Class UX design” for websites?
I think the Cava ordering menu is amazing: https://cava.com/. Kudos to whoever designed that.
cottonseed··on Ask HN: What's your favorite corporate power move?
Being late isn't a power move, it's just makes you look disorganized and disrespectful. This would have 100% the opposite of your intended effect with me.

Real power moves are exercising real power: saying no and being able to back it up, expressing a dissenting opinion, solving problems and providing real value, brining in money, controlling budgets or headcount, threatening to quit because you can/will, leading by organizing people to get something done, articulating a framing for situations that cut through the BS, understand people's motivations and what's actually going on, etc. These moves "take up space," but don't confuse taking up space for power. People with actual power won't.

So get to those meetings on time.

cottonseed··on Ask HN: What's compensation like in the greater Boston area?
Where have you looked? Glassdoor salaries? Cost of living comparisons between GBA and places you're familiar with? At the end of the day, your salary is whatever you can get. Go apply for some jobs.
cottonseed··on The internet is an SEO landfill
This is perfect. Perma-ban, so sweet. Thanks GraemeL!

Too bad search engines aren't using this to improve search results.

cottonseed··on The internet is an SEO landfill
I just wish search engines would let me block sites. There are maybe a dozen SEO land grabs in my domain that are trashing my search results and provide no value.
cottonseed··on Lessons learned from rewriting code
When I hear about rewrites, I always think about this comment:

https://news.ycombinator.com/item?id=11554288

cottonseed··on Kubernetes at CERN: Use Cases, Integration and Challenges
Was this recorded? I'd like to watch the talk.
cottonseed··on Ask HN: 23, Too late to study Mathematics?
23? Ha! No. I went back to school at 34 and got my PhD in pure math from Princeton when I was 40.
cottonseed··on Ask HN: Started a job recently, great compensation but dreading work. Tips?
nunez is one of today's luck 10,000! https://xkcd.com/1053/
cottonseed··on Ask HN: Types of questions to ask team members before accepting role?
The Developer's Guide to Interviewing has some great ideas: https://medium.freecodecamp.org/how-to-interview-as-a-develo...
cottonseed··on Ask HN: What types of knowledge are worth knowing?
I feel like when people post on Ask HN, they should describe what they've already done to answer the question. Many posts seem like the height of laziness and the SN is terrible.
cottonseed··on Ask HN: Why don't people write more (from their expertise)?
Have you ever been to a library?
cottonseed··on Ask HN: What are some good projects/resources to learn distributed systems?
You might find some of these references useful:

Resources for Getting Started with Distributed Systems https://caitiem.com/2017/09/07/getting-started-with-distribu...

Distributed systems theory for the distributed systems engineer http://www.the-paper-trail.org/post/2014-08-09-distributed-s...

Distributed Systems for Fun and Profit http://book.mixu.net/distsys/

Notes on the Theory of Distributed Systems http://cs-www.cs.yale.edu/homes/aspnes/classes/465/notes.pdf

cottonseed··on Machine Learning in Google Bigquery
Can I ask more about the structure of the data and what you'd like to do with it? My email is in my profile.
cottonseed··on Machine Learning in Google Bigquery
Linear regression makes a lot of sense since it can be done streaming in a single pass. We have it as an aggregator in Hail: https://hail.is/docs/devel/aggregators.html#hail.expr.aggreg...

I wonder how they're doing logistic regression. It doesn't have a closed form and normally solved with an iterative, multi-pass optimization. edit: Read more closely. They're using (non-stochastic) batch gradient descent.

cottonseed··on Ask HN: What are the code bases like at the really big product companies?
There are plenty of open-source projects where there are WAY more than 4 contributors/team members. The Python language implementation. LLVM. Linux. Firefox. Have a look.

Google has written a bit about their monorepo culture. Here, for example: https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...

cottonseed··on Amazon’s share of the US e-commerce market is now 49%
> After that I mainly use Amazon for small household consumables.

I got fake Dove soap from Amazon.

cottonseed··on Show HN: Hacker News job trends
I keep making the same posting but I've hired ~4 people and I'm still growing. One posting equals one job is a bad assumption.
cottonseed··on Ask HN: Who is hiring? (July 2018)
At least three Broad teams post on HN, mine included (I'm not the poster). They're all growing and from what I can tell, the churn is very low.
cottonseed··on Darpa invests $100M in a silicon compiler
> with the possible exception of Lattice stuff > For Xilinx chips, where exactly nothing is publicly documented at the lower levels of the stack, they have to spend mountains of time re-discovering everything.

The situation with Lattice parts was the same; they reverse engineered them.

← PreviousPage 2 of 18Next →