HNHacker News
TopNewBestAskShowJobs

dorkitude

1,590 karma · joined July 22, 2010

applied AI researcher and engineer from rural America, living in Berkeley, California.

CTO of Endgame.io, ex-founder of Keen.io

made @hipsterhacker, @techcrunchonion, and some other tech comedy stuff that would be more incriminating were I to share it ;)

submissionscomments
dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
This is a subject near and dear to my heart, and I'm sure we could have a lovely chat about this -- got Skype?
dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
I agree with you 100% that it would be foolish to "use instances of user-defined classes where built-in types will suffice."

If we disagree, it's because I don't see "where built-in types will suffice" and immediately think "always", as perhaps you do.

This miscommunication is largely my fault. I shouldn't have marked the repository or its corresponding HN post with "Stop passing naked dictionaries around!” I must admit, my marketing impulse got the best of me there. Naked dictionaries with untyped values are obviously one of Python's strengths, and the very option of using them instead of a heavier system is a big part of why dynamic languages are so important, both to me and to the craft as a whole. It would be a more accurate reflection of my perspective if I had rather said "Stop always passing naked dictionaries around, especially in cases where you have to write schema validators for them!"

Statically typed languages are a drag because you've got type checking always, and you don't always need it -- in fact, as any dynamic programmer knows, you almost never need it.[1]

But sometimes you do. It's a fact that data schemas are useful at times. In instances where your data is in fact naturally schema’d,[2] I’ll invoke some PEP 20 dogma of my own: “Explicit is better than implicit.” There are situations in which you could catch a data problem at the time you’re storing it, but with dictionaries you only catch it later, once it’s being used (hopefully your test coverage surfaces it, or it might not even happen until you’ve deployed to production).[3] Recognizing this, you may then write a one-off schema validator for each of these situations, but that puts the onus on every developer to call that validator every time he or she is populating a dictionary of that classification (not very DRY).

In the six months or so since I wrote the first version, I’ve found DStruct to be extremely useful for lots of scenarios, like remote resources (I just wrote a use case here: https://gist.github.com/1398138), mock objects (I just wrote a use case here: https://gist.github.com/36118077f75eaed8c731), future-proofing, backwards-compatibility, and pretty much any time you’re doing outside-in development (e.g. writing a view or serializer first, and having it build up its own mock data objects, to be replaced by model instances once the model layer is complete).

In my heavily content-driven game framework, the game designer imports content via CSV.[4] With DStruct and a handful of code changes (amounting essentially to `return Decoration(input_dict)` or `return LevelMilestone(input_dict)`, instead of just `return input_dict`), we have been able to save enormous human time, because the server now detects schema-related content problems at content import time, rather than having to wait for the full content test suite to run (and hopefully have enough coverage that it can detect a missing or malformed attribute!).

I put it DStruct out there because it simply kept on saving me time, code, and technical risk in a variety of projects. Everyone is free to take it or leave it, but my preference is that they fully understand it before doing so.

-dorkitude

PS: I don't come from Java; I'm a Pythonist born and raised (even way back at IMSA, where I believe we met once or twice ;) I think in terms of metaprogramming, multiple inheritance, dynamic typing, and runtime mutation. I understand the kneejerk "oh god another Java bureaucrat" reaction, because it's a reaction I myself feel when I see certain patterns being hamfistedly jammed into Python code. DStruct is not such an occurrence.

[1] The debate about DStruct is not a debate dynamic vs. static typing in language design (we’re all pythonists here): it's a debate about the degree of the qualifier "almost" in this sentence.

[2] Sometimes you’re mocking something that’s schema’d; sometimes the interface is beyond your control; sometimes you’re retrofitting a class that used to be ORM and now is not; sometimes you’re importing CSV’s; sometimes you’re writing a feed aggregator; the list of naturally schema’d data sources goes on and on.

[3] This is particularly true in Ruby, where they always seem to use Hash-as-argument in an attempt to get the future-proofing power we get for free in Python via keyword args.

[4] And not just store items, settings, and game objects, but even the rules and economic structure! The game framework referenced here is the best work of my career to date. Unfortunately it is a closed source project, but suffice it to say the concepts baked into DStruct (and a handful of other killer libraries I will release in the coming year) had a significant positive effect in terms of code bloat, development time, quality, and risk.

dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
So you're saying one shouldn't write significant programs in Python?

With that I must disagree :)

dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
Trolls are worse than collaborators in every way.

(Only one of our two statements is true.)

dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
(responded in http://news.ycombinator.com/item?id=3274914)
dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
Explicit types are neither pythonic nor unpythonic: always using explicit types is unpythonic, but explicit is better than implicit is an important piece of 'import this'.

I released DStruct because it has been insanely useful in my work. Take it or leave it, but do so pragmatically: dogma is a terrible way to make decisions.

dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
RE namedtuple: http://news.ycombinator.com/item?id=3274995
dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
Namedtuple is a very useful module, but it doesn't enforce typing. Furthermore, it doesn't expose extensible classes, unless you want to write the sort of code you see when you pass `verbose=True` into the namedtuple constructor (http://drktd.com/C4QF). So while a game's BaseBuilding may require the "x", "y", and "building_type_id" fields, your FarmVille Plant may add "planted_time" in it's schema, plus a "harvestable_after_time" computed property.

In the abstract, tuples simply arent suites to all problems (any more than are dicts): order doesn't always matter, and immutability certainly is not always desired.

(As for those dependencies, they're submoduled in the repo: try `git clone --recursive`)

dorkitude··on Stop passing naked dictionaries around: Introducing DStruct for Python
I must admit, I chose the title of this submission poorly. It should have read "stop passing naked dictionaries around for everything". Of course, dictionaries are incredibly useful when called for (for instance, when you want to iterate over the keys!).

But I have frequently seen them being used when class instances would be better suited.

Dictionaries' values cannot be replaced with methods via @property, and as a result, I've found they often need to be replaced with objects at some point. DStruct is a response to that problem where your access interfaces aren't forwards-compatible with the great majority of types you might choose in the future (ORM/ODM instances are a big one).

But I've also found it quite useful in the other direction: when replacing what was once an ORM/ODM class with something mocked (e.g. the data source that was once a database has become a remote resource, local file, parser output, or configuration dict), having to run around changing attribute accessors to dict accessors is a drag at best, and impossible at worst (e.g. If you were using @property extensively to return computed attributes, or you know at hydration time that code elsewhere relies on a schema (I do both a times, as anyone writing large, maintainable applications should.).

As another commenter also mentioned, my approach does yield prettier code.

While it's not always sufficient cause, prettier code is in fact a good thing.

dorkitude··on Clojure & Python, Side by Side
SQL and MongoDB MapReduce side by side: http://dbpedias.com/w/images/9/90/MongoDB.JPG
dorkitude··on Startup School 2011 Live Stream
Definitely.

(among other points!)

dorkitude··on Startup School 2011 Live Stream
I can't thank you enough for taking the time to write up this feedback. I'll post again when we rewrite it later today.

We've had customers and revenue since day 0, and since we're three technical cofounders, we have largely (and, I now see, mistakenly) focused on building our platform as quickly and thoroughly as possible, to the expense of everything else. Our plan was purely serial: focus on the tech first, then build an image and a brand[1] later.

The whole reason we applied to YC was to learn how to better communicate what we do. A YC S11 founder made it clear to me that YC could help us to fix our communication problems before they became problems, and only then did I apply. He said YC could help us learn how to communicate to three audiences: investors, the press, and customers. I give us a F, F, and C- for those respective audiences. We will get to A, A, A.

[1]: Frankly, we're not even happy with the name "Schmetrics", now that we know how hard it is for people to pronounce and spell.

dorkitude··on Startup School 2011 Live Stream
That was me.

Sorry if I was confusing/confused -- I have an extreme personality (I'm far too abstract), which is a big part of why complementary cofounders are so important for people like me. Had they been present, I'm sure we would've done better :)

dorkitude··on Panic of the Plutocrats
The substance -- not the source -- is what should be considered in every encounter with an idea.
dorkitude··on Locale and The Good Entrepreneur Test
Wow, I think I picked a bad time to blog this pro-Bay Area stuff: http://news.ycombinator.com/item?id=3041845
dorkitude··on Google Drive: Is the Dropbox Party Over?
That headcount is part of the reason it's easier to compete with Google.

The design of the Google organization rewards micro-optimizations (use data for every decision, or else your ass isn't covered) and risk-aversion (set low KPI's and game them to get the biggest bonus each quarter).

By contrast, the design of the innovation space in general rewards peak-jumping, risk-taking, and a healthy blend of intuition and data. And it already selected Dropbox.

Don't get me wrong, the (layers and layers of) leadership at Google want to make killer products. But at the level of the lowly implementer, not only is making killer products only a peripheral concern, but those who can and want to make killer products tend to self-select out of Google.

dorkitude··on Google Drive: Is the Dropbox Party Over?
I expect GDrive to compete with Backblaze and Crashplan (cheap + inconvenient compared to Dropbox), but its effect on the Dropbox userbase to be minimal.
dorkitude··on Google Drive: Is the Dropbox Party Over?
Dropbox is a dominant product because they understand user experience. Google's in-house products are, increasingly, failures because they don't.

* Of course, there are a lot of smart people at Google. It's not that there aren't Googlers who understand user experience, it's that the Google organism rejects their intuition and refuses to understand it. They methodically optimize UX to a local maximum, and throw up their hands when nobody is impressed, like a Vulcan trying to compose music.

dorkitude··on Is New York's Tech Boom Sustainable? (PG asks)
YC has expanded, but in a centralized fashion.

When it comes to expanding to NYC (which amounts to doubling the number of offices), here are some considerations off the top of my head:

* To have double the # of locations, you would need to hire a bunch more partners in order to double the sum talent/experience in your pool for Office Hours (lest the talent/experience density be diminished, harming YC and all its companies). But it's not just about sum talent/experience, because there's a big chunk of experience that no startup vet out there has: experience being a YC partner.

* YC dinners are about connections. To maintain the same quality of connections in a second location, the new partners would of course have to be as connected and as trusted in NYC as are the SV partners in SV.

* Launch emails (AKA the YC "press release") are typically written by PG. There is only one PG, and perhaps he can't or doesn't want to write twice as many launch emails every year. For all we know, going from ~8 potential launches to ~40 may have pushed him to the limit. If PG were made to write twice as many, he would only be half as intimate with each company.

* One of YC's most valuable assets is the reputation of its selection filter. If YC were to accept twice as many startups (assuming twice the # of applicants, of course), the partners and advisors running its current selection process would need to be either mirrored in NYC or overworked remotely. Again, this is a finely-tuned machine, and it's not as simple as "hire 5-10 smart startup vets and stick them in NYC to start interviewing candidates".

TL;DR: While I'm sure YC has considered (and are perhaps still considering) expanding to NYC, for you to claim that you understand their business better than they do is presumptuous. To suggest that "elitism" is YC's rationale for staying in SV is insulting, and to suggest that it is the sole potential rationale is woefully lacking in vision.

dorkitude··on Zynga's Profits Down by 95%
It's actually become quite rare for them to gate core objectives in this manner (I haven't seen it in a new game in at least a year, and I've spent a lot of time researching their games).

The general rule: if you have the patience to wait for your energy to recharge, you can play for free (and without neighbors) for quite a while (weeks or months). You might miss out on some fancy extras, but even still, it's not bad in terms of length and depth, when compared to the 'free trial' phase of so-called "hardcore" games.

dorkitude··on Is New York's Tech Boom Sustainable? (PG asks)
I think I could name several reasons not to start a second branch when the mothership is doing so well, regardless of the locations involved.
dorkitude··on Zynga's Profits Down by 95%
This is no longer the case. In terms of game design, Zynga is now leading the way on many fronts.

You could make a case that this is entirely due to the fact that they ran out of companies to copy, but that's beside the point. Nowadays, it's just inaccurate to claim that Zynga doesn't innovate.

dorkitude··on Poll: What would you pay for a SF SV Startup Shuttle?
I love this. While I agree with other comments here that it would be good to target employers, I urge you to continue to invite individual members, lest this promising idea devolve into yet another "large(r) company has a shuttle and I don't" situation!
dorkitude··on Ask HN: Screw Facebook and Go Incognito on it
Sadly, Incognito Mode also doesn't work with extensions like 1Password (indispensable for me)
dorkitude··on Why All Employees Should Be VIPs At Your Company
Facebook hardly represents all of Bay Area tech.

And it certainly doesn't represent startups (at all).

dorkitude··on Why Facebook's 'Frictionless Sharing' violates HTTP
The worst HTTP violator by far is the little "up" arrow I didn't click to upvote this article.
dorkitude··on Walk Away From Your Computer. Seriously.
Absolutely valid.

When I'm doing something difficult, nothing is more important than giving my subconscious some time to work out a problem.

dorkitude··on Motor City Mojo: The Startup Renaissance In Detroit
Thought you might find this interesting:

http://blog.dorkitude.com/post/10648678183/locale-and-the-go...

dorkitude··on Feynman On The Importance Of Playing
While I sympathize with your specific situation, I can't help but suspect that, in the general case, there's something of a selection bias when it comes to playfulness and autonomy. What I mean is, people who are dedicated to maintaining their playfulness will tend to opt out of systems that don't allow it.

Anecdotally speaking, a lot of the creative and playful people I know seem to spend their careers moving, incrementally of course, from less autonomy to more autonomy. I see these moves happen when people leave a job, accept a new job, start a business, switch majors, transfer universities, or apply to specific grad program.

My point is that it could have been something essential to Feynman's character that led him toward the amount of autonomy he's famous for having had.

dorkitude··on Facebook is scaring me
Actually, Facebook's 'publish_stream' extended permission, which has existed for 1-2 years, already gave developers the ability to publish on your wall, without getting explicit confirmation on each article.

Similarly, for Spotify to do this Activity stuff, you need to give them an extended permission called 'publish_actions'.

If anything, actions are less dangerous, since the size and visibility of the content is significantly more limited than are wall posts.

Anyone saying otherwise either is linkbaiting or truly does not understand the Facebook platform.

← PreviousPage 4 of 6Next →