HNHacker News
TopNewBestAskShowJobs

dbatten

1,570 karma · joined June 20, 2016

submissionscomments
dbatten··on Enabling Rich Statistical Analyses with Differential Privacy
Yes, you're absolutely right. In our case, we were effectively doing one massive annual dump of this data into an online tool (vs. multiple releases of k-anonymous data), so we didn't have to worry about the more complicated cases of two separate data releases causing disclosures.

(That, and the regulations that governed our program said to suppress cells based on 3 or less individuals... so, that's what we had to do.)

dbatten··on Enabling Rich Statistical Analyses with Differential Privacy
Sure. But to see it, we'll have to expand the example a little. Suppose you release the average salary for people that got an associates degree from the community college system - just a single number. Then you release that number broken down by campus. Then you release it for all campuses together, but broken down by subject area. Then you do it broken down by all campuses AND all subject areas. Then by all campuses, subject areas, and industry of employment. Etc. Ad nauseum.

Now suppose you had one person who got an associate's in art from campus X. You hide that value. But you now need to go searching across multiple dimensions to find "buddy cells" to suppress. You need to make sure that you suppress the value of associate's in art, but from a different campus. And you need to suppress the value of associate's from the same campus, but in a different subject. Otherwise, somebody can algebra-out the originally suppressed value. The more dimensions you have, the more you have to search across to hide data.

But here's where the optimization goal comes in - you don't like hiding data. You want to release as much data as possible. So you want to find a way to release the maximum amount of data while still protecting individual privacy.

The greedy approach is to go through every suppressed cell and make sure that it has a "buddy" cell across every possible dimension of aggregation. If it does NOT have a buddy cell, then select the cell in that dimension based on the smallest number of people, and suppress that one as well. But now you have to make sure that THAT cell is protected and can't be algebra'd-out, so you have to cycle through again. You keep doing this, always selecting the smallest possible buddy cells, until you have at least 2 cells suppressed across any dimension that can be aggregated.

Of course, if you really want to release as much data as possible, you can treat it as a binary integer programming problem where your goal is to minimize the sum of the N of data underlying suppressed cells and your decision variables are whether or not to suppress a given cell...

I hope that helps?

dbatten··on Enabling Rich Statistical Analyses with Differential Privacy
Back when I worked for the state (of North Carolina), I did a lot of public data reporting for state-run education programs (e.g., community colleges, public universities). Our system had access to enrollment and graduation data from those programs, as well as wage data on everybody working in North Carolina in a job covered by the state's unemployment insurance program (which was most jobs). We could therefore report on things like how many people were studying what subjects at what institutions, what the X-year graduation rates were, how much money people were making Y-years after graduation, what industries they were employed in, etc. It was an extremely cool data set.

Of course, the major concern in publishing that data was precisely this - people's income is super sensitive. If there was some program at some university that only graduated one student in a particular year, you couldn't report the average wage outcomes for that program, because you'd essentially be putting that student's salary online for everybody to see. Instead of adding randomness to the data, as described here, we'd simply hide any information that was based on 3 or less individuals, or which could be entered into a mathematical formula to enable you to DERIVE information on 3 or less individuals.

For example, suppose you reported the mean salary of everybody who graduated with an art degree from community colleges in 2010, then reported the mean salary for everybody who graduated with an art degree from each individual community college in 2010. Suppose further that one of those individual community colleges only had a single art degree graduate and you hid the data. Under those conditions, somebody could still do some simple algebra to calculate the hidden value. So you'd have to hide the value from a second community college to prevent somebody from working out the unknown value at the first.

As you reported more and more data across a higher and higher number of dimensions, the problem grew more and more complex. It actually ended up being really cool to reason about, and we developed both a greedy heuristic and a binary-integer-programming approach to solving to near or true optimality, respectively.

dbatten··on Practical typewriting: By the all-finger method (1894)
Hold your hand up in front of your face, and try to stick each finger up, in turn, starting with the index finger. You'll find that you can easily extend or stick up each finger except for the ring finger, which will experience a lot of tension. The ring finger can only be moved up fully and easily if the middle finger is moved up with it.

I am a classically trained pianist, but I've never heard of anyone having surgery to free up their ring finger. Doesn't mean it hasn't happened, though.

dbatten··on The Ghost in the MP3 (2014)
Are we actually inherently drawn to analog artifacts including things like film grain? Or are we accustomed to seeing/hearing it? If you plucked a member of an un-contacted tribe out of the Amazon, would they prefer film grain and vinyl? I'm not sure, but I'm willing to hear the evidence presented.

I remember the first time I saw 60hz video on a flat screen TV. I thought the motion looked so incredibly unnatural it was almost unwatchable. Now, it just seems normal. It had nothing to do with 60hz video itself - it had to do with the expectations built up by years of watching 29.97hz interlaced NTSC video...

dbatten··on The Ghost in the MP3 (2014)
I think the implication was that, once mp3 is no longer a household format, there will be a certain type of person who feels a nostalgia for the "sound" of mp3s, maybe even advocating that it was "better" in some way than the sound of whatever comes next...

This is basically what's happened with vinyl records at this point, right? People have a nostalgia for their sound, even though by any scientific or technical measure a vinyl record represents music/sound less faithfully than, for example, lossless CD audio...

dbatten··on Is this plane landing or departing? (2017)
Well, that's interesting. The person who originally wrote the Aviation SE question was claiming to be the original photographer and saying he couldn't remember if the plane was taking off or landing... But, if the internet had access to the whole sequence of images, surely he did as well? So why was he asking? Was it just a puzzle?

EDIT: From a different comment, it's clear that the photographer later found the whole series of pictures AFTER asking the question, and posted them online. So the photographer was confused, but later found the rest of his photos, which definitively show the plane LANDING. Case closed.

dbatten··on Is this plane landing or departing? (2017)
I remember this showing up in the SO sidebar a long time ago (what a time-suck that thing is). At the time, I remember thinking that some of these people must be trolling - and I still do.

For example, the people arguing that the wheels have already touched down. It seems abundantly obvious to me that the wheels have NOT touched down, and that the small strip of black asphalt running parallel to the wheels is a taxi-way in the background which has lots of small aircraft sitting on it. I get the uneasy feeling that anybody that sees it any differently is just trying to gaslight me. But, then, that was also the feeling I got about the blue dress, and that turned out to be all too genuine... People genuinely saw it differently, they weren't trolling.

To me, the lack of heat blur behind the engines seems to be reasonably conclusive evidence that it's landing... But who knows.

dbatten··on How does a gas pump know to shut itself off? (1981)
Yes. Petrol is "gas" in the US. And Canada too maybe?
dbatten··on Babylonian Mathematics (2003) [pdf]
I don't have the math in front of me for the first question, but I've seen it worked out in the past. It's on the order of 1 in a billion.

On the second question, considering "all the numbers in the Bible," an arbitrarily chosen number or set of numbers from the Bible will contain a pattern very, very often, because that's how people thought in that culture.

Jesus didn't have 12 disciples by accident. They symbolically represent the 12 tribes of Israel.

The genealogy in Matthew 1 doesn't contain 3 sets of 14 generations by accident. The author had to skip at least a few kings from the Jewish tradition to make it work out. And the fact that it's intentional is underscored by him explicitly saying "so, there were 3 sets of 14 generations..." (He's making a theological point, but the details of that aren't relevant to the question at hand.)

I could go on... Intentional numerical patterns are, in fact, quite common! And this gives us more, not less, reason to suspect that the Genesis 5 genealogy has an intentional pattern.

dbatten··on Babylonian Mathematics (2003) [pdf]
Yes, absolutely. Good point. Though one would still need to describe the pattern and how it was helpful as a memory aid... None of the proposals I've seen are simple enough that they seem useful for that purpose. But I'm admittedly from a different culture and time, so maybe what was obvious to them isn't obvious to me/us.
dbatten··on Babylonian Mathematics (2003) [pdf]
Related unsolved mystery in ancient mathematics - the Biblical genealogy from Adam to Noah in Genesis 5 contains 10 individuals. For each of the individuals, the genealogy reports the age at which their descendant was born, the remaining number of years in their life, and their age at death. 30 numbers in total. In the Masoretic Text (often considered the most reliable), those 30 numbers all end in a 0, 2, 5, 7, or 9. The chances of this happening by chance are effectively 0.

Possible explanations include either 1) that the numbers were corrupted, possibly when being converted from a sexagesimal system into decimal, or 2) that the author never intended the numbers to be "real" numbers in the sense we would understand them, but was instead intending to use a pattern to teach symbolic meaning. However, to my knowledge, nobody has ever offered a completely convincing interpretation of the puzzle. If #1 is correct, what were the original values and what was the process by which they were corrupted? If #2, what is the pattern that the author was constructing, and what is it intended to communicate?

Other genealogies in Genesis have similar weird features, and some have proposed theories that involve the other genealogies as well, but that's beyond the scope of this comment.

See - https://en.wikipedia.org/wiki/Genealogies_of_Genesis

dbatten··on Game Programming Patterns: Event Queue (2014)
Reminds me of the excellent post about the architecture of multi-player in the original Age of Empires game:

https://www.gamasutra.com/view/feature/131503/1500_archers_o...

You had high-ping 28.8k internet, and you wanted to be able to have immersive battles in a world with hundreds of units controlled by several players. There's no way you could send real-time information about the status of every object in the game. So they didn't. They made the game simulation completely deterministic and then simply shipped user input around on the network. User input was processed on a slight (couple hundred milliseconds) delay (to allow for network transmission speeds) and executed simultaneously in "game time" on everybody's machine, so the simulation stayed in perfect sync. Voila, no worrying about lag.

dbatten··on Woman makes $420k by buying insurance on flights she predicted would get delayed
Unfortunately, this article leaves the exact nature of information she was relying on vague/confusing.

> "she was able to receive relevant information beforehand that tells her if the flight was going to be delayed or cancelled. Thereafter, she would purchase the tickets to the flights that are likely to be delayed or cancelled, before checking if there is any extreme weather along the route the flight is going to take that day."

Specifically, I'm not sure what the "before checking for extreme weather along the route" bit is supposed to mean, especially since it seems to imply that she checked for extreme weather after she already purchased the insurance? Something is odd with the grammar here. Furthermore, the quote never specifies what the "relevant information" actually was.

So... does this mean that she received inside information (e.g., from a former coworker) that the flight had been cancelled?

Or does it mean that she was looking at publicly-available information (e.g., weather data) to make the call?

I think the ethics and usefulness of some aspect of this are dependent on the answer to that question, and I'm not sure the article says definitively one way or another.

With that being said, I think the ethical problems with the fake identities seem more clear...

dbatten··on Power tool manufacturers and who owns them
How did you discover it was actually a rebranded DeWalt (as opposed to a drill manufactured to less demanding specs by a company that also happens to own the DeWalt brand)?

I own a good number of power tools, mostly DeWalt, though I usually check Consumer Reports and other reviews and have bought from other companies when the DeWalt option didn't look like it was holding up well. If I could verify that a homeowner-oriented brand (as opposed to a tradesman-oriented brand) like Black and Decker was literally just re-skinning a higher-end tool and selling it for less, I'd happily take the cost savings, though.

dbatten··on Ask HN: How do you do home surveillance?
I know lots of people with home security or surveillance systems. I'm from a pretty conservative southern (US) subculture, so I also know a lot of people with guns that they own for home defense. And a lot of people who are really worried about break-ins and what not.

I know of zero stories in which these people's home security or guns prevented or helped resolve a break-in or other crime. In fact, unless I'm forgetting something, I don't know that I really know anybody that's had their house robbed at all (this is a privilege, I know)...

I do know other stories, though... Like somebody going downstairs in the middle of the night, gun-drawn because a relative got home from a trip a day early without telling anybody... Or somebody rushing home from work (armed again) because they got alerts from their home security system and couldn't access their cameras... only to find a Sheriff's deputy already in the house and... absolutely nothing amiss other than their kid leaving a door open.

And we haven't even started talking about privacy concerns.

Personally, I lock my doors at night and when I leave the house. If somebody wants to smash the glass in the french doors and walk right in, well, they're gonna be pretty disappointed in what I've got in there. I'm OK with that setup.

I know OP's question is about making the tech work in a safe way, but I can't help but wonder if all of the surveillance/security stuff is really necessary, unless you live in a high-crime area...

dbatten··on Tencent may have accidentally leaked real data on Wuhan virus deaths
I was wondering about this as well, but I'd suspect any signal would get lost in the noise. People getting sick and sleeping more, people working from home, people checking online for news about the virus, the government shutting down internet access to prevent the spread of panic, etc. are all things that would drastically change internet usage patterns, but wouldn't necessarily be indicative of anybody dying.

Not necessarily saying any of those things (e.g., shutting off internet access) have or will occur - I'm pretty ignorant of the situation on the ground. I'm just saying that from a statistical perspective, I'm afraid it'd be a needle/haystack situation.

dbatten··on Housing is a human right: How Finland is eradicating homelessness
Interesting! This seems to be the crux of the disagreement. What thinkers are you referencing? I'd love to read them.

At first blush, my thought is that you could still create a secular argument for natural rights, but I'm open to being persuaded otherwise.

dbatten··on Housing is a human right: How Finland is eradicating homelessness
> but surely it’s worth considering what society should work collectively to provide

Agreed!

> the mere fact that abject suffering exists in the world should not mean we can’t decide that there are higher order human needs within the effective purview of society

Again, agreed! I'm not arguing that we can't talk about universal housing. We can! We should! I'm just arguing against conflating what I see as two distinct types of rights - those that you have because you are a human, and those that governments choose to guarantee to their citizens.

> I get the impression that you’re playing pretty fast and loose with the word “innate”, to basically refer to the rights you think are most important

I'm not so sure. There's a big difference between a person (or government) taking away something I have by default as part of my humanity (e.g., taking away my life) vs. a person (or government) being required to provide me with something that's not guaranteed by my humanity (e.g., the fact that I will have food is not guaranteed just by virtue of me being a living creature with human DNA).

In the end, I can't stress this enough - I'm not saying a social safety net is bad, or that we shouldn't create political rights where the government should be obligated to provide food or shelter or whatever. Maybe those rights should exist! I'm just saying let's not conflate the concepts.

dbatten··on Housing is a human right: How Finland is eradicating homelessness
Of course it's in the UN's declaration... I just disagree philosophically with the UN's declaration.

Human rights, in my mind, are something that existed before the UN, and exist independent of the UN. And there's centuries of political philosophy that agree with me on that point. See, for example language like "endowed by their creator with certain unalienable rights" in the US Declaration of Independence. That tradition of political philosophy sees rights as inherent to humanity (in this case God-given, but it doesn't have to be), not guaranteed by some statement of a government or group of governments.

dbatten··on Housing is a human right: How Finland is eradicating homelessness
Ensuring you have counsel is intended to make sure that the government can't unfairly imprison/punish a person who is ignorant of the legal system and incapable of defending themselves. The end goal is still to be a check on government power.

That's still far different from the government being obligated to provide you housing.

(To be clear - I'm not taking a position on universal housing. It might be a great idea! I just think it's a different class of right, and I think the distinction is worth preserving.)

dbatten··on Housing is a human right: How Finland is eradicating homelessness
Right... but that's still different from "the government has to pay for you to have a house."
dbatten··on Housing is a human right: How Finland is eradicating homelessness
I agree. I really wish people would tweak the rhetoric just slightly - "housing should be a right" is a perfectly logical humanitarian cause to rally behind. It doesn't assume the right is innate, doesn't call it a "human" right, but still says housing should be something everybody in our society is entitled to. Reasonable people can then disagree about whether the humanitarian benefits of making it a right are worth the costs, etc.

Calling things that aren't innate to our humanity "human rights" is absurd - for housing, but especially for things like the internet. Should we try politicians in The Hague because they don't provide universal housing for their people? Should we impose international sanctions on countries that don't have widespread broadband? Of course not.

Conflating these concepts also devalues the concept of human rights, and reduces our ability to actually focus on human rights violations. Slavery, political oppression, violence, etc. are still a very real and special class of problem in our world. People's innate human dignity is being trampled every day by oppressive, violent regimes (and, even in some cases, by liberal democracies). Let's not water down our view of these travesties by conflating them with debates about the generosity of the welfare state in rich Western democracies.

dbatten··on Housing is a human right: How Finland is eradicating homelessness
Yes, on a surface level, these are framed as "stuff the government has to do for the people," but the root issue is still making sure the government doesn't take away the rights and freedoms people naturally have.

Written a different way, this would say "the government can't take away your freedoms by arresting you, accusing you of a crime, but then delaying or denying a fair trial to figure out if you're actually guilty."

dbatten··on The Rise of SQL-Based Data Modeling
Agreed on regression. You should probably be doing that level of analytical work in a stats package. Perhaps I should have chosen a better example to illustrate the "SQL doesn't facilitate creating abstract libraries that operate on arbitrary data" point.
dbatten··on The Rise of SQL-Based Data Modeling
> This implies that SQL is not reusable, causing similar code with slightly different logic to be repeated all over the place. For example, one cannot easily write a SQL ‘library’ for accounting purposes and distribute it to the rest of your team for reuse whenever accounting-related analysis is required.

Data scientist here. I think some of this problem is handled by effective use of views. Oh, everybody is constantly joining these three accounting-related tables and aggregating by, say, order number? Have your Data Engineer/DBA/analyst/whoever create a view that takes care of that. Boom. Now everybody's using the same data, calculated the same way, nobody's reinventing the wheel, and you don't have to worry about somebody fat-fingering something when they re-write that query for the 10th time.

With that being said, I still think there's some truth to this criticism, in that it's not as easy/common to be able to build an abstract query that does a common operation on arbitrary data. You can't import trend_forecast.sql, hand it arbitrary time-series data, and generate an N-month linear forecast from your historical data points. At least, not easily in ANSI SQL.

dbatten··on Wirecutter CEO Sells Top Reviews for Kickbacks (2014)
I absolutely agree. Consumer Reports is indispensable.

My only wish is that they'd focus more on reliability testing. I can evaluate for myself whether that new sedan handles nicely (or whatever). But it's wonderful to have somebody warn me the wheels are going to fall off after 70k miles...

dbatten··on There is no place like Route 301 (2015)
We used to drive from North Carolina up to New Jersey for Thanksgiving every year. We'd take 301 out of Richmond up until we got into Delaware. Going over the more-than-4-mile Chesapeake Bay Bridge[1] was always a treat!

1: https://en.wikipedia.org/wiki/Chesapeake_Bay_Bridge

dbatten··on The town's so full of these confounded dials (195 BCE)
I sympathize with this a lot.

"A Severe Mercy" by Sheldon Vanauken is one of my favorite books, if not my favorite. A significant theme of that work is "moments made eternity," where eternity is used to connote both a sense of timelessness and of heaven. The author finds that his (and his wife's) most joyous times were precisely those moments where they lost track of time and simply enjoyed something (e.g., gazing at the night sky stretching out above the ocean) as long as they cared to. When they were done, they couldn't tell if minutes or hours had passed, but it didn't matter. In fact, that was part of the joy of the moment. Ultimately, they find themselves longing for the Christian ideal of heaven, where eternity will be all they have and they'll no longer be constrained by time.

I too find that I enjoy such moments of timelessness... one in particular sticks out in my head. I had hiked with my 2-year-old son down to a creek in a local state park. I sat in the grass and watched while he threw pine cones and sticks into the water, splashed around, and just had a blast doing what boys do. When I checked the time, I realized that we had been there for nearly 3 hours, though I could have sworn it was only a few minutes.

I find clocks as useful as anybody else, but I also find escaping them for a bit to be a delight...

dbatten··on USS Pueblo
This is a thing that happens, actually! Just not with naval vessels...

https://www.theguardian.com/world/2010/nov/14/max-hardberger...

← PreviousPage 2 of 8Next →