szopa

Well, a spreadsheet used as intended is mostly read-only, that's no necessarily true if it's backing a website :)

Huh. I bet that in the Docs SRE team at Google there's a senior SRE who's having a "I felt a great disturbance in the Force moment." Google Sheets is awesome, people at Google are smart, but they are not capable of magic.

In the case of Sheets, the most reasonable sharding scheme for the database backing it would be based on some collection of spreadsheets (in the extreme case it would be just one spreadsheet, but that probably wouldn't be practical). The important assumption here is that all the traffic pertaining to one spreadsheet will ultimately go to one server. So, if there's too much traffic coming related to one spreadsheet, you cannot really scale horizontally by adding more servers – you have to give it more resources. A single Google Sheet is sometimes used by lots of users at the same time, but they are able to limit it from the frontend (for example by making it degrade gracefully).

If the traffic comes from the API, it's going to be a little bit more tricky – especially that IIRC the current rate limiting for the API uses a daily quota, leaving room for some really nice spikes.

So, as soon as a website built using this hack get somehow popular someone is going to have a really rotten day with a lot of pages.

(Of course, this is pure speculation on my side, I have no idea how Google Sheets is actually built – I would be very curious if there's a smart way of overcoming this sort of issue.)

Yeah, being oncall for something like a well behaved backend system may be quite nice. I was an SRE for YouTube, and it was an almost constant bloodbath. YouTube's code changes at a relatively fast pace, there's a ton of developers, and it depends on a ton of different backends, and a problem with any of them would make us suffer. To make it worse I had a bit of bad luck, and was a magnet for weird and unlikely outages (bracing for my shift was a running joke in the team). So, if I was oncall at home it usually meant that shit started happening before breakfast and I didn't get enough quiet time to take my 10 minute bus ride to the office :)

This said, I really enjoyed the experience. Yes, it was tiring and stressful, but it was also super interesting and exciting. Being responsible for such a huge site was incredible, and the feeling of figuring out how to overcome a big outage was exhilarating. I actually miss the pager drama from time to time (my wife does not).

I would love to see the sericon experiment described in the article, but I cannot find it on YouTube.

What is more, it is possible to find an Agnieszka working at Nabino through a Polish LinkedIn clone...

The iterator pattern I have in https://github.com/ryszard/goskiplist uses an interface similar to

  type Iterator interface {
      // Next returns true if the iterator contains subsequent elements
      // and advances its state to the next element if that is possible.
      Next() (ok bool)
      // Value returns the current value.
      Value() interface{}
  }

which you then use like this:

  iter := s.Iterator()
  for iter.Next() {
      fmt.Printf("%s\n", iter.Value())
  }

I think this is a pretty good pattern: it's concise, looks less ugly than the callback and closure based iterators, and it's not using anything heavyweight like channels, so there should be no significant performance degradation.

Out of curiosity, I added this pattern to OP's test code as StatefulIterator (https://github.com/ewencp/golang-iterators-benchmark/pull/1), in 2 versions: one in which the iterator is a struct, and one in which the iterator is an interface. The results look like this (the 4 last rows are my additions):

  BenchmarkIntsCallbackIterator	         500	   3478962 ns/op
  BenchmarkDataCallbackIterator	         500	   4437885 ns/op
  BenchmarkIntsChannelIterator	          10	 186095017 ns/op
  BenchmarkDataChannelIterator	          10	 188627451 ns/op
  BenchmarkIntsBufferedChannelIterator	  20	  90941766 ns/op
  BenchmarkDataBufferedChannelIterator	  20	  90175550 ns/op
  BenchmarkIntsClosureIterator	         500	   5580257 ns/op
  BenchmarkDataClosureIterator	         500	   6183472 ns/op
  BenchmarkIntStatefulIterator	        1000	   2653265 ns/op
  BenchmarkDataStatefulIterator	         500	   3582051 ns/op
  BenchmarkIntStatefulIteratorInterface	 200	   7516778 ns/op
  BenchmarkDataStatefulIteratorInterface 200	   8250702 ns/op

  ok  	_/Users/ryszard/Projects/golang-iterators-benchmark	33.717s

As you can see, the struct version is the fastest of the lot: 2653265 ns for ints and 3582051 ns structs (versus the callback version with 3478962 ns and 4437885 ns, respectively). The interface version is slower, but still in the same ballpark as the callback and closure based ones, while being much more readable (but that's only my personal opinion). The not-so-stellar performance of interfaces is a bit disappointing, but that is a known issue.

I am not sure what OP wants to say with this submission... Is it that the US treats differently people who defect to them than people who defect from them? That's... not very surprising. In some way it feels more fair than the way the Soviets treated Kim Philby, a British spy and defector: http://en.wikipedia.org/wiki/Kim_Philby (tl;dr: he wasn't a KGB colonel like he was promised, he was kept essentially in house arrest, and had a 500 rubel allowance).

As for Kukliński, being a Pole myself, I have mixed feelings. I very much prefer that my country is in the American sphere of influence (and not the Soviet as it used to for almost 50 years). However, if WW3 had happened, we would've been be on the Soviet side. Zbigniew Brzeziński said that Kuklinski’s information had permitted them to make counterplans to disrupt command-and-control facilities rather than only relying on a massive counterattack on forward positions, which would have hit Poland. However, I have no dellusions that anybody in the US would have any doubts about turning Poland into a radioactive wasteland if that served the American interests of the moment...

Also, the Polish government after the fall of Communism wasn't very eager to pardon Kukliński. That happened only after Clinton's administration threatened to block Poland's NATO accession if he weren't exonerated.

A prairie dog shaped robot could work better.

I went quickly through the paper, and there are some interesting ideas, like separating scheduling, sequencing, and storage.

Also, there are some details that leave me a bit confused... For example, they mention they use ZooKeeper for Paxos, while ZooKeeper uses a different protocol, ZooKeeper Atomic Broadcast.

> But it didn't touch the important question -- is the development time for Go faster than for scripting languages with critical sections in C?

At YouTube (which is a predominantly Python shop) we use Go for some elements of our infrastructure, and I spend approximately half of my time working in each language (main project in Go, integration tests in Python). In my experience easy things (say, split and massage some strings) are more complicated/annoying in Go than in Python (and much more verbose). However, difficult things usually end up being much simpler in Go than in Python – especially if they involve concurrency (Python threading is a trainwreck). The difficult parts of our code consume most of our brainpower, so that's a net gain from our point of view. What is more, for some weird reason Go code seems to be less susceptible to bitrot (go fmt FTW).

The one thing that's really huge in this proposition is that it would allow spouses of H1B visa holders to work in the US. The fact they cannot do it is a particularly cruel aspect of the current immigration system in the US.

We moved to the Bay Area when I got a job at Google a year and a couple of months ago. We really like it in California: my job is super interesting, the weather is nice, and the people are amazing. I cannot imagine any place in the world where I could have a bigger impact being a programmer. However, going back to Europe is a topic of our everyday conversations. Why?

My wife is one of the smartest people I know. She is the webdesigner you would love to work with (you know, those who can actually code their stuff in a way that works with all browsers, use source control, and are not afraid of using the command line). She worked as a project manager and her team loved her. She has interesting things to say about startups, business models, and Goedel's theorem. She used to have a career back in Europe, but in the US she's almost a non-person. She can live here, yes, she can drive a car, but she doesn't even have her own SSN. She cannot do any productive work, and getting an H1 visa if you are not a software engineer is extremely hard (we had this conversation with a couple of immigration lawyers).

I have my dream job, but at the same time I cannot help the feeling that I am ruining my wife's life.

Interesting. Do you have any way of checking that the change has actually propagated through the system before starting to act on it? Is the system consistent at all times?

If I understand correctly, the client can connect to any instance and its request will get routed appropriately. Let's assume that you take a master offline and promote one of the replicas to be a new master. Won't that lead to a window in which (from the point of view of different instances) there are two masters at the same time and some writes are sent to the wrong instance?

EDIT:

One solution for such things is to use something like Zookeeper (or some other system whose documentation mentions "Paxos" ;)). Have you considered that? How does what you are doing compare with that?

Nice work! It seems that you are well aware of the tradeoffs that you are taking and communicating it openly in your documentation (and your choices seem to be very reasonable). I really like the tone of your communication – it seems essentially BS/koolaid free.

1. How much data can you put in one instance before seeing performance degradation? I know that you still working on good benchmarks – but do you have any ballpark figures?

2. How does replication work? Is it closer to row/document or statement based (or something completely different)? How fast is the replication?

3. What is your envisioned used of the replication? Are replicas supposed to serve read traffic, or their goal is to keep the data safe in case of a catastrophe?

4. Can you tell me something more about cluster configuration propagation? The Advanced FAQ answer doesn't get into much detail.

5. Am I correct to assume that you are using protocol buffers? What motivated your choice?

The origins of /etc are lost in history. Wikipedia [1] says that at Bell Labs /etc was pronounced "et caetera," and contained files that didn't belong elsewhere. And it had the advantage over conf or misc that it was only 3 letters.

[1] http://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

I DO work at YouTube, and for some time I've been working on CMS, YouTube's frontend to ContentID that allows copyright owners to claim their content, set automatic policies, etc. I can confirm that what Slapshot wrote was essentially correct.

This case was an automatic audio match. Somebody from BMF has looked at it, and the claim has been released. Note that this happened around 5 hours after the dispute, and less than 8 hours after the automatic claim was created, which I believe is a pretty great turnaround time on a weekend.

A dead comment by DanBC has some interesting links about how Clippy was dumbed down before its release in Office 97:

"Clippy was crippled by MS when they implemented the working research project into the broken version used in Office. (http://robotzeitgeist.com/2009/08/lumiere-project-origins-an...) The original implementation was deemed "too cautious" by PHBs, and thus a stupid kludge was added to make clippy appear more often. (http://lesswrong.com/lw/7wt/funny_even_clippy_can_be_blamed_...) (Interestingly that URL mentions siri before it was released)."

What you are saying is incorrect.

YouTube has deals with major content producers (music labels, movie studios, etc.) which work in the following way: whenever we discover a video that infringes their copyright, they can choose to either block it, monetize it or track it. What blocking means is quite obvious; monetizing means that we put ads in front of the video and share the revenue; tracking means no ads are shown, but the owner can access analytics about the video (how often it is uploaded, how many people watch it, where from, and so on). You can read more about it here: [1].

As you may imagine, this system changes the incentives for copyright owners. When they decide to block some video, all they achieve is making some YouTube uploader unhappy. If they decide to monetize, on the other hand, they start making money from the upload, and the clip becomes, effectively, a marketing tool for their product. If you, say, watch clips from a TV series on YouTube, that may be a signal that you like the series and will eventually watch it on TV or iTunes. What is more, uploaders try to select clips they find interesting and think other people would like to watch – so this in some way turns YouTube into crowdsourced advertising.

(Disclaimer: I work as a programmer at Google and I work on YouTube.)

[1] http://www.youtube.com/t/contentid

This reminds me that in Argentina in primary school we would make an improvised football (as in: soccer) by crumpling some paper and then wrapping it in scotch tape. It was of course much smaller than the real ball, but it worked surprisingly well (and broken windows were not an issue). Now I know why :-)

I am 26, and when I was living in Europe (Poland) I didn't even bother to get a driver's license - getting around by public transportation was cheaper and faster (especially if you take in account the time you need to find a parking spot and walk from it to your destination). However, moving to the Bay Area made me reconsider. Not being able to drive is a serious handicap. Not only there's no public transportation, but also everything seems to be optimized for cars (no sidewalks in some places, traffic lights giving a low priority to pedestrians, highways all over the place). I tried biking, but the cars seem unfriendly in a way that scares me.

It is really interesting how the socially acceptable level of honesty varies from place to place. I've recently moved from Poland to Northern California, and the expression "dancing on eggshells" describes very well my experience.

In Poland, I would classify myself as maybe a little bit above the average in terms of the sort of honesty described in the article (and that mostly with people I knew very well). Here, if I don't control myself I end up being the awkward says-before-thinking guy...

What bothers me most, however, isn't not being able to speak my mind; it's the lack of feedback from people around me. I am always left wondering if stuff like "I really like your other glasses" isn't actually code for "you look like an idiot in those glasses, and I am embarrassed to be seen with you on the street." I can only hope that with time I will be able to calibrate my sensors for California...

(On a side note, I feel really glad to be married. Dating must be hell in California ;)

That's sort of confusing. In a number of places I've read an explanation that what is described in the Wikipedia article as "Mid-Atlantic English" was in fact higher class New York English before WWII (one of its most characteristic features was a non-rhotic "r"). These sources also tend to use Franklin Delano Roosevelt as an example of this sort of accent.

For example: http://en.wikipedia.org/wiki/New_York_dialect#History (that article quotes the

ACtually, lingua franca was NOT French. Also known as _sabir_, it was a pidgin language used by the sailors in the Mediterranean (it went extinct in the XIX century). It was a mix of various romance languages and Arabic and Turkish. At first the main romance language mixed was French, then the influence of Spanish and Italian become more notable. The name was coined by the Turks (for whom people speaking romance languages must have looked similar enough to call them all "Franks").

It's funny how now most people seem to know Maciej Cegłowski as "the creator of Pinboard." His blog, Idle Words [1] is one of the best I know of. I would even say that the piece about Scott and scurvy [2] was the most interesting thing I read on the Internet in the last few years.

So, I am very happy about Pinboard.in being successful, but I think it's a bit of a shame that such frivolous endeavors take Maciej's time from writing his blog :P

(A bit more seriously, it's really nice to find a compatriot doing something cool in more than one domain.)

[1] http://www.idlewords.com/

[2] http://www.idlewords.com/2010/03/scott_and_scurvy.htm

I had a free weekend, and the thsrs idea by Ironic Sans [1] looked like something too cool to let it go (and http://thsrs.com/ didn't work too well for me). Also, it was a chance to implement my very own instant search ;-)

The thing is done in Django+PostgreSQL+jQueryUI and uses the Wordnet 2.0 thesaurus which comes with OpenOffice. My wife was kind enough to make it look not ugly.

[1] http://www.ironicsans.com/2008/07/idea_thsrs_the_shorter_the...

You've listed three browser based editors and one which seems to be completely focused on JavaScript editing. If I understand correctly, Kod is supposed to be a universal editor for programmers, only scripted in JS.

(But thanks for the link to cloud9, I didn't know about it and it looks interesting.)

EDIT: meh, it was actually 4 browser based editors :(

Comments like the parent make me sad. It could be summed up as "node.js, damn hipsters." It's just another form of fanboyism, except negative.

There are not many good, scriptable editors for programmers. Actually, I would consider only 3 for serious use: Emacs, Vim and TextMate. Of these, only one has really good integration with OS X.

If I was considering switching editors, the language that is used to extend it would be one of the most important factors (I love Emacs, but Emacs Lisp is seriously broken). The fact that Kod uses node.js means two things:

1) It's JavaScript, so it's high level, has nice semantics and it's extremely popular;

2) It uses a reasonably fast dialect of JS, and has the necessary infrastructure, like CommonJS modules and access to the filesystem.

So, Node.js is the thing that makes this topic actually hacker-news-worthy.

Are you the author of Kod (that's what some of your comments suggest)? If yes, do you have a roadmap? Do you want to make it open source? I would be pretty excited to try it, out even if it was in scary-alpha state.

This is quite exciting. Anything is known about the authors or a release schedule? I would love to beta test it.

It's because Emacs is one of the few editors where you can actually manage 90 and more buffers -- thanks to the fact that you can refer to them by name.

Emacs is a fantastic editor (and I think I can say that I am what you call a power user), but Emacs Lisp is a terrible, terrible language. The more I use it, the more I hate it. And it's not because of the lispiness or parens -- Common Lisp was my Language of Choice for many years.

Having an editor that I could extend in a real programming language (and JS is actually very nice once you grasp stuff like prototype inheritance and learn to avoid the... uhm, bad parts) would be simply awesome. And if it was pretty and integrated with OS X I would be willing to pay hard cash for it.