AaronBBrown

In busy systems, GOMAXPROCS=cpu.limits will still lead to the process exceeding its cfs quota and be throttled as the Go runtime has additional threads beyond just GOMAXPROCS. IME, the only way to (nearly) guarantee there's no throttling is to set cpu.limits=GOMAXPROCS+1 to leave some room for these system threads. Unfortunately, there's broad adoption of uber/automaxprocs (which makes the same error) and utilizing the downward API to do this for other cases.

The truth is that most "big data" problems aren't big and can often be solved with awk and xargs.

Clicked expecting this to be about the history (https://officialrealbook.com/history/) of jazz fake books, which eventually evolved into the Real Book!

I remember the pure magic of playing Sierra games after adding a Sound Blaster. Dr Sbaitso was also a hoot.

This is a design flaw in Kubernetes. The article doesn't really explain what's happening though. The real problem is that there is no synchronization between the ingress controller (which manages the ingress software configuration, e.g. nginx from the Endpoints resources), kube-proxy (which manages iptables rules from the Endpoints resource), and kubelet (which sends the signals to the container). A presStop hook w/ a sleep equivalent to an acceptable timeout will handle the 99%+ cases (and the cases it doesn't will have exceeded your timeout anyhow). Things become more complicated when there are sidecar containers (say an envoy or nginx routing to another container in the same pod) and that often requires shenanigans such as shared emptyDir{} volumes that waits (with fsnotify or similar) for socket files to be closed to ensure requests are fully completed.

Right out of Dvorak Symphony 9!

Hey, this is Aaron from GitHub. We're using devicemapper w/ LVM backed pools. Would love to hear about your experience there. We definitely see this problem during periods of high container churn.

iTerm works with Cmd-[0-9] just fine.

Have you tried `hub`? https://github.com/github/hub

    ~/src/bar (foo =)$ hub pull-request
    https://github.com/foo/bar/pull/5

There is also `hub fork` and others:

    These GitHub commands are provided by hub:

       pull-request   Open a pull request on GitHub
       fork           Make a fork of a remote repository on GitHub and add as remote
       create         Create this repository on GitHub and add GitHub as origin
       browse         Open a GitHub page in the default browser
       compare        Open a compare page on GitHub
       release        List or create releases (beta)
       issue          List or create issues (beta)
       ci-status      Show the CI status of a commit

With close to a decade of remote work experience under my belt at companies that are more than 50% remote (GitHub) down to being the only remote guy, I have concluded that some things are best left in the physical world...

There is an actual whiteboard on my wall and a webcam on a stand. When I want to whiteboard something, I get up, switch camera angles, and literally draw it out. I have found this to be far more productive than any whiteboard apps out there.

For something more interactive, Google Draw isn't awful, but it's not a pen and paper.

I have had 3 Ecobee thermostats in my house for over 3 years. They have been extremely reliable and have an API, so I can do all sorts of fun stuff with them. Right now I'm working on hooking them up so my themostats change settings when the house is empty based on the presence of family member phones on our LAN.

And all of the config management tools have a comparable concept. Ansible 2.0 has 16 levels of variable precedence, Puppet has hiera which gives you arbitrary levels of precedence, and Chef has attributes, of which there are 11-15, depending on how you count.

To make this a fair comparison, you should be comparing chef-solo (or chef-zero) deployments to Ansible since the closet thing Ansible has to a dynamic source of truth is dynamic inventories. But we'll table the discussion of whether or not I should be required to build my own CMDB from scratch or not for now...

The Chef Server stack is somewhat complicated, though with an omnibus install you will rarely have to interact with it at that level. These days, it scales well up to thousands of client nodes with no meaningful effort.

I'm not sure what you mean by the server using its own versioning. Cookbooks are versioned, as they are deployable artifacts. Environments then define which versions of all the cookbooks exist, so you have an easy promotion strategy. The challenge with Chef Server and DVCS is that you have to centralize the uploading of Chef artifacts (roles, environments, data bags, cookbooks) to Chef Server if you have a team with more than 1 person. That challenge is not unique to Chef, however. Hopefully, a team of Ansible users are not running Ansible playbooks directly from their workstations. If you try to upload a version of a cookbook that is older than the current version, it yells at you, so it makes certain bad behaviors more difficult.

There is plenty I don't like about Chef. And Puppet. And Ansible...I've worked with all of them, but I still haven't seen a valid argument to support the phrase "over-engineered" when used on Chef. Of all the configuration management products out there, Chef is still the most flexible and the most powerful.

In what way is attribute precedence over-engineered? You can ignore most of it for most cases and just use default for everything, but when you need it, it's there for you. I don't see how it's more or less simple than what Ansible has defined: http://docs.ansible.com/ansible/playbooks_variables.html#var.... I count 16 precedence rules in Ansible 2.0.

Every time a new configuration management utility comes out, everyone loves it because of it's simplicity or some arbitrary measure of "lightweight," but once Systems Engineers need to solve real automation problems in the real world, they start adopting features to work around the illusion of being "lightweight."

I like Ansible for automating my home network where I have simple problems and no need for environment separation. For professional work, I stick with Chef or Puppet.

in what way is Chef over-engineered?

It's not a schooner, it's a sailboat!

By now, I have spent more than half of my 15+ year professional career as a remote employee. My first job, back in 2000 was as a developer working for a Boston-based company from my apartment in upstate NY. We exchanged source code via CD in the mail, used a telephone and screen shared with Microsoft Netmeeting. That... was hard.

Several companies later, I find it actually difficult to work in an office environment. I'm not a hermit or even a strong introvert. I enjoy the company of other people from all walks of life, but there are so many distractions in an office environment that prevents me from doing my best work. As a Technical Operations engineer (SysEng, DevOps, SRE, or buzzword-dujour), most of my work is contemplative and best performed away from a keyboard. At home, I am free to work while running, walking, or sitting in the bathtub (TMI) with no fear of judgement from my peers. When I do need access to someone, I have a variety of tools at my disposal to suit the required latency and interactivity of the conversation.

Plus, pants are optional and slippers are the prescribed footware.

There are always challenges, though. For me, strict enforcement of work hours is a must. Sometimes I still get in the zone, lose track of time and work late, but usually I sit down around 9AM and take off around 5-6PM. Otherwise, work takes over my life and I end up neglecting my responsibilities as a husband, father, and homeowner. I am lucky to have a wife who understands my work boundaries and children who have never known a father who leaves the house for work each day. "Daddy is working" is enough for them to get them out of my office. Oddly, I grew up in a similar environment, as my father worked from a home office when I was a kid.

The biggest challenge for me has been while working for a medium-sized (1000+ employee) company in a Director role for a mostly colocated company. As an individual contributor, communication with my team and boss is relatively easy, but as a leader it is incredibly difficult to inject myself into important conversations via chat and email when those people are literally sitting next to each other. Having trusted "spies" within the organization that will keep their ears open and report back is a must or else you end up completely out of the loop.

Having worked primarily at colocated companies where I'm in the minority as a remote employee, I'm very excited to be joining an established remote first company next year and seeing how they do it in the "big leagues."

I am brand new to Go am trying to learn it together with my bright, but young (6 year old) son. Can anyone recommend some good introductory resources for to better learn the basic concepts and scoring? While general gameplay seems relatively straight-forward, I've found there seem to be a variety of rules and I don't understand the algorithm for scoring and determining the winner.

The second part of your question is a big part of the answer...an IRC server needs to be well tended in order to support the features that people want. I run a Technical Operations (DevOps, SysEng, whatever you want to call it) team...the last thing I want them spending their time doing is futzing with IRC software when there is a solution that takes 5 minutes to set up and is ready to go. Businesses should focus on their core competencies.

For me, the biggest problem was when my computer was turned off overnight, I had to go to some external source to read the transcript of the conversations that transpired rather than having it available immediately, in my application. DMs can also be sent when the user is offline, which is great for "hey, get back to me when you're back on."

I was able to solve some of these issues by setting up my own Quassel or ZNC server, but these were hacks and the clients were no where near as refined as Slack or HipChat.

Unless I'm currently unemployed, if someone asked me to do this I would politely say, "thank you, but no thank you." It has been my experience that there are oodles of development and operations jobs out there and not enough candidates to fill the positions. As a prospective employee, if I have multiple companies interested in hiring me, I'm not going to take days off of work in the off-chance that I might get an offer from you. And giving that time up for free is a non-starter. At least pay me a market-rate contracting fee.

I keep hearing this advice repeated. Maybe I live in Crazytown, but I doubt it's realistic at any scale and likely self-selects for young, inexperienced, or presently unemployed candidates.

This is why I purchased an ecobee (www.ecobee.com) instead of Nest, despite the obviously superior interface provided by Nest.

Strictly speaking they are 16 bytes when stored as binary, which sounds grat, but when represented as strings (as they usually are) they are 36 characters which is 36-bytes in ASCII, but 108(!) bytes in 3-byte unicode.

So, in practice they are terrible primary keys in systems that do not have a native UUID type. In that case, an 8-byte bigint generated externally by a ticket service like Snowflake is often far superior.

I was half expecting this to be about Apache Kafka.

What language would you write it in?

All decent musicians adjust every note on every instrument (where possible) in order to be in tune with themselves and the others they are performing with. Musicians that don't do this quickly find themselves out of a job (or making millions as the next big pop act). Mostly, this is not done particularly consciously unless you find yourself performing with a group that is either far more accomplished or far less accomplished than yourself. Of course, there are exceptions, namely open strings and piano. However, even in the case of conventional stringed instruments, it is common to adjust fingerings to avoid open strings in order to maintain tone color and more easily adjust intonation.

Basically, while studying temperament is interesting, the reality is it rarely causes practical issues. Most pianos are tuned with equal temperament, which is good enough and almost every other western instrument has enough flexibility to adjust "on the fly."

I went with ecobee thermostats (ecobee.com) because it already has a rich API. It doesn't have motion detection like Nest, but it does have a very useful web interface and a decent app for iPhone and Android. Since I work from home and have a stay at home wife with two young children, the motion detection was not a selling point for me. I wanted something a bit more Cisco router and less Apple Airport Extreme. I've been very happy with the 3 that I have in my house. My only complaint (and this is true of all connected thermostats) is that it relies on the company's infrastructure. If ecobee goes out of business, I lose much of the utility of the devices, though they don't turn into bricks.

That is an implementation problem. The behavior is still incorrect. MySQL manages to do this correctly, so it's clearly not impossible. I can't come up with any other conclusion except that transaction isolation in pg is fundamentally broken when I see this type of case show up.

This violates the promise of read committed. In RC, all transactions are supposed to see the most recently committed version of a record. The fact that it was a delete, then an insert is an implementation detail and "not my problem" as a DBA. At the time the transaction in session 1 commits, the most recently committed version of k='a' is v='c'. PostgreSQL, in the situation, seems to be operating in read uncommitted isolation level...session 2 is treating the uncommitted delete in session 1 as the most recent version.

I'd like to give PostgreSQL the benefit of the doubt here, but this behavior seems fundamentally incorrect to me.

The DBA in me is having a difficult time understanding PostgreSQL's behavior with regard to the delete/insert within a transaction?

In MySQL, I get the exact behavior that I expect with anything above RC isolation level (which is the default for PostgreSQL):

  session 1> create table foo ( k varchar(10) not null, v varchar(10) not null ) ;
  session 1> set transaction isolation level read committed;
  session 2> set transaction isolation level read committed;
  session 1> insert into foo (k,v) values ('a','b');
  session 1> begin;
  session 2> begin;
  session 1> delete from foo where k = 'a';
  session 2> delete from foo where k = 'a';
  -- transaction hangs here as expected since session 1 has a lock on this record
  session 1> insert into foo (k,v) values ('a','c');
  session 1> commit;
  -- session 2 executes the delete
  session 2> insert into foo (k,v) values ('a','e');
  session 2> commit;
  session 2> select * from foo;
  +---+---+
  | k | v |
  +---+---+
  | a | e |
  +---+---+

When I do this in PostgreSQL, I end up with 2 rows in the table, which seems fundamentally incorrect and to me implies that PostgreSQL's implementation of read committed is flawed and should be fixed. The transaction in session 2, at RC isolation, should see the most recently committed result, which is the single record from session 1. I'm not CJ Date, though so am very likely missing some subtlety.

PostgreSQL after the sequence above:

  test=# select * from foo;
   k | v
  ---+---
   a | c
   a | e
  (2 rows)

What am I missing? Why is this considered correct behavior?

It does, but it's not very useful, as you cannot have a VIP span multiple AZs and typically a spontaneous instance failure is correlated with an AZ problem. ENIs (and IPs) are tied to a subnet, which is tied to an AZ. Even the example only utilizes a single availability zone. While there may be a tiny bit of additional redundancy added by doing this, you still need a way to fail over to another availability zone.