thinkersilver

This is how it worked out for me. I tried to build the system myself and note the abstractions, edge case. It didn't make me an expert in all things but gave me the right questions to ask. As you do enough of these your expertise will crystallise in a particular domain.

Started by building my own build system, monitoring solutions, taking at stab orchestration problems, graph databases etc.

The problem statement would be super simple like how do I install my application on 100 servers? And I will start off with bash, tar, ssh and work my way up.

The goal isn't too finish. You want to discover the problems first hand, decide upfront how you would evaluate an acceptable solution, attempt a naive solution, and then look at an open source system, note the family of algorithms + data structures then loop back and move onto the next set of problems in the domain.

Ideally the system you're building is distributed.

Mostly I work from the outside going in and counterintuitively I start with the two non-functional requirements, as questions: how fast do you want it to go, secondly how badly can the system fail. I find that I'm peculiar in this approach.

This won't work for everyone. It has for me because it's allowed me to build a handy library of interrogative and evaluative heuristics and develop strong understanding system architectures and design

Am looking forward to the book. Read the first one in the early mid 2000s and it changed my view of programming forever. Java in the modern climate isn't the best choice anymore to do this.

The move to a less class-centric approach is a much needed and welcomed shift. It helps to illustrate the spirit of the pattern or technique without getting weighed down with the restrictiveness and arguably dogmatic nature of java.

The books are similar but very different at the same time.

Martin's book takes an application focused approach. How do I think about changes to my data in a distributed environment? You can see this thinking in the projects he's been involved with on Kafka and at LinkedIn.

Brendan, however take on distributed system is from the angle of heterogenous distributed workloads and the architectural blocks needed for running such systems reliably. Note that Burns worked at Google and was one of the founders of Kubernetes so you will find a bit of a container orchestration slant in the book too.

That's a pretty big 'if', though. Hiring good people is difficult. Having bad people is more common and that may be the justification for methodologies.

They don't exist to make good people more productive but to make mediocre hires marginally productive.

This methodology is an effective way to leverage the developers in your team. It does rely on a certain discipline that juniours often don't have to drive and refine an exploratory solution into production code.

It requires curiosity, an openness to measuring and validating an idea and then driving it home. This sounds a lot like what I'd want from a team lead; but as someone pointed out some juniors aren't destined for leadership either.

I love the idea and have made it work, a lot less though than desirable.

I can understand the frustration. Been watching this space to see a plausible story on having storage volumes available to containers from any host.

Histpath and emptydir aren't really acceptable as solutions since they add complexity to your cluster set up.

There are some cluster storage technoloies out there but there are no tutorials, or overviews detailing, the pros and cons, and their performance limitations in a k8s environment.

Networking was the hot topic this time last year but storage is the topic no-one is willing to talk about. Tectonic is attempting to build a solution designed for the container use-case[1]. But that's a long way off.

If you are building cloud-native applications on-prem and you need state you're on your own for now.

[1] https://coreos.com/blog/torus-distributed-storage-by-coreos....

Long way to go in catching up with Azure and AWS in relation to market share not in capability. I should have clarified that.

ECS wasn't too bad. The approach they took made sense when it first came out but Amazon haven't evolved the idea since then. Personally I think that GCE's edge won't come from their container support since AWS and Azure both support k8s but will come from their value-add analytics and data services like BigQuery, Dataflow and so on. If Azure and AWS can't compete on that front then GCE will start to cannabilise the customers who are only still using the respective cloud providers only for infrastructure. GCE still has a long way to go before it becomes a serious contender.

First off, I'd like to say that I think this is great and we need more books in this space.

I hope this can be seen as constructive criticism but I do have a few comments on the layout and content.

1. Free chapter - perhaps chapter one or two may have been a better choice. I'd like to know the philosophy behind monitoring expressed in the book before diving into the details.

2. capacity planning - A chapter on this would have been great. Most teams I've worked with have struggled with sizing, planning resources required and archiving strategies with their monitoring solutions.

3. Monitoring strategies for different levels in the stack - where do I start, what should my short term goals be and so on.

4. naming of some of the chapters are too focused on the technology- for example the chapter on logstash could have been renamed to something to do with application logging or log scrapping.

5. visualisation and communication of results - there could have been a chapter on dashboard and reporting. This is a common issue with teams trying to understand how to do this.

This was written in a bit of a hurry but I hope my points came through.

The ClusterHQ guys behind the FlockerHQ found this out the hard way [0]. Initially Flocker was meant to provide a container data migration tool on top of ZFS, now it is a front-end to more established storage systems like Cinder,vSan,EBS,GPD and so on.

[0] https://docs.clusterhq.com/en/latest/faq/#what-happened-to-z...

The logger in me appended hostname and the tty to the file name and the log message. You could also add a user name to this as well[0].

I love this idea because you can easily analyse your commands then modify or automate your workflow where it makes sense. I've done this in the past to create a two letter aliases to frequently used bash commands.

[0] export PROMPT_COMMAND='if [ "$(id -u)" -ne 0 ]; then echo "$(date "+%Y-%m-%d.%H:%M:%S") $(hostname) $(tty) $(pwd) $(history 1)" >> ~/.logs/bash-history-$(date "+%Y-%m-%d").log.`hostname`; fi'

I love the jvm, I love the tooling and the ecosystem, but Java I do not love. The author presents many of Java 7/8 features as a reason for Java's greatness. The language features of Java 7/8 are really not that compelling. Let's consider that C# has had much a more progressive approach to language design for a while now.

Most places I've worked at recently have moved to Java 8 because 7 has been EOLed. It feels like we were pushed onto 8 rather than enticed by its shininess.

What makes Java great isn't java it's the jvm. It's such a brilliant piece of engineering. What makes the jvm 'meh' is it's association with java. The arguments the author makes about Java being great are in my eyes really about the JVM, its tooling and its languages.

Canon is known to do this alot. I think the magic lantern software illustrated this quite well when they started unlocking raw capabilities in cameras as old as the 50d, a ten yr old body.

Upgrading your camera becomes a frustrating ordeal. Never knowing if you're getting a good deal or you're getting conned.

Speaking mainly from experience, kubernetes isn't that hard to get set up. Unfortunately the documentation tries hard to convince you otherwise.

Leading up to Kubernetes v1, getting set up was quite painful but the api is now stable. So now I have an environment running Kafka, Cassandra, Spark and various java based services and am quite happy with it so far.

I've started using the hawtio kubernetes console to visualise my containers but may write my own tool to do this. I might also write a blog post detailing how to install, configure pods, expose a service and use the logs to diagnose problems. It's usually the first thing you'd want to know when starting out sound of GCE infrastructure.

The title is slightly misleading but not entirely. Docker was, at first, a container technology. However, after it received its funding it has tried to convince the community and investors that it can be more and started to provide container orchestration tools, i.e Swarm, Machine and so on. Docker has received most of its momentum and attention because of the work they did around the container format. Docker build, run, stop, encapsulates most users experience of Docker.

Docker wants to be known as a platform providing container management tools and it needs to do this quickly because Mesosphere and Kubernetes provide this type of functionality at a more mature level, albeit using somewhat different philosophies. The container format part of Docker is ultimately replaceable.

Google (Ventures) backing Tectonic is significant because Tectonic will provide a commercial enterprise-ready distribution of Kubernetes supporting Rocket. Rocket doesn't need to reach feature-parity with Docker to be a notable replacement because Docker already does too much. All Rocket needs to do is provide the much needed enterprise features that Docker is lacking and integrate well with Mesosphere or Kubernetes. If this happens before Swarm and Machine mature, we could be wondering 6 months down the road what the hype around Docker was all about.

If Tectonic succeeds in the enterprise marketplace then Google will have stealthily marginalised Docker using community efforts through rkt and kubernetes and not had to fight them directly.

Any word on how far off Docker extensions/plugins are? Haven't seen much recent talk on the github issues raised around Dec.

The pace of Docker interoperability is frustratingly slow

Kubernetes is going to become the standard api for container orchestration only because there are no other tools out there trying to do as much. There was a vacuum around container orchestration tooling and Google got there first. Kubernetes components can be swapped out for other community-driven efforts, take mesos as an example, which can be used to replace the default k8s scheduler. With k8s you can avoid lock-in with different cloud providers. I think Google is hoping that we end up on their cloud platform but its nice to see that it is being built from the ground up to be used with other cloud platforms.

There's a fair bit of overlap between what OpenStack/LXD does and Docker/Kubernetes or DCOS do. Both technology stacks have approached the problem from different directions.

Kubernetes is trying to be the standard for container orchestration and cluster federation for hybrid cloud environments but lacks mature storage provisioning and network isolation tools. Openstack has the virtualisation infrastructure box ticked but lacks the container orchestration story. Docker has the advantage that it is developer friendly and low friction and Openstack ( fairly or unfairly so) is perceived to be complex and unwieldy.

Personally, I think Open Stack/LXD is a dead-end and Ubuntu should take on Openstack/Mirano/Kubernetes while the Docker ecosystem matures. When Docker/Kubernetes storage/network tools have matured ditch OpenStack entirely.

Moving from what the book describes to a real workable system is a bit of leap. I remember hambergs.no article. It helped immensely and I ended up writing up my own system with a few personal tweaks. GTD has many interpretations and many implementations and it is hard to know what to pick. It is just as easy to lose site of the goal: Make your life easier and more productive.

This was my experience. The question I had were surprisingly hard to answer.

1. What is GTD and does it work? 2. If GTD is failing for me is it because I am not doing it wrong? 3. If GTD is failing for me is it because my implementation is wrong (i.e wrong tool or badly designed app)

The problem with the GTD system, as much as I have enjoyed using it, is that there is no reference system to verify correct usage. The usual path taken, for those who are patient and motivated, is that of hacking your way through the books, tutorials and apps out there until it feels right. The 'it feels right' feeling comes if you actually feel like you are on top of your work load. The system you end up with that works for you (when you eventually get to this place), may look very different from what the book described or what you initially envisaged.

GTD is more like meditation than a system. It's hard to learn it by reading about it. You only know you are doing it right when your life feels lighter and more productive.

His comments on logging are spot on. It's the only part of the post that I can agree on. You can get process monitoring as long as your tools are cgroup aware. Network monitoring is not so easy. This has been pointed out on the docker blog in the past.

Getting your logs out of docker is a PITA, though. Your best bet is to use syslog and configure each application to send their logs to a syslog server. It's a consistent widely supported way of shipping your logs around. Relying on stdout logs isn't always enough. Many applications do understand syslog out of the box but do not necessarily send important messages to standard out. Dockers management of stdout logs shouldn't be relied on at this stage.

Logspout does look interesting and I wish I knew about this a few months ago but see my above comment on stdout.

Not necessarily. It has little to do with learning to speak in a higher status manner. The comment box doesn't allow for a clear concise explanation but the language shift is something that the brain does on a subconscious level. A person's emotional state is related to how they see themselves in relation to their world at that moment. The 'language shift' is a manifestation of that. It is a tell in a way, the same way different clusters of body movements are, unlike body language, though, it is something that goes undetected by either party and hence can't be actively expressed or detected. That's why the LWC software was built. Check out the book that Pennebaker wrote. It does a better job explaining this than the article does and it will clear any confusion. Pennebaker doesn't provide an explanation for why their's a need to be more personal when assuming a lower status or when experiencing emotional trauma but he does provide plenty of evidence for the relationship.

This looks like an interesting project. The struggle I've had in the past with similar libraries is the level of extensibility provided by the API. It's not clear from he docs how to add your own custom viz. I would love a d3 based chart library that would provide an easy to use chart api,such as we see in c3js. Which I do like. But also a DSL that is not as low level as D3 but provides enough flexiblity for creating new visualizations. There have been some nice declarative charting libs posted on HN, which links I can't find right now. Keep up the good work, I'll be following this project closely.

I'm a native speaker of English, and I speak a few other European languages. I'd like to add my experience. I listened to the recording and in both cases it sounded like gibberish. Hearing the translation did not help and I listened to it 8-9 times trying to get the effect. This result did not surprise me since, I tend to be quite pedantic on pronunciation when hear language being spoken, even languages which are my second or third. I've been known to correct native speakers of lanaguage I am learning. After reading this article I think that I am not very good at using prior knowledge to understanding phrases spoken in any language. If I were, I would not hold onto pronunciation as crutch for comprehension, hence my need for everyone to speak clearly and well for me to follow. It's just a thought.

I remember reading the first chapter on an AI book during college which laid out the objectives of AI research into two schools of thought.The first school of thought believes that AI can be achieved by mimicking real conscious beings. Remember when man first tried to fly, most devices mimicked birds and failed horribly. The other school of thought believes that intelligence has well-defined principles, when discovered, would produce real intelligence (not mimicry), having the effect that the final product may not resemble what we see on day to day basis. Compare planes, which use the principles of flight, to the early flapping machines. A bird and plane both fly but they are very different in the way they do it. Approaching flight from the mimicry angle is hard, it's only recently in the last 10 years we have made light-weight flapping machines that fly well;yet we have had planes for over a hundred years once we knew the principles.

Transcripts would be handy. I doubt a conversation with a 13 year old boy is a good way to measure AI? It's not the best metric to have but it is the most universal and most widely agreed on that we have. It seems like we are happier with small gains in mimicry for now, since real intelligence is hard. Really hard.

Accuracy aside. What do you you if you need to look at your keyboard to find an unfamiliar symbol. I can touch-type fairly fast but my error rate goes up dramatically when typing non-alphabet keys.

What would be neat is if you could bring up the tablet keyboard through a gesture, like clapping your palms together, select the unfamiliar symbol, gesture to remove the keyboard then continue.

Although I like the idea of something like this, I can't imagine myself using it for a long period of time.

Raycasting brings back memories! I remember poring through raycasting techniques to make my Wolf and Doom clones as a teenager. When I was almost done I demoed this for a group of friends at our school's computer club and boy were they were impressed up until the point the clipping algorithm failed and objects stopped disappearing when they fell out of the players field of view. Was teased for months after that. ... .

As a teenager I collected dictionaries for this very reason,it was exciting because each word became more than the sum of its letters but a treasure hunt that would last hours. Halfway through the article I thought he should just use an old edition of Merriam Webster or an Oxford Dictionary that predates the early 80s only to find that he does mention it. My word toolbox now consists of:

* Old Dictionary , Oxford and sometimes Websters

* Roget's Thesaurus

* Online etymology tool

* English Wordnet

* Oxford's Modern English Usage by Fowler ( 2nd or 3rd edition ) - it's a hoot to read.

* Samuel Johnson's Dictionary - if you want a nice bedside chuckle.

There are many ways of slicing and dicing this data set to fit a particular writing style( e.g like Stephens King's no adverb policy,); If the data is ingested in a tree form , searching for word replacements based on Part Of Speech, sound or syllables , mitre is not that hard.

Using these techniques doesn't necessarily make you a better writer but it sure does make it more fun!

Great idea. Obscure landing page.

It's practically impossible to know what this does,unless you click on the [our board] link. Even after scrolling all the way to the bottom of the page where you are asked to part with money.

It would be a shame for such a great idea to be passed by due to lack of clarity.

Agree that job market demand is better a metric to gauge a languages longevity. Which language skills are employers throwing their money at?

Added a few links below for programming language job trends for Java and couple others. The source is itjobswatch.co.uk . Haven't got an equivalent for US yet but Java is doing just fine

Job Demand and Salary Trends:

Java [http://www.itjobswatch.co.uk/jobs/uk/java.do#demand_trend]

C# Sharp [http://www.itjobswatch.co.uk/jobs/uk/csharp.do#demand_trend

python http://www.itjobswatch.co.uk/jobs/uk/python.do#demand_trend]

Been using gitlab for solo dev at work and at home for everything that needs tracking, not only coding. Like you, attempted an install and gave up and used bitnami in the end. The only disadvantage is that the bitnami installers are a few point releases behind.