Hacker News Reader

drothlison Google agrees to pay £130m in back taxes after an audit by UK tax authorities

Second paragraph of the bbc article.

There's a really gorgeous technique for implementing good error messages with LR parsers: http://lambda-the-ultimate.org/node/4781#comment-76724

(I haven't used the above personally, but the Golang parser does.)

Personally I have used PEG parsers and I was able to provide great error messages, by using the "cut" operator in the appropriate places in the grammar (see comments earlier in the thread I linked above).

drothlison Ask HN: How are some people exceptionally productive?

I like this quote attributed to John Carmack:

"Focused, hard work is the real key to success. Keep your eyes on the goal, and just keep taking the next step towards completing it. If you aren't sure which way to do something, do it both ways and see which works better."

See also this blog post by someone who worked with Carmack: http://bookofhook.blogspot.co.uk/2013/03/smart-guy-productiv...

drothlison How to make Selenium tests reliable, scalable, and maintainable

> This led to chronic misuse around the code (I found 4-5 tests that simply forgot to wrap it in an assert effectively rendering the test completely worthless).

Yes, I've been bitten by that too -- it's too easy to forget the "assert". This morning it occurred to me that I could write a pylint (static analysis) checker to catch that, so I've done just that: https://github.com/stb-tester/stb-tester/commit/5e5bdbb

drothlison How to make Selenium tests reliable, scalable, and maintainable

  > getWithRetry takes a function with a return value
  > 
  >   def numberOfChildren(implicit user: LucidUser): Int = {
  >    getWithRetry() {
  >      user.driver.getCssElement(visibleCss).children.size
  >    }
  >   }
  > 
  > predicateWithRetry takes function that returns a boolean and will retry on any false values
  > 
  >   def onPage(implicit user: LucidUser): Boolean = {
  >    predicateWithRetry() {
  >      user.driver.getCurrentUrl.contains(pageUrl)
  >    }
  >   }

At first I didn't get the difference between `getWithRetry` and `predicateWithRetry`, but then I noticed that the former throws an exception whereas the latter returns false. I infer that `getWithRetry` will handle exceptions thrown by the retried function.

In stb-tester[1] (a UI tool/framework targeted more at consumer electronics devices where the only access you have to the system-under-test is an HDMI output) after a few years we've settled on a `wait_until` function, which waits until the retried function returns a "truthy" value. `wait_until` returns whatever the retried function returns:

  def miniguide_is_up():
      return match("miniguide.png")

  press(Key.INFO)
  assert wait_until(miniguide_is_up)
  # or:
  if wait_until(miniguide_is_up): ...

(This is Python code.)

Since we use `assert` instead of throwing exceptions in our retried function, `wait_until` seems to fill both the roles of `getWithRetry` and `predicateWithRetry`. I suppose that you've chosen to go with 2 separate functions because so many of the APIs provided by Selenium throw exceptions instead of returning true/false.

  > doWithRetry takes a function with no return type
  >
  >   def clickFillColorWell(implicit user: LucidUser) {
  >    doWithRetry() {
  >      user.clickElementByCss("#fill-colorwell-color-well-wrapper")
  >    }

Unlike Selenium, when testing the UI of an external device we have no way of noticing whether an action failed, other than by checking the device's video output. For example we have `press` to send an infrared signal ("press a button on the remote control"), but that will never throw unless you've forgotten to plug in your infrared emitter. I haven't come up with a really natural way of specifying the retry of actions. We have `press_until_match`, but that's not very general. The best I have come up with is `do_until`, which takes two functions: The action to do, and the predicate to say whether the action succeeded.

  do_until(
      lambda: press(Key.INFO),
      miniguide_is_up)

It's not ideal, given the limitations around Python's lambdas (anonymous functions). Using Python's normal looping constructs is also not ideal:

  # Could get into an infinite loop if the system-under-test fails
  while not miniguide_is_up():
      press(Key.INFO)

  # This is very verbose, and it uses an obscure Python feature: `for...else`[2]
  for _ in range(10):
      press(Key.INFO)
      if miniguide_is_up():
          break
  else:
      assert False, "Miniguide didn't appear after pressing INFO 10 times"

Thanks for the article, I enjoyed it and it has reminded me to write up more of my experiences with UI testing. I take it that the article's sample code is Scala? I like its syntax for anonymous functions.

[1] http://stb-tester.com [2] https://docs.python.org/2/reference/compound_stmts.html#the-...

drothlison How to make Selenium tests reliable, scalable, and maintainable

According to that documentation you linked, it just polls until `default_max_wait_time` (which defaults to 2 seconds).

drothlison How to make Selenium tests reliable, scalable, and maintainable

I'm not sure this satisfies your parent poster's requirement of: "if it fails, you know about it pretty soon, without the need for long timeouts."

drothlison How to make Selenium tests reliable, scalable, and maintainable

If using Selenium's Python bindings, you can take a screenshot from Selenium and convert it to OpenCV format like this:

    cv2.imdecode(
        numpy.asarray(
            bytearray(base64.decodestring(driver.get_screenshot_as_base64())),
            dtype=numpy.uint8),
        cv2.CV_LOAD_IMAGE_UNCHANGED)

(where `driver` is your WebDriver object, e.g. `WebDriver.Chrome()`).

Then to match that frame against a previously-captured "template" image, you can use stb-tester's[1] "match" function[2] which allows you to specify things like the region to ignore and tweak the matching sensitivity.

[1] http://stb-tester.com [2] http://stb-tester.com/stb-tester-one/rev2015.1/python-api#st...

drothlison Reasons for Python async/await

A major motivation for the new syntax is to support refactoring (currently moving "yield" into a sub-function makes the calling function no longer a generator).

See the PEP's "rationale and goals"[1] and this article[2] summarising the PEP.

[1] https://www.python.org/dev/peps/pep-0492/#rationale-and-goal... [2] https://lwn.net/Articles/643786/

drothlison ErgoDox EZ: An incredible mechanical keyboard

This is a funding campaign to mass-produce a fully assembled version of the ErgoDox open-source hardware design.

drothlison GNU make insanity: finding the value of the -j parameter

The "bash-completion" tab completion scripts for "make" do just that (run "make -npq") so presumably it's an acceptable thing to do for interactive use, that works reasonably well for some (most?) makefiles.

"At distribution scale"... judging by your username, I take it your use case is something like finding out if a source tarball provides the standard targets like "install" and "distcheck", for all packages in debian? I can see how that would be rather painful.

I wonder if there's room for adding a "--dry-run-yes-really-dont-run-any-recipes-at-all" flag to make. As far as I remember the main use case for the current behaviour is (a) to run recipes that create makefiles that are then included into the main makefile, used in build systems that automatically figure out header dependencies and things like that; and (b) process makefiles in subdirectories for "recursive make" build systems. I wonder if you don't do those steps, would you actually miss out on any significant information re. top-level targets? I imagine that use case (a) is used to add prerequisites to existing targets, rather than creating new targets.

Or maybe a more general solution, instead of "--dry-run-yes-really", would be providing access to make's parser as a library, instead of having to parse the output of "make -p". I'm sure IDEs could make good use of that. Who's got time to implement that though.

drothlison GNU make insanity: finding the value of the -j parameter

Buildsome [1] has make-like syntax and some interesting ideas, though I haven't personally tried it.

[1] https://github.com/ElastiLotem/buildsome

drothlison Bazel – Correct, reproducible, fast builds for everyone

I gather that it runs builds inside a chroot where the only available files are the dependencies you specified explicitly (including the compiler[1]), at least in "strict" mode[2]. Or else it must monitor what files are opened during the build step and fails the build if it saw an unexpected file being opened.

It never explains any of this explicitly, but there are hints. [1], [2], [3].

[1] "Many rules also have additional attributes for rule-specific kinds of dependency, e.g. 'compiler'" -- http://bazel.io/docs/build-ref.html#types_of_dependencies

[2] http://bazel.io/docs/build-encyclopedia.html#cc_binary.hdrs_...

[3] "The build system runs tests in an isolated directory where only files listed as 'data' are available" -- http://bazel.io/docs/build-ref.html#data

Edit: A comment below seems to suggest that this is not the case: "Within Google we use a form of sandboxing to enforce that" (emphasis mine). -- https://news.ycombinator.com/item?id=9259147

drothlison Docker containers on the desktop

"Docker does not support user namespaces"[1] so root inside a container == root on the host. Getting security right with user namespaces is hard though[2].

And as another commenter pointed out, you can't give a user permission to actually start a docker container without also giving them root access to the host.

[1] https://docs.docker.com/articles/security/ [2] http://lwn.net/Articles/626665/

drothlison The State of Go

> Source "annotations" are lost after a force-push (why not keep around a read-only view of old comments? We have lost some valuable discussions on GH pull requests)

Yes, that is incredibly annoying. I discovered that if you add your comments in the "Files changed" tab (which shows the diff of the entire pull request) instead of the "Commits" tab (which shows the diff commit-by-commit), then the comments aren't lost when you force-push.

Just FYI, might make life a bit easier if you're stuck with Github.

drothlison Moved ~/.local/share/steam. Ran steam. It deleted everything owned by user

David A. Wheeler recommends setting IFS=$'\n\t' to handle spaces in filenames & variables.

http://www.dwheeler.com/essays/filenames-in-shell.html

drothlison Is Docker ready for production?

> If I want to keep my containers separated, I can’t have them communicate with a UNIX socket, unless I create a shared volume. Once again it’s a no go for me.

What do you mean by "shared volume" -- docker's "-v" to bind-mount the socket into the container? How else would you like to expose the socket to the container? Why is it a no go?

drothlison Emacs Rocks

Possibly not much. I've never used XCode for C++ development but I imagine it has pretty nifty Clang integration, given Apple's investment in Clang. Anyone who suggests using ctags, GNU Global, gtags, CEDET's own parser (implemented in elisp!), etc. for C++ navigation is basically saying "use a C parser to parse C++ code", which works well enough if you're writing 1990s style C++, I suppose. There are projects out there to integrate with Clang (such as rtags mentioned by another poster) but it will require hours/days of setup & frustration, and the result won't be as good as a dedicated C++ ide.

BTW I say this as an emacs fan and the author of http://david.rothlis.net/emacs/howtolearn.html and https://github.com/drothlis/clang-ctags/

Where emacs shines (for me, at least) is when you spend a lot of time in different languages. Maybe you spend 50% of the time in C++, but the other 50% in shell, makefiles, python, etc. -- and the benefit of better C++ code navigation begins to seem like a micro-optimisation that isn't worth the tradeoff compared to Emacs's support for navigation, version control, compilation, linting, grepping, etc etc, all of which are consistent across all languages. Not to mention Emacs's great support for text manipulation in general.

Also Emacs seems to me the best bet for the "one true editor" you'll still be using in 20 years (by which time hopefully few of us will still be using C++).

I'll also point out that on OS X, Emacs has a lower learning curve than any other OS, at least as far as the basic keystrokes: OS X's Command key still works for the usual shortcuts (cut, copy, paste, save, open, quit, undo) while the quirky emacs bindings are on the Control and Option keys. This doesn't make it any easier to set up a C++ navigation solution, of course.

drothlison Please remove mitsuhiko/*

drothlison Ask HN: How do you maintain a high velocity in productivity?

What you do when waiting for some computer operation to execute can determine if you are a zen coder, or distraction junkie coder. The main idea here is to: Not lose focus, and Take a micro break.

[...]

-- http://www.componentowl.com/blog/zen-coder-vs-distraction-ju...

drothlison Sikuli Script

I do a similar style of UI test automation for set-top boxes / smart TVs, with stb-tester[1].

We've found that the "Got kicked off for every build" continuous integration process you mention is the crucial part to achieving success with this type of test automation -- if you're going to invest the effort in writing reliable tests, you want to be getting value out of them by running them as often as possible and as early as possible.

[1] http://stb-tester.com/

drothlison Sikuli Script

OCR is never perfect. I do a lot of automated UI testing in a way similar to Sikuli, and while we do rely on OCR a lot, you have to use certain workarounds (like fuzzy matching instead of looking for a perfect match of your expected text).

Ultimately Tesseract was primarily designed to operate on text which had been printed and then scanned, whereas the text on screen is lower resolution, anti-aliased, on a coloured background, etc etc.

Some further details of our OCR investigations here: http://stb-tester.com/blog/2014/04/14/improving-ocr-accuracy...

The TLDR version is: Training Tesseract on your font doesn't help; scaling up the text 3x before passing it to tesseract gives a massive improvement (I don't know if Sikuli does this); normalising ligatures & punctuation gives an additional slight improvement.

drothlison The Definitive Guide to Syntax Highlighting

The article doesn't mention highlighting the declarations/definitions of types & variables. I find it far less obtrusive, and more useful, than lexical highlighting. It's surprisingly easy to do in Emacs, and it works reasonably well across most language modes. Example screenshots: http://david.rothlis.net/code_presentation/distracting_synta...

drothlison Testing Emacs Packages: surprisingly non-awful

If you have 30 minutes: Steve Yegge's "Emergency Elisp". http://steve-yegge.blogspot.co.uk/2008/01/emergency-elisp.ht...

If you have 6 months: Structure and Interpretation of Computer Programs (highly recommended, I enjoyed it a great deal). http://mitpress.mit.edu/sicp/

Emacs-specific tutorial on using the built-in help and the elisp packages' source code when you need to figure out how to do a particular customisation: http://david.rothlis.net/emacs/customize_c.html

drothlison Atom Is Now Open Source

I found this concept quite neat: An "always on" debugger (for javascript) http://lambda-the-ultimate.org/node/4945

drothlison The cost of Linux's page fault handling

"Just". The "mostly" in pymake's tagline (below) is the tricky bit.

pymake: A mostly GNU-compatible python implementation of `make`.

drothlison Ask HN: Your favorite practical, underutilized programming tip?

    cat > asdf.txt <<EOF
    a b
    c d e
    f g h i j
    EOF
    cat asdf.txt | awk '{print $1}'
    cat asdf.txt | awk '{print $2}'
    cat asdf.txt | awk '{print $(3-1)}'
    cat asdf.txt | awk '{print NF}'
    cat asdf.txt | awk '{print $NF}'

drothlison A Motion-Sensing Keyboard

With the old fingerworks touch-pad keyboards[1] single-finger taps were for pressing keys, two-finger movements moved the mouse cursor and two-finger taps were for clicks. It worked really well. The main annoyance was the lack of feedback when typing, a problem which this Microsoft version solves! If the gesture & tap recognition is sufficiently good. Fingerworks went on to be acquired by Apple and its technology was incorporated into the first iPhone.

[1] https://www.google.co.uk/search?q=fingerworks&tbm=isch

drothlison Pyringe: Debugger capable of attaching to and injecting into Python processes

With gdb you have "py-bt" etc: https://docs.python.org/devguide/gdb.html

drothlison Ask and Show HN: Startups, do you have your integration tests?

The difference between "X Testcases" and "Y Free Writes" per month is a bit confusing, but I think I understand what you mean.

My main concern would be that managing test failures (is it a real bug, or is the test just fragile to minor app changes) would be just as much work as having the tests in-house. Probably more work, since I can't just look at the code. Your reporting/logs would have to be excellent and very detailed.