cortesi

Nearly all of them, depending on what exactly you're looking for. Simple things like being able to exclude results from forked repos can save a huge amount of time, and being able to limit results by language, creation date and even number of stars (to find personal repos) has come in useful.

Github search does many, many things you can't trivially recreate through a Google search:

https://github.com/search/advanced

You can access all of this functionality with ghrabber.

One of my suggestions to Github is that they disable indexing of dotfiles of all persuasions (including contents of dot-directories), unless the repo owner explicitly opts in. That would make it much harder to find a very large fraction of the more obvious leaks.

I published a tiny script that makes mass grabbing of files from Github easy (https://github.com/cortesi/ghrabber), and wrote about some of the interesting things one could find with it. For example, there are hundreds of complete browser profiles on Github, including cookies, browsing history, etc:

http://corte.si/posts/hacks/github-browserstate

I've also written about some less security critical things, like shell history (http://corte.si/posts/hacks/github-shhistory) custom aspell dictionaries (http://corte.si/posts/hacks/github-spellingdicts), and seeing if one could come up with ideas for command-line tools by looking at common pipe chains from shell histories (http://corte.si/posts/hacks/github-pipechains).

I've held back on some of the more damaging leaks that are easy to exploit en-masse with a tool like this (some are discussed in the linked post, but there are many more), because there's just no way to counteract this effectively without co-operation from Github. I've reported this to Github with concrete suggestions for improving things, but have never received a response.

Intelligence is the wrong word in this case. These are complicated emergent behaviours that arise from extremely simple rules applied locally and iteratively by each individual. There's no planning, forethought, or central driving will at play here.

If this is intelligence, then so is bone growth, which follows a similarly simple algorithm, is also honed by evolution, and is also inspiring real-world applications, e.g.:

http://www.engineering.com/DesignSoftware/DesignSoftwareArti...

https://www.fastcodesign.com/1671102/a-3-d-printed-house-tha...

The world of social insects is filled with ideas for tackling resource tradeoffs of this sort. These are usually not signs of "collective intelligence", as much as they are a manifestation of good algorithm design, honed by aeons of evolution. In fact, we've successfully pinched and applied quite a few social insect resource tradeoff algorithms in computer science.

It's a marvel that we can actually understand these biological algorithms well enough to code them up. I wrote about an example of this in a blog post about EO Wilson's "Superorganisms" a while ago (http://corte.si/posts/books/superorganism/index.html). Bee hives need to find the right balance between foragers out in the field, and nectar producers staying home to work in the hive. They have an elegant way of doing this, that works even when each bee can only rely on local knowledge - that is, each bee makes a decision about what it can see around itself, without any global integrative intelligence required. The algorithm goes like this (see the blog post for more detail):

1 - Not enough nectar collectors in the field? If yes, and you also have immediate knowledge of a producing flower patch, perform the waggle dance.

2 - Is the flower patch rich or the weather fine or the day early or does the colony need substantially more food? Perform the dance with appropriately greater vivacity and persistence.

3 - Not enough active foragers to send into the field? Perform the shaking maneuver.

4 - Not enough nectar processors in the hive to handle the nectar inflow? Perform the tremble dance.

A variant of this algorithm, directly inspired by bees, has been used to balance resource allocation in server farms:

http://www2.isye.gatech.edu/~ctovey/publications/papers/bee....

No problem! Hope it was useful.

It's worth thinking carefully about what you expect to gain from Open Sourcing code, before you do it. One of my projects (https://mitmproxy.org) is pretty widely used commercially (both as a tool for analysts, and as a back-end component). We have a vibrant pool of contributors, many of whom use mitmproxy in their day jobs, but the only financial contribution from a commercial user has been from Google, and that at arm's length via a GSOC sponsorship.

I would never complain about this, though - we don't solicit financial contributions, and I didn't start the project expecting to benefit financially. I get a lot of direct enjoyment out of working on mitmproxy, and I see it as a way to share something neat with like-minded people. In that sense, it has been immensely "profitable": I've met and collaborated with many interesting folks because of mitmproxy.

If you don't feel these indirect benefits make a project worth your while, you might want to consider doing something else with your time. There's no likely future in which small open source projects will be deluged with either contributors or money.

Devd doesn't appear in my gulpfiles at all, actually. Gulp literally just renders to disk. I then start devd separately with the appropriate livereload flags to handle the service end of things.

For instance, here's a devd command-line from the docs (https://github.com/cortesi/devd) that handles proxying local services, with a static directory overlaid. Gulp would be rendering assets into that static directory, the change would be noticed by devd, and liverload would be triggered.

  devd -l \
  -w ./src/ \
  /=http://localhost:8888 \
  /api/=http://localhost:8889 \
  /static/=./assets

Slinky looks very neat! I do feel it inhabits a very different niche from devd, though. I have no ambitions for devd to compete with tools like Grunt, Gulp or Slinky.

Devd has a simple job: it just serves files and triggers livereload when files change, and is completely agnostic about the build process. That means it works very well as a complement to Gulp - in fact, this is one of my primary use cases for devd. My gulp setups are simpler now: I've ditched Connect and node livereload libraries, and instead gulp JUST renders files to disk. It's early days, but having two sharp tools each doing what they do best seems to work well.

I should add that I definitely don't think there's one "right way" to do this stuff. I love the fact that we're free to explore variations to see what works and what doesn't. The more things we try the merrier!

Right - these are the frequency dot plots I first saw in Greg Conti's work (some examples here: http://sebug.net/paper/Meeting-Documents/BlackHat-USA2008/BH...). They're definitely on the way.

Regarding other space-filling curves - I'm in two minds about this. I've implemented z-order and the H-curve for binvis, but for different reasons neither of these are as useful as the Hilbert curve, so they're not exposed in the interface. My feeling now is that I should pick one locality preserving layout and one intuitive layout to keep things simple and usable. I want binvis to be a workaday tool, so I'm keeping my space-filling curve geekery to another project (https://github.com/cortesi/scurve).

Frequency analysis and byte dot plots are definitely on the way. Do you have a concrete example of the digraph frequency feature you can link me to? Any other feature suggestions are welcome too.

Author of the post here. I'm working on a browser-based interactive analysis tool based on these visualizations. It's nowhere near ready for release, but those who are interested can find a demo here:

http://binvis.io

This is not even an alpha yet - I plan to announce the initial release in the next month or so. Eventually, I would like to add the ability to upload, share and annotate files for analysis. Comments and criticisms welcome!

Thanks for the heads up. We'll track that down.

We're looking at the sound set bug right now. Also, see my other comment about the intrusive sounds. We plan to have some demo streams tuned for real-world use (i.e. not as alarming) soon.

Apologies! The demo is tuned to be way more intrusive than a real-world data stream would be. Our theory is that if we have people's attention for 15 seconds, we should show them the full range of sound intrusiveness. Hope it wasn't too startling.

There are so many possibilities. A few weeks from now, we should have the ability for users to create, edit and share their own soundpacks.

I'm somewhat late to the party, but I spent some time playing with procedural colour picking using space-filling curves (the Hilbert curve in particular). I describe the basic technique here:

http://corte.si/posts/code/hilbert/portrait/index.html

And you can see what this looks like when used to visualize sorting algorithms at large scale here:

http://corte.si/posts/code/sortvis-fruitsalad/index.html

And here:

http://sortvis.org/

I've also published an Open Source project that might be useful to anyone who wants to play with these ideas:

https://github.com/cortesi/scurve

There are some limitations to the process as described - most obviously the fact that I'm using the RGB color space which is not the best match for human color perception. RGB is convenient because it is a regular cube and therefore easy to map onto the Hilbert curve, but with a little work the process could be generalized.

I certainly don't want to give the impression that we're unaware of the long history of auditory display projects. In fact, it's my reading in these areas that resulted in this project. Keep an eye on my blog for some posts delving into our antecedents and the research around sound perception soon.

That said, what we're trying to do specifically - which is sonification as a service, and trying to adequately cover a very wide range of different use cases and sound sources at once - is probably new. I don't think that matters much, though, and "newness" is the least interesting aspect of the project.

When we began, the idea was to synthesise everything, and have many perceptual variables to play with. Then we realised that we could validate the idea using pre-rendered sound snippets. We'll turn back to something more sophisticated once we start adding interfaces for levels and continuous measurement, and I'm sure that will wash through to more flexibility in discrete sounds as well.

Have a look at my co-conspirator's blog post about Choir:

http://alexdong.com/choir-dot-io-explained/

We definitely see Choir fitting in where you can't look at or interact with a screen. Cars and wearable computing are areas we're excited about. First, though, we want to experiment on the desktop, find out what makes a good audio interface, and solve our own burning needs regarding more mundane monitoring situations.

It's not just you - the GitHub demo is a tad annoying. The idea was that we might have people's attention for 15 seconds on average, and we wanted them to get a feel for the full spectrum of sounds in that time. So, it's tuned to be much more intrusive than a production stream might be.

Also, whether any particular sound is annoying or pleasant is complicated (we're just figuring out the parameters now) and subjective. So, we're working on letting users create, edit and share sound packs to see what smarter and more talented folks than us come up with.

Yes, absolutely. My initial idea was to synthesise all the audio ourselves, using a native desktop client and libpd. Then I realised that we could test 95% of the idea using web audio and pre-compiled sound snippets. And here we are.

Continuous data is one of the very next things we are implementing, partly because a sufficiently dense discrete set of events becomes a frequency, partly to cater for measurements like load. We plan to indicate magnitude with pitch and volume, but there are some complexities in the API and representation that we're working through first.

I've added Mixpanel to our list of proposed integrations. We'll get round to all of them in time. That feeling of being in touch with your applications and servers that you experienced at Dropbox is _exactly_ what we're aiming at. I'll pass on the hipsters with boomboxes, though. ;)

We're collecting integration ideas over here:

http://choir.uservoice.com/forums/217059-general/category/70...

If there's anything else on your wishlist, just chuck it in tnere.

Fix will be inbound shortly. We've had a bit of an unexpected bump in traffic today, and we're waiting for our moment in the sun to fade before we revamp things.

Thanks, ordered. There's a substantial body of research in this area, and we're just becoming conversant with it now.

Right, this is exactly the kind of thing we're thinking about now. We already have integration with a bunch of services (GitHub, BitBucket, etc. etc.). At the moment, you still have to pick sounds for these on signup, but very soon we'll have flexible, sane defaults. Once this happens we will have onboarding in a few minutes for all of our pre-rolled integrations.

Please drop your email into the invites queue. We're working hard on Choir and will have a blog with new features up and running soon. When that happens, we'll drop a line to everyone in the queue.

All of this will be possible soon. We're working on the ability for users to completely customize the sound palette.

We're still thinking about which components will be opened, but we're primarily thinking of this as a service. Our pricing model will probably be based on number of concurrent listeners.

So, an API for the player aspect of Choir is not in the works at the moment. We've worked hard on the other end, though, making it as easy as possible to feed data into Choir and get sound out.