Hacker News Reader

polyfractalon Nim binary size from 160 KB to 150 Bytes

It's not so much about "never nil", but rather "never accidentally null". Rust's compiler prevents you from moving data into a method which then nulls it out, leaving a dangling pointer in the calling code. At best this dangling pointer will look at garbage and cause a crash or undefined behavior. At worst, it will look at other, actively-used memory and cause a security vulnerability. Rust will just refuse to compile until this problem is fixed.

Static analysis of these lifetimes allow a whole class of errors to be avoided (dangling pointers, double-free, iterator invalidation, etc).

Rust's Options are handy for a lot of stuff (async APIs, concurrent code that may/may not succeed, error codes, etc). But they are just icing really, not really the main thrust of Rust's memory model.

> It's certainly not easier to use and reason about, IMO.

To offer a counter-viewpoint, I find that Option<> (and Result<>) are very easy to reason about. They tell you exactly what to expect from a function, and you don't have to guess if you need to catch exceptions or let them throw higher up. Everything is explicit, the only surprises are panics which are cataclysmic anyhow.

> And it seems just as likely you'll end up crashing your program due to a bounds-check error (which may happen more often since Rust encourages indexing over references due to this very design.. at least, so I've read).

If you use iterators in Rust, you never need to worry about out of bounds errors. In fact they skip range checking altogether, since you are guaranteed that the value you are iterating on won't change under your feet (no iterator invalidation, etc), which generates more efficient code[1]

If you use explicit indexing, then yes, you can have a runtime panic if you go OOB. But that's the nature of explicit indexing. It also has to include those safety checks, so will be slower code.

[1] https://doc.rust-lang.org/book/iterators.html

polyfractalon My 5.5 Months on Soylent: The Joys of Not Cooking

Yes, this. So much. And because I'm a biologist by training and can't resist:

> Bacterias bring us health

Unless it is e. coli, salmonella, listeria, botulism, etc etc. Some bacteria is helpful, most is neutral and some will make you very sick or kill you.

> I think we need the bacterias and all the life on the food

There is just as much bacteria on your skin, on the doorknob and on the basket holding you bananas to service your "daily dose of bacteria". Seriously, bacteria is everywhere. There is good evidence that ultra-clean lifestyles can increase allergies as you mentioned, but that is where you try to sterilize everything. It isn't where you simply cook food.

> Today I'm living on 90% fruits

Please make sure you are getting enough iron, or you could be on the road to anemia and serious health complications. :)

> When I eat dairy or something cooked my poop goes bad on the next day, it's like our body is the best doctor around.

Or, different foods have different molecular structure, which is digested and metabolized by different pathways in your stomach and intestines, leading to different aromatic by-products. Just because you don't like the smell doesn't mean there is something "wrong". I don't like the smell of manure all that much, but it is a very "good" by-product, if you want to label things good or bad.

polyfractalon Slack was hacked

It would remove a lot of nice search features, however. If you just index tokens without positional information, you have a much harder time performing phrase matching. If you include positional information, you can probably crack the encryption because some tokens are statistically more likely to appear next to each other than others.

If you index shingles (phrase chunks) instead, you lose out on sloppy phrases...you can only match exact phrases. I imagine you can perform a similar statistical attack too.

Hell, just getting the term dictionary would probably allow you to reverse engineer the tokens, since written language follows a very predictable power law.

Hashing also removes the ability to highlight search results, which significantly degrades search functionality for an end user.

Basically, yes, you can do search with encrypted tokens...but it will be a very poor search experience.

polyfractalon Postgres full text search is good enough

As other's have mentioned, it's entirely possible to have multiple fields with different analysis chains. Elastisearch handles multi-languages very well. You just have to start changing the defaults, since the defaults assume a single language for all fields (namely English).

E.g. "title_german" and "title_english" can each have their own analyzer specific to the language. Or you could have a single field "title" which then uses multifields to index a "title.english" and "title.german" field.

The key is that at search time you need to use a query that understands analysis. So you should use something like the `match` query, or `multi_match` for multiple fields. These queries will analyze the input text according to the analyzer configured on the underlying field (e.g. english or german)

There is a ton more information in The Definitive Guide book, in the chapter on languages: http://www.elasticsearch.org/guide/en/elasticsearch/guide/cu...

Topics include: language analysis chains, pitfalls of multiple languages per doc (at index vs search time), one language per field schemas, one language per doc schemas, multi-language per field schemas, etc

polyfractalon Elasticsearch – the definitive guide

FWIW, we didn't choose the animal. O'Reilly's design department does a voodoo incantation and choose an animal...the authors/editors have zero input :)

I quite like the snake though, think it looks nice

polyfractalon Elasticsearch – the definitive guide

Unsure if you are talking about "The Definitive Guide" from the original link, or "Exploring Elasticsearch" from the OP.

If you are talking about "The Definitive Guide", it is targeted to Elasticsearch 1.0. Most of the APIs and concepts are backwards compatible, but we wanted to target 1.0 because it brings a lot of great new features.

polyfractalon Goodbye Academia

> Systematic hard work can get you results in academia.

This is not necessarily true. At least in life sciences, there is a big element of pure, dumb luck. Biological systems are inherently noisy, and no matter how diligent we try to be about our processes and protocols, there is always luck.

I spent nearly two years of my life performing a single protocol (an endocytic receptor internallization assay) and the line between "good results" and "wasted a week" was incredibly thin. Some things just require luck to work out, no matter how careful you are.

I left academic biology because I didn't want luck playing into my career. In academic biology you must be incredibly smart, incredibly dedicated, willing to work long hours with little pay, AND be incredibly lucky.

I looked around at the post-docs in my department (at MIT, mind you) and saw brilliant people who would never produce a top-tier article, who had spent so long in their post-doc that they had no chance of ever becoming a professor. They would probably wash out to some industry job at Merck testing cholesterol drugs after wasting 10 years of their life pursuing some fictitious dream.

Truth be told, I wasn't as brilliant as most of the people around me, so I made a judgment call and left. It was the right decision, I'm happier than ever (and actually make money too).

polyfractalon Elasticsearch 1.0.0 released

Not the OP, but GC issues in Elasticsearch basically boil down to memory pressure (obviously), which is usually caused by facets. Facets eat a lot of memory, especially if you are faceting high-cardinality fields - think fields like "tags" or any analyzed field. High cardinality, analyzed strings is the easiest way to blow out the heap.

There are other reasons, but that is like 90% of GC issues. To solve it, you need to make sure your faceted fields are configured well (usually not_analyzed) and assess how much memory is available. You may be able to index and even full-text search ten billion docs on a single machine, but faceting it may just be too much to ask for a single node.

Omiting norms, disabling bloom filters on old indices and enabling doc values are other ways to help alleviate field-data pressure.

Other GC culprits can be: too large bulk requests, unbounded threadpool queues, or something like parent/child/scripts/filter cache keys eating all your memory. Also don't go above 30gb heaps, the JVM becomes unhappy :)

polyfractalon Elasticsearch 1.0.0 released

The pre-Snapshot/restore method is:

- Pause indexing

- Issue a flush request

- Rsync data directories somewhere

- Resume indexing

This is technically a very naive approach, since a simple rsync of the data dirs will include replicas too. If you were more diligent you could check the state files in each shard directory and only copy out the primaries.

polyfractalon Elasticsearch 1.0.0 released

Regarding updates, you can use the Update API for partial updates, and include a script to do things like "counter += 1" or "add value to existing array".

Internally it is still reindexing the entire document, but from your application's perspective, the Update API is a lot friendlier.

http://www.elasticsearch.org/guide/en/elasticsearch/referenc...

polyfractalon Elasticsearch 1.0.0 released

FWIW, this is a place ES devs are spending a lot of time thinking about. For example, 1.0 introduces a new "Circuit Breaker" [1] feature which will help prevent over-eager facets from blowing out the heap. It's just one part of a very large effort to make ES handle exceptional events more gracefully (in particular, memory related).

Another example are disk-based doc values [2], which are essentially pre-computed field data structures that are stored on disk. This moves Field Data off heap and allows the OS to manage memory evictions, to help minimize GCs and OOM blowouts.

[1] http://www.elasticsearch.org/guide/en/elasticsearch/referenc...

[2] http://www.elasticsearch.org/blog/disk-based-field-data-a-k-...

polyfractalon Elasticsearch 1.0.0 released

In addition to the various geo filters/queries, there are also two aggregations for geo related stuff:

Geohash Grid: http://www.elasticsearch.org/guide/en/elasticsearch/referenc...

Geodistance: http://www.elasticsearch.org/guide/en/elasticsearch/referenc...

polyfractalon 37signals Works Remotely [video]

That was an April Fool's joke (posted on April 1st). And even if the date wasn't the first...the whole article reads like an over-the-top satire.

polyfractalon f.lux has been updated to a new version

Haha, what? That is the most bizarre insult? argument? that I've seen in a while. People use greater-than characters all the time for quotations. It's been used in email forever. But bonus points for trying to be edgy I guess.

To your question: All the studies are basically examining the principle that f.lux operates on. Shift colors away from blue and you can limit the decrease in melatonin production. I don't understand your question?

If you mean, "why has no one done a study on f.lux itself?", then I'm assuming no one has done a study on f.lux per se because (I'm just guessing here) no serious academic researcher really gives two shits about a piece of software on the internet.

However, that doesn't mean their research cannot be applied. F.lux is applied science, working on a fairly solid base of academic research. Real science with, yunno, quantifiable and statistically significant results. It isn't "fanboyism" or "science"...it is peer-reviewed literature.

The articles I linked specifically talk about how spectral blue light decreases melatonin production (a hormone directly related to how sleepy you feel at night) and a corresponding increase in alertness.

Doing a little extrapolation, if you decrease the amount of blue light that you are exposed to in the evening you decrease the impact on melatonin production and increased alertness.

So sure, I doubt anyone has stuck a person in front of f.lux and measured salivatory melatonin levels...but they have done the equivalent with blue and red lights. It isn't reaching to say that f.lux is doing exactly the same thing.

polyfractalon f.lux has been updated to a new version

sigh Fine, here are more articles. If you don't want to believe in peer-reviewed academic literature, I'm not sure why I'm bothering. Statistically significant results can be obtained with few participants. Behold the power of math.

http://www.ncbi.nlm.nih.gov/pubmed/23926261

Nocturnal light exposure impairs affective responses in a wavelength-dependent manner.

>Our results demonstrate that exposure to LAN influences behavior and neuronal plasticity and that this effect is likely mediated by ipRGCs. Modern sources of LAN that contain blue wavelengths may be particularly disruptive to the circadian system, potentially contributing to altered mood regulation.

http://www.ncbi.nlm.nih.gov/pubmed/23358248

Out of the lab and into the bathroom: evening short-term exposure to conventional light suppresses melatonin and increases alertness perception.

>Subjective alertness was significantly increased after exposure to three of the lighting conditions which included blue spectral components in their spectra. Evening exposure to conventional lamps in an everyday setting influences melatonin excretion and alertness perception within 30 min.

http://www.ncbi.nlm.nih.gov/pubmed/21298068

Non-visual effects of light on melatonin, alertness and cognitive performance: can blue-enriched light keep us alert?

Exposure to light at 6500K induced greater melatonin suppression, together with enhanced subjective alertness, well-being and visual comfort. With respect to cognitive performance, light at 6500K led to significantly faster reaction times in tasks associated with sustained attention (Psychomotor Vigilance and GO/NOGO Task), but not in tasks associated with executive function (Paced Visual Serial Addition Task). [...] Our findings suggest that the sensitivity of the human alerting and cognitive response to polychromatic light at levels as low as 40 lux, is blue-shifted relative to the three-cone visual photopic system. Thus, the selection of commercially available compact fluorescent lights with different colour temperatures significantly impacts on circadian physiology and cognitive performance at home and in the workplace.

polyfractalon f.lux has been updated to a new version

Result from a quick pubmed search (I haven't actually read the paper):

http://www.ncbi.nlm.nih.gov/pubmed/24110034

>Effects of LED-backlit computer screen and emotional selfregulation on human melatonin production.

>[...] Finally, we can also report that the blue light of LED-backlit computer screen significantly suppress melatonin production (91%) more than red light (78%) and no light (44%).

polyfractalon Rust 0.8 released

I would really love if the reference docs pointed to the underlying code. Many times I'll skim through the reference docs looking for something that I need...then search github for the same section in code so that I can see how it is actually used (or implemented, since the compiler is often the best place to learn Rust itself).

Would be great if the docs just linked straight to the function/crate in github.

polyfractalon Most detailed Github issue ever

Another good example is the Aggregations issue, which details a new faceting framework coming out soonish:

https://github.com/elasticsearch/elasticsearch/issues/3300

polyfractalon FBI Admits It Controlled Tor Servers Behind Mass Malware Attack

Sure, but you can make that argument for virtually any substance. A particular combination of genetics and <insert molecular compound> will equal a bad reaction in some portion of the population. It's practically guaranteed due to the nature of genetics and statistics.

polyfractalon FBI Admits It Controlled Tor Servers Behind Mass Malware Attack

This is ridiculous. Acetaminophen is Tylenol. Are you seriously trying to make the argument that Tylenol is a poison?

Arsenic is a poison. Lead is a poison. Cyanide is a poison. Tylenol becomes toxic at a certain threshold (like every other substance on the earth for that matter).

Sure, it is toxic to your liver in large doses. That isn't why Tylenol 3 is Schedule II while pure Codeine is Schedule II. The difference is in dosage. Tylenol 3 tends to come in doses of around 5mg Codeine to 300-500mg Tylenol. You get T3 when you have a tooth taken out, or maybe some minor stitches. You are also only given a few pills.

You get pure Codeine when you break a bone or stick a pencil through your eye. Or one of the stronger codeine-derivatives (oxycodeine, etc). These come in larger doses and are more prone to abuse.

Hence Schedule III.

I'm no fan of the DEA or the schedule system...but the "Government" isn't poisoning anyone. That's pure FUBAR and true tin-foil-hattery. Its a classification system for the abuse potential of perfectly legal drugs.

polyfractalon Down syndrome reversed in newborn mice with single injection

Drosophila names are truly the best.

Contrast that to C. elegans naming: dpy-5, let-37, unc-11, egl-2. The prefix means something ('unc' is uncoordinated) but otherwise very bland.

polyfractalon Exceptions (2003)

Sure, but the query goes through the layer of functions that actually deals with the database connection. If an error occurs, that layer is responsible and the "higher" layers simply pass on the exception that is thrown.

Point is: the function that deals with database response should never be responsible for dealing with database connection errors.

polyfractalon Exceptions (2003)

Functions should be scoped appropriately so that they only deal with one thing at a time. A function that checks for network connectivity isn't going to be checking for missing rows. Those are different problems, and should be handled by different functions.

Using your example, we have a function that queries a database. It will be given a valid database connection, and return the result of the query.

There is no "valid database connection" logic, since that is handled elsewhere. There is no result validation logic, since that is handled upstream. This function only cares about A) querying and B) returning a value (possibly null). The only exceptions that is handles is when something specific to it's domain goes wrong - for example, unauthorized access to a table. That is an error that is above networking (the connection worked fine) but clearly not a data validation problem (no data), so we handle the exception here.

If you find yourself throwing exceptions "across problem domains", that's a good indicator that your functions are doing too much.

polyfractalon Practical AngularJS Examples

Here is one of my angular projects:

https://github.com/polyfractal/elasticsearch-inquisitor

I'm not sure I would classify this as a "big" app, but it has four tabs and is slowly growing in functionality. The menu bar (and highlighting) is a very simple controller and service.

Obligatory disclaimer: I'm by no means an angular expert.

polyfractalon Ask HN: Who is hiring? (August 2013)

Thanks! It took me a few revisions...it's surprisingly hard to make a clear set of job listings with just ascii text.

On a related note, if you are semi interested in the Web Dev position but think "pfft, wordpress?", we are open to ditching WP entirely and rolling an in-house site/CMS depending on how persuasive you are ;)

polyfractalon Ask HN: Who is hiring? (August 2013)

Elasticsearch is hiring! Most of our positions are location-independent (remote), but a few are for the Los Altos area.

Generic Hiring page: http://elasticsearch.com/about/jobs/

------------------------------------------ About Elasticsearch ------------------------------------------

We build an open source search and real-time analytics engine. Elasticsearch is built on top of Lucene, so it benefits from the amazing capabilities that Lucene brings to information retrieval and general search. Elasticsearch is distributed right out of the box and is designed to scale horizontally - sharding, dynamic cluster resizing, transparent search routing.

The company is almost entirely distributed. I live in Charleston, SC - my colleagues live in Amsterdam, Prague, Barcelona, Bucharest, Nashville, Atlanta and San Francisco. The talent at this company is truly amazing, I feel like a small fish in a very big pond every day. And it's a great feeling, I've learned so much since joining the company.

If you want to get in with a company that is making waves, learn from truly excellent developers about distributed systems, then Elasticsearch is the place for you.

Ok, enough gushing. Here are the positions (I've never made a job ad before, I'm just a dev...if you have questions let me know!):

------------------------------------------ Front-end Developer [Los Altos] ------------------------------------------

Our sites are built in WordPress (yeah, sorry...) and we need someone that is capable of building custom features, managing scalability/performance and can generally mash WP into submission. For example, we have a custom plugin that imports Jekyll docs from Github and converts them to WP pages.

* Strong experience in PHP, WordPress customization, HTML5/CSS3, JS

* Rapid prototyping of our two web properties

* Build landing pages to support marketing campaigns

* Responsible for performance, scalability, latency on client, etc.

Email zachary.tong@elasticsearch.com with a resume or portfolio, I'll send back some more details.

------------------------------------------ UI/Visualization developer [Remote] ------------------------------------------

A problem with "big data" is that it is often very hard to visualize. We are working on ways to make data more manageable, and are looking for someone to take a lead role in developing/creating new visualizations.

* Strong JS developer (framework agnostic, although we do have a lot of Angular and Ember folks here)

* Experience with a visualization framework (ideally D3)

* Worked with large amounts of data, and be capable of transforming this data into a visualization

* Portfolio would be great!

Email jobs@elasticsearch.com

------------------------------------------ Distributed Systems Developer [Remote] ------------------------------------------

Distribution and management of data across a cluster is what defines Elasticsearch. We are looking for someone who will own this part of the core product and work to improve our distributed model. Ideally the candidate will have previously implemented Paxos, RAFT, Zookeeper or some other distributed algorithm.

* Strong Java programming skills

* Experience with distributed systems, node communication and management

* Knowledge, use or development of cluster management algorithms like Paxos, RAFT, and Apache Zookeeper

Email jobs@elasticsearch.com

http://elasticsearch.com/job/distributed-systems-software-en...

------------------------------------------ .Net Developer [Remote] ------------------------------------------

We have programmers working in many different languages (Python, Ruby, PHP, etc) but are lacking .Net. We are looking for a dev to take the lead on all things .Net

* Innovate and lead our .NET development efforts and contribute to technical architecture, design, code, and test areas that you own.

* Collaborate with other development teams, quality engineering team, and documentation team to execute on product deliverables.

* Evangelism of our product to the .NET community at large.

Email jobs@elasticsearch.com

http://elasticsearch.com/job/net-software-engineer-2/

------------------------------------------ Java Software Developer [Remote] ------------------------------------------

Catchall job category. Are you a great Java programmer? Have you worked with Elasticsearch before? You should apply! We are always looking for more core developers.

Email jobs@elasticsearch.com

http://elasticsearch.com/job/software-developer/

------------------------------------------ Support Manager [Remote] ------------------------------------------

* Insuring customer issues are resolved within our committed service level agreements.

* Drive market-leading customer satisfaction rates, with proactive engagement of unsatisfied customers to ensure issues resolved.

* Build out the internal team, tools, and processes to optimize an efficient and effective support structure. * Develop real-time metrics, which track performance of the support organization.

* Maintain strong relationships with our technical team for the delivery of support, and sales team in the selling our support products.

Email jobs@elasticsearch.com

http://elasticsearch.com/job/support-manager/

polyfractalon Standard Treasury (YC S13) Aims To Make Software That Eats Commercial Banking

Is a) not common?

I use ING Direct (recently bought by Capital One and recoined "Capital One 360"). You can either wire directly from one checking account to another using an ACH transfer. Or you can do a Person2Person payment which is similar to a paypal transfer. Enter your recipient's email address, ING sends them an email, they follow the directions and "pick up" the money, transfer occurs.

Both are free to the best of my knowledge.

Edit: ING is a Dutch company, so maybe that's why.

polyfractalon PHP 5.5

To be fair, that is an optional extension and not bundled into PHP by default.

polyfractalon Tesla Clashes With Car Dealers

NPR Planet Money has a great episode on the history of car dealerships: http://www.npr.org/blogs/money/2013/02/12/171814201/episode-...

Basically, everyone in this thread is right.

- Local dealerships were originally founded because in the 1920s, cars were big/expensive and hard to move around the country. Cars needed a lot of repairs, so local dealers to service them made sense.

- During the Great Depression, car manufacturers continued to make cars despite no one buying them. So they forced dealerships to purchase cars, or else get banned from selling their brand ever again (effectively terminating the business, obviously bad if you have loans and other capital investments). This was before overseas competition from Japanese/Korean manufacturers existed, so losing your contract to Ford meant you were pretty sunk.

- Dealerships fought back by getting legislation passed to protect them legally, slowly granting more power to the dealerships over time. Now the balance of power is far, far on the side of dealers - manufacturers cannot shut down dealerships, cannot control who runs them, must provide cars to dealers even if they are in poorly profitable areas, cannot tell the dealers what to sell, no real way to incentive dealers to not be "car salesmen" assholes, etc etc.

polyfractalon Edward Snowden Q&A

> [...] if I were a Chinese spy, why wouldn't I have flown directly into Beijing? I could be living in a palace petting a phoenix by now.

If I didn't like this guy before...I certainly do now.