Hacker News Reader

karterkon Typesense: Open-Source Alternative to Algolia

We support English and other European languages (supports fuzzy search normalizing accented chars).

While it does not support stemming, with fuzzy prefix matching, it will largely work and practically more useful.

No typo or fuzzy correction for Chinese text yet.

karterkon Typesense: Open-Source Alternative to Algolia

Typesense's primary focus is speed and developer convenience. It makes an assumption (which is true for perhaps 99% of the time) that memory is cheap enough for indexing most datasets. Especially given the effort of development time and the benefits from a solid search user experience.

Other libraries like Elastic offer more customization but also has a steeper learning curve.

karterkon Typesense: Open-Source Alternative to Algolia

Yes, for typo correction + instant search, Lucene definitely is not fast enough on large datasets. There are also some limitations with fuzzy searching when you also want to sort/rank documents at the same time. Lucene is also a very generic mature library for a wider set of usecases.

karterkon SEO lessons learned by writing a blog post every day for two years

This essentially comes down to quantity vs quality. Keyword research, link building etc. helps you optimize for your time. You can achieve the same traffic from fewer posts. Otherwise you have to keep churning out posts and make up for it in terms of long tail traffic.

karterkon Open Source, SaaS and Monetization

This is a topic that I have been thinking a lot on recently. I've been working on a typo-tolerant instant search engine for a couple of years now called Typesense (https://github.com/typesense/typesense).

It has not been wildly popular (mostly because I've not really started marketing it) but it's loved by those who use it.

While exploring how to support its development, there were only 2 options: either to offer a hosted version or premium features which will take some features away from the open source version (essentially open core).

A hosted version is essentially ruled out since it's a lot of work and will eventually be bettered by AWS :) Open core will mean crippling the main product in some ways.

Tricky choice. Eventually I ended up with a middle-ground. Offer a premium version but price it so reasonable (in my case 500 USD / year) that it becomes a no-brainer decision. The only downside to this approach is of course you can't probably run a company off it but it certainly can support 2-3 people comfortably IF it succeeds.

karterkon Bayard: a full-text search and indexing server written in Rust

Elasticseach is notoriously hard to roll out and develop against (for smaller companies especially), and so I am happy to see smaller projects in this space.

I've also been working on a light, fast, typo-tolerant search engine: https://github.com/typesense/typesense

It's been around for a couple of years now, and have a few happy customers who have had great success in replacing $X0,000/year popular hosted search with Typesense!

karterkon In the Pancreas, Common Fungi May Drive Cancer

Back when I was in college, someone gifted me Randy Pausch's "The Last Lecture". It was my first introduction to this terrible disease. Little did I know then, that a decade later, I would lose my beloved father in-law to the same disease.

When he was diagnosed, I digged into Randy's old blog, and realized how little things have changed since then. There is now a new cocktail of Chemotherapy drugs, but apart from that we have hardly progressed on this front. I don't think we could have really saved him even with an early diagnosis.

Surgery seems to be the only permanent, generic cure as far as most cancers are concerned. But with Pancreas, from what I read, even that gave a very grim future (as in Randy's case). If at all you survive the Whipples, you will be entirely on expensive enzyme supplements for the rest of your life AND carry a significant risk of metastasis.

karterkon Ask HN: Anyone ever come back _to_ strongly typed languages?

raises hands

Most of my early carrier involved PHP, JavaScript and Python. I started appreciating type systems only when I began working on larger code bases. Types might slow you down initially but I personally find it very comforting to have that extra safety once the codebase crosses a certain size.

Most of the bad rep that statically typed languages get are from a bygone era that involved a lot of typing. That has changed a lot now. With better IDEs and languages with implicit type inference, it's a lot less verbose these days.

I have been writing a search engine in C++ (https://github.com/typesense/typesense/) and even C++ with the auto keyword and Clion IDE is no longer as verbose. And type safety has saved so many potential bugs.

However, for web development though, there are far less options. I hope more dynamically typed languages allow optional types in future. That's where the world is heading towards.

karterkon Darpa wants help cracking the election security problem

India has been conducting elections via electronic voting machines for a while now: https://en.wikipedia.org/wiki/Electronic_voting_in_India

In India's case, the paper ballots actually caused a lot of rigging as booth capturing was ramptant. EVMs greatly helped tackle that.

A small sample of the votes are also verified using voter-verified paper audit trail (VVPAT).

karterkon Ask HN: Are Lucene/Solr/ES Still Used for Search?

This was exactly my pain point as well. For smaller projects, ES is a overkill. So I decided to do something about it! I started working on an open source, really, developer friendly search engine that just works. It's pretty stable now and quite a few people use it and like it: https://github.com/typesense/typesense

Would love to hear your feedback.

karterkon Ask HN: Are Lucene/Solr/ES Still Used for Search?

Yes! Please take a look at an open source search engine I am working on. You will definitely like it:

https://github.com/typesense/typesense

Would love to hear your feedback :)

karterkon Ask HN: Are Lucene/Solr/ES Still Used for Search?

I have been working on an open source alternative: https://github.com/typesense/typesense

Would love to hear your feedback :)

karterkon Cedarwood: Efficiently-Updatable Double Array Trie in Rust

The trie is my favorite data structure. It allows you to build inverted indices (i.e. hashmaps) that can also be used for prefix lookups of tokens and for spelling/typo correction.

With a trie, it's very efficient to calculate Levenshtein distances. Here's a post (http://stevehanov.ca/blog/?id=114) that has a good primer on that.

If you love this sort of thing, here's a fast and neat memory efficient Trie variant: https://db.in.tum.de/~leis/papers/ART.pdf (Adaptive Radix Tree).

I've used ART as a building block to create an open source typo tolerant search engine called Typesense (https://github.com/typesense/typesense).

karterkon Ask HN: What are you working on?

5 years ago, I had a crazy idea that there must be an open source engine that had a really simple learning curve and worked out of the box.

The result: https://github.com/typesense/typesense

It took me over 3 years to get the nuts and bolts right. It's now production ready and being used in quite a few production environments. Of course, there is more work to do and so it's still actively being worked upon.

karterkon Show HN: Fist – Persistent Full Text Search and Indexing Server Written in C

I was merely highlighting the convenience of a HTTP based RESTish API. It will be a familiar pattern instead of a custom text based protocol over tcp.

karterkon Show HN: Fist – Persistent Full Text Search and Indexing Server Written in C

Looks like the project is still under development but I think that having a http based interface would make this much more accessible to be used easily from multiple languages.

Disclosure: I work on a similar open source search engine (https://github.com/typesense/typesense)

karterkon Ask HN: Old guys, what do you miss about development in the 70s, 80s, 90s, 00s?

There were very few ready-made frameworks and libraries to get things done, so it was fun writing those meta libraries that runs your business. Of course, purely from a business point of view, this is not ideal and is extremely wasteful. At the same time, it really helped me understand the systems that we use. Maybe that's why people survived without Stackoverflow :)

In comparison, modern software development is almost all about composing or stitching together a bunch of different services. There are other complexities around orchestration and breadth of the systems involved, but very rarely one goes deep into a particular technical problem.

After a really long time, I'm getting a kick out of working on a highly technical problem: an open source search engine from scratch (https://github.com/typesense/typesense). It has been deeply rewarding and definitely something that's missing in a typical web oriented software development career today.

karterkon SaaS vs. Open Core: Which Business Model Is Best?

I've been working on an open source search engine (https://github.com/typesense/typesense) and this exact deliberation has been on my mind.

Ideally, one could offer both on-premise and hosted versions (like Gitlab), but that's really tricky and needs a lot of resources to pull off successfully.

karterkon Ask HN: What is a broke person's way to learn data engineering?

Most people (regardless of where they are in the world) are not going to be able to afford a cluster of any meaningful size. So, the best option is to learn these tools by running them on your local machine. You can use Virtualbox or Docker to set up virtual hosts within your machine to simulate a cluster.

Also, data engineering is a vast field. Pick an area that you would like to pursue first and go deep: Machine learning (feature extraction+training), Hadoop/Spark job processing, event streaming & aggregation etc.

Then try to find a entry level job that would allow you to apply what you learned and also get to expand your knowledge of running actual clusters.

karterkon Ask HN: Are you put off building something because it already exists?

Good question, and something that's always on the minds of open core companies these days. In my case, I don't mind, and in fact would welcome it! It would solve my no.1 pain point: discoverability.

I'm not planning to build a billion dollar startup out of Typesense, so I don't face the same problems that Mongo and ES do. I also don't plan to offer a hosted version of Typesense. If somebody else fills that void, I will happy to let them do that. For an infrastructure product, there will always be people who want to run it in-house -- there is enough $$$ to be made in that space.

karterkon Ask HN: Are you put off building something because it already exists?

Great, I would love to hear your feedback on Typesense then. It's not meant to be a full-blown ES replacement. Elasticsearch is a beast but most people don't need that (atleast to begin with).

karterkon Ask HN: Are you put off building something because it already exists?

Started this as an open source project with the goal of first creating an ecosystem. So far, I have been getting good reviews from people who are using it as a search solution.

Eventually, there will be certain premium features that maybe only 10% of people will really want, and I will charge for that. Playing the long game here.

karterkon Ask HN: Are you put off building something because it already exists?

It depends on the size of the total addressable market. If the size of your target market is big, even if there are bigger established players, there is always a chance for a small, nimble product to carve out a niche. Established players often head upmarket (especially if they are VC funded). This often gives an opening for a new entrant to capture some of the SMB market share.

An example closer to me: the search engine market is dominated by Elasticsearch, which is open source, extremely popular and is now even a public company. However, I decided to work on a small project called Typesense (https://github.com/typesense/typesense) to try and carve out a niche (and perhaps charge for it later).

I decided to take a dive simply because of the size of the market -- search is required in almost every SaaS application, and there are plenty of downsides to ES that can be addressed with a nimble, alternative search engine. And that's what I am doing :)

karterkon Inflammation might be the root of preventable disease

This is a good point and I was about to write a comment to say the same. Nigella Sativa is also commonly used in many parts of Asia. It also has an anti-inflammatory property. You might be interested to read this: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3252704/

karterkon Chinese spies stole trade secrets from Dutch chip machine maker ASML: report

Read again, that's not what the OP is insinuating. In any case, this is a serious issue that deserves a serious discussion without getting angry.

karterkon Ask HN: Preference? Keyword Search or Semantic Search

Semantic based search systems are really great at finding the original intent behind queries. Not everybody is good at finding the right keywords to express their ideas or intent. In such a scenario, semantic meaning based querying rewriting helps a lot.

However, semantic search is a hard machine learning problem and it requires a good volume of search queries so that they can be mapped to the results returned and mine for patterns.

What's the middle ground? You can show a machine learning based "related searches" as a hint in the sidebar. That way, you can help the user construct their search phrases.

Shamless plug: if you like Algolia but would like to host the search engine yourself and don't want to manage Elasticsearch, take a look at Typesense: https://github.com/typesense/typesense

karterkon Bleve: Full-text search and indexing for Go

I've also been working on a simple, fast, typo-tolerant search engine. Shameless plug: https://github.com/typesense/typesense

karterkon Xapiand: A fast, simple, modern search and storage engine

If you are looking for an easy to run/manage, typo tolerant search engine, I've been working on this: https://github.com/typesense/typesense

karterkon Lambda School Announces $14M Series A Led by GV

I have been following Lambda School and I am really happy to see them succeeding in this space.

I've a question: how is Lambda School different from the other many other coding bootcamp programs (apart from the unique model of paying the fee only upon finding a job with adequate salary)?

karterkon Redis is not “open core”

I don't know why Antirez is on the defensive here about being open core. So what? If that's a viable model for keeping Redis sustainable, then that's all that matters.

For what it's worth, having closed sourced extensions for an open source licensed core product (in this case Redis) is the canonical definition of "open core", and this post does not really prove otherwise. Sure, there could be some minor variation here (with Redis Labs not owning Redis directly), but largely it's the same model for all practical purposes.