2,909 karma · joined October 8, 2010
Thanks for this! I implemented Entity Resolution from scratch 2 years back and was mostly able to do deterministic linkages using identifiers or using TF-IDF on names as a suggestion for human intervention. Will be writing about my learnings in upcoming months [1].
Was mostly looking at research papers before but will take a look at your project - your documentation looks good!
[1] https://www.sheshbabu.com/posts/entity-resolution-challenges...
> figure out how to pull out entropy scores Can you elaborate on this? "entropy scores" == how unique a value is?
> "Hammonds was innovative, but she couldn't do what she set out to do," says one DB insider. "So we're just sitting here, waiting to see what happens next."
Yikes! Sounds like an impossible uphill task.
Is there any examples of such turnarounds in the recent years?
Just curious, how does microservices help here?
Just checking, do you use stop words (removing “the”, “an” etc)? Can also add “its”.
What would be a good alternative for it?
[1] https://kubernetes.io/docs/tasks/access-application-cluster/...
Do you know what is the reason for these results? Is it because of C++ (MemGraph) vs Java (Neo4j)?
Any other queries where MemGraph does poorly compared to Neo4j? What are the downsides to using MemGraph?
Offtopic, does Memgraph have something similar to NetworkX's connected components [1]? Wondering what's the performance difference between both for different sizes of graphs.
[1] https://networkx.org/documentation/stable/reference/algorith...
How does it compare to igraph, graph-tool etc - https://graph-tool.skewed.de/performance
* Levenshtein
* Jaccard
* Cosine
* Jaro–Winkler
* Soundex
and many many more!
Very easy to read and explains the tradeoffs of the algorithms
I used Streamlit to build a small prototype and without any prior knowledge, I was productive within a couple of minutes! Really impressed with how simple and easy to learn you've made it.
I think Streamlit might be a good platform to build internal apps, only thing missing is a way to provide authentication. Is this something on your roadmap?
I've written more about it here: http://www.sheshbabu.com/posts/frontend-config-management/
It's also useful when you want to quickly build a "good enough" version of a feature like search so you can get it in front of your users fast and iterate on their feedback. Most of the time, they'd be quite happy with the results and you don't have to spend time on something like managing Elasticsearch.
I wrote a post on how you can use postgres to add search capabilities with support for queries like
jaguar speed -car
ipad OR iphone
"chocolate chip" recipe
http://www.sheshbabu.com/posts/minimal-viable-search-using-p...
Here's a really good guide on how to do this - https://docs.gitlab.com/ee/development/avoiding_downtime_in_...
Here are some migration libraries:
That's a pretty interesting technique!
I built a privacy friendly analytics project https://github.com/sheshbabu/freshlytics and I've been wondering how to correctly count unique visitors to a website. I don't store cookies or any PII data at all so by definition, it's hard to distinguish between two different visits - are they from same person or different people?
An alternative approach is used by Simple Analytics - https://docs.simpleanalytics.com/uniques where they use referrer header to derive unique visits. They mention that they don't use IP addresses as they're considered fingerprinting.
But it looks like a hash function (whose salt gets rotated daily) strikes a good balance between fingerprinting while maintaining user privacy. Any downsides to this approach?
You can use Taurus [1] to write scenarios in yaml files instead of using JMeter's JMX.