> alignment tooling is fascinating, as we increasingly want to re-fit->embed over time as our envs change and compare, eg, day-over-day analysis. This area is not well-defined yet common for anyone operational so seems ripe for innovation
Training the parametric UMAP is a little more expensive, but the new landmarked based updating really does allow you to steadily update with new data and have new clusters appear as required. Happy to chat as always, so reach out if you haven't already looked at this and it seems interesting.
> And algorithms can only predict content that you've seen before. It'll never surprise you with something different. It keeps you in a little bubble.
This is not true at all, algorithms can predict things you haven't seen before, and can take you well outside your bubble. A lot of the existing recommendation algorithms on social media etc. do keep you in a bubble, but that's a very specific choice 'cause apparently that's where the money is at. There's enough work in multi-armed-bandit explore/exploit systems that we definitely could have excellent algorithms that do exactly the kind of curation the author would like. The issue is not algorithms, but rather incentives on media recommendation and consumption. People say they would like something new, but they keep going back to the places that feed them more of the comfortable same.
Assuming you have a dimension-reduction or manifold learning tool of choice (UMAP,PacMAP,t-SNE,PyMDE,etc.) then DataMapPlot (https://datamapplot.readthedocs.io/en/latest/) is a library specifically designed to make visualizations of the outputs of your dimension reduction.
If you just want in-memory then PyNNDescent (https://github.com/lmcinnes/pynndescent) can work pretty well. It should install easily with pip, works well at the scales you mention, and supports a large number of metrics, including cosine.
For suitable specialized cases thins can be quite efficient. For persistent H_0 of VR-complexes in low-dimensional space there is an O(N log(N)) algorithm for N data points; that's decently fast. If you want H_1 I believe (but cannot prove) that there exists an O(N^2 log(N)) algorithm. Beyond H_1 things get painful. Since most PH software is written for general cases they don't tend to avail themselves of these special case shortcuts. Given that H_0 and H_1 of VR-complexes of low dimensional data covers a vast amount of the use cases I think specialized code for this would be worthwhile.
It is really not that much slower for training (see the paper), and if you are interested in pipelines the difference is not so great considering you are looking at a one off training time vs. lots of inference.
If this is a thing you want to be able to do efficiently then ParametricUMAP (see [docs](https://umap-learn.readthedocs.io/en/latest/parametric_umap....) and [the paper](https://arxiv.org/abs/2009.12981)) will be very effective. It uses a neural network to learn a mapping directly from data to embedding space using a UMAP loss. Pushing new data through is only slightly more expensive than PCA, so being part of an inference pipeline is fine.
Density based clustering with high dimensional data will tend to struggle. This is because, in high enough dimensions, you need a lot of samples to see any density. Also distances start to look very similar (from the curse of dimensionality). To get any traction on such things you need some form of dimension reduction. For something like this non-linear techniques are going to be better. If you want a pipeline of standard parts then something like:
Pretrained-CNN --> UMAP --> HDBSCAN
can turn out relatively reasonable results, especially if the UMAP you use for the clustering is to more than 2 or 3 dimensions (often 5 to 20 is good, depending on the data). You can, of course, still use a 2D UMAP to visualize the results. If you want such a pipeline packaged up then consider the PixPlot package, designed for exactly this use case, from the Yale Digital Humanities Lab: https://github.com/YaleDHLab/pix-plot
* Disclaimer: I am highly biased, as an author of both HDBSCAN and UMAP implementations.
I suspect that this is because GPT-2 doesn't have any overarching narrative that it is piecing together. Ultimately it is like a super-powerful Markov based text generator -- predicting what comes next from what has come before. It has longer "memory" than a Markov model, and a lot more complexity, but where a person often formulates a plan for the next few sentences and the direction they should go, GPT-2 doesn't really work that way. And hence it sounds like dream logic because in dreams your brain is just throwing together "what comes next" without an overall plan. Of course your brain is also back-patching and retconning all sorts of stuff in dreams too, but that's a different matter.
I think you are making a false dichotomy here. It is perfectly possible for the article to be right, and there is still a future with general artificial intelligence and a singularity. If you believe the singularity is inevitable then you should read the article as saying that we are woefully misjudging where the asymptote is -- yes current progress looks impressive, but there are some really big steps that we are currently ignoring, and real human level AI is a century or two out, not a decade or two out. That's perfectly possible, and most philosophers who work in consciousness and philosophy of mind (as well as a very large portion of the machine learning community) will tell you that there is still some big hurdles that we don't even have the faintest idea how to cross (the whitehouse report on AI described it as a "chasm").
I believe bokeh can handle streaming data quite well. I remember at least one demo of various spectrogram and related plots updated live from the microphone on the presenter's laptop. It seemed impressive.
You might want [PyX](http://pyx.sourceforge.net/) which takes TeX formula input and can return you SVGs. You could also combine that with Sympy for formula manipulation etc.
I think he grasped the point. The trick here is reframing. Someone says "objects fall to the ground is an undeniable observation", and the other responds "No, your looking at it the wrong way; objects do something but you just think that's falling to the ground, in practice it's something else that has the appearance of falling to the ground". The responder isn't denying your observations, they are denying your interpretation of what you've seen.
The same applies to eliminitavists. They aren't denying that something happens that feels to you like experience, they are suggesting, however, that it may not be the thing that you think it is. They'll often point to people's own misunderstandings of their consciousness as examples that we can, in fact, actually be wrong about our own first person experiences. One example of this is the fact that we think we experience full field colour vision. In practice we have colour vision only in a narrow field in the center of our full visual field. You can perform experiments on yourself to demonstrate that this is in fact true. Another example is the sense of continuity to consciousness, which again, can be demonstrated to actually be false. If our "consciousness" is sufficiently different in reality from what we normally presume to be consciousness then, in some real sense, consciousness as we commonly think of it indeed doesn't exist. That doesn't mean there is nothing, just that calling it consciousness with all the associations that implies is perhaps sufficiently misleading as to be wrong.
Yes, his 'Consciousness is an Illusion' in clickbait is a sense. Mostly he is trying draw attention to the fact that the thing he is trying to explain is not the thing other people are saying is unexplainable. The commonly accepted notion of what consciousness is ... he doesn't believe that exists. But that's just because he thinks we are deluding ourselves about what we are actually experiencing as consciousness -- the thing we think about when we talk about consciousness? That's just an illusion, and the real workings are just mundane, and you'll just say "that's not really consciousness" if I explain them.
I think elimitavists like Dennett don't deny that things go on, just that consciousness, in he form of this magical thing, doesn't exist. The demonstration of that being that many of the things people believe about consciousness that make it so magical and hard to explain simply aren't true. People are often mistaken about their own consciousness, so perhaps the remarkable mysterious thing that we can't explain doesn't exist. Instead there could be something much simpler and explainable, much in the mold of explanations that Dennett gives. The catch being that many people simply claim that the thing that Dennett can explain isn't consciousness because it doesn't do the magical things they think consciousness does.
It is worth noting that scikit-learn does value clear understandable implementations, so you can actually pop open the source code and expect to find something other than a black box. Now, in many cases you'll have optimization work that means a slightly less obvious approach is taken, but the scikit-learn maintainers do work hard to try and ensure that, if you want to learn, you should be able to open up the code and do so.
scikit-learn doesn't have a strong neural network codebase -- for anything not NN based they've largely got you covered (along with good infrastructure tooling for pipelines, cross validation, hyper-parameter searching etc.). Contrary to the impression you may get if you only follow the current buzzwords there is a great deal of value in machine learning right now beyond NNs and deep learning. On the other hand if deep learning is what you want to do, scikit-learn is not currently the best library for that.
It's a common sentiment. [Ernest Rutherford](https://en.wikiquote.org/wiki/Ernest_Rutherford) is known for the quote "If you can't explain your physics to a barmaid it is probably not very good physics. "; the point being that if you can't find and explainable version of your theory, you don't really understand your theory yourself. There are also (potentially apocryphal) stories of Rutherford talking over his work with the cleaning staff as means to help himself make his own theories clearer -- the challenge of explaining it was useful in understanding it.
It uses t-SNE but there other other working parts here, including word2vec (and some nice compression of a pre-trained model), keyword searching to provide context for terms, and clustering to find natural dense groups. It's a nice pipeline that fits together a bunch of independently interesting parts into a single system that can produce quite remarkable results.
Personally I think that would make some sense. I suspect the catch is in presenting the results to the user: you have to present in 2D, and if you cluster in higher dimensions you may get results that, while perfectly valid, look strange when collapsed to 2D (overlapping/mixed clusters etc.)
To my mind this isn't so bad -- the notion of overlapping clusters is something I think most people actually accept (think of clusters as tags rather than a partitioning and you can get the idea). However I can see why you may prefer to have your clusters present more clearly in 2D.
Broken actually does a lot of stuff client side and you can simply update the data. It's much closer to d3 for interactivity,and really quite powerful. It does lack some features ATM, but it continues to grow space.
And yet there are 3 feet to a yard, 16 ounces to a pound, 32 fluid ounces to a quart, and so on. Yes it would be lovely if our number system were base 12 but it isn't that's not going to change any time soon. Realistically we need to work with the number system we've got, and at the very least be consistent.e
I don't know of any implementations yet. On the other hand it isn't that hard to get the basics working (see http://nbviewer.jupyter.org/github/lmcinnes/hdbscan/blob/mas... for an explanation of the algorithm). The trickier part is getting good performance, but for small datasets that doesn't really matter. Hopefully someone will put together an implementation for R soon.
Desnity Peak clustering is an interesting idea, but is still centroid based and will have many of the same issues as K-Means (but may pick better centroids for this case). It also involves a little bit of parameter tuning (particularly with regard to bandwidth), and can be relatively slow if not suitably accelerated with space indexing structures such as kd-trees or cover-trees.
The examples are all in 2D because it allows someone to visualise what's going on. In higher dimensions things get messier and you have to rely on cluster quality measures ... which are often bad (or, more often, defined to be the objective function that a particular clustering algorithm optimizes, and hence give a false sense of how "well" the clustering has done). I've worked with HDBSCAN quite successfully on mid-range dimensionality data (50 to 100 dimensions). I agree that it doesn't work as well on truly high dimensional data, but then little can once the curse of dimensionality truly kicks in. Your best bet, at that point, is the assume that your data actually lies on some lower dimensional manifold (i.e. the intrinsic dimensionality is much lower) and apply suitable dimension reduction techniques (t-SNE, Robust PCA, etc.) and then cluster.
If you want to use an HDBSCAN algorithm on graphs then I suggest you look into Spectral Clustering (which traditionally uses K-Means, but could use HDBSCAN instead. Otherwise you may want to consider graph specific algorithms such as Louvain.
It's included in the upper left corner of the plots. To be fair, these are for the sklearn implementations, some of which are excellent, but I can't speak for the performance of all of them.
I agree that on some level more data sets would be nice, but I felt that it cluttered and obscured the exposition. Instead I used the one synthetic dataset, but crafted in to have various properties (noise, cluster shape, variable density, non-standard distributions) that will confound many different clustering approaches ... it is meant to be the "hard" case that with all the difficulties and confounding factors rolled into one dataset.
Oddly enough HDBSCAN can be recast as a persistent homology computation; the trick is in simplicial complex construction; you need something slightly more density aware than traditional Rips complexes. I am currently working on designing a version of Carlsson and Memoli's Multiparameter Hierarchical Clustering based on the persistent homology interpretation of HDBSCAN.