pythux

Interesting take, but there is no indication that the new Rust-based stack will be « less expensive » to maintain than the current one. I am curious to see the next posts in the series. I would also like to know if they considered other alternatives; they mention Elm, what about TypeScript. The issues they mention seem to also have to do with the specific choice of frameworks or tools: React, Webpack, etc. Not so much with JavaScript itself?

Hi,

I work at Cliqz in the engineering team and have been involved with this advent project. I'd like to provide some background.

For a while, we have been thinking internally about having a tech blog for Cliqz engineering teams. It is fairly common for tech companies to have a separate blog for technical content, separated from the main website. We picked a name: 0x65.dev (0x65 being the hexadecimal representation of 101, the idea was that we would write about the tech at Cliqz in an accessible way).

To make sure we'd commit to writing on this new blog, we thought of organizing an "advent" for Christmas, where we would publish one article per day digging on one of the aspects of what we do: search, privacy, browsers. When creating the list of topics, we did not optimize for Hacker News (or any other community for that matter, we also post and twitter and Reddit equally), as this was suggested by some other comment. We really wanted to create a coherent schedule to explain why and how we do things, to have a logical progression in the topics (looking at the posts from first to last, it should hopefully make some kind of sense!). We started on day one with "Why the world needs more search engines", to motivate our efforts in this area, and started detailing on following days the way we built our engine from scratch.

One thing we wanted especially, was for engineers to write about the projects they work on, to get their perspective, without any filter. So far, many different people have contributed. If some of the technologies or keywords we use seem "trendy", it's because we are ourselves hyped about those, and we use them internally to build our products.

Before the event we never anticipated this would resonate so much with the HN community, and while very pleasant, it was not the goal. We were of course extremely enthusiastic when we saw the great feedback from the community. If anything, this proves that the topics which we are passionate about are also of interest for a part of the HN community (many of us at Cliqz read HN every day; in fact we all used our personal accounts to submit, comment or discuss about the posts). It also shows that what we do is not disconnected from reality, and that there is real interest around the topics of search, independence, privacy; a good validation for us!

The "last time Cliqz came up" can be found here: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... We have been posting multiple articles on our tech blog, explaining what we do and how we do it in great details. Your link points to an old thread of more than two years ago.

There were more recent discussions about Cliqz no latter than this month, in particular here: https://news.ycombinator.com/item?id=21676252

[disclaimer: I work at Cliqz]

Nice read! I am not very familiar with this field of research but, could runtimes of other languages (say, Node.js or Python) benefit from such optimizations? What about libraries like libuv, I guess they must be fairly fine-tuned already? Or is this something that is specific to Go and would be hard in other contexts?

I used to love keybase and used it for a while, but somehow since the crypto coins donations I started getting followed by random people and getting some "spam". So nowadays I avoid it and stick to Signal. I really like the idea behind it though and I hope they manage to solve of these issues eventually!

Sorry if I expressed myself poorly but that is exactly the point I wanted to make. To maintain backward compatibility, new constructs with partial overlap with existing ones are introduced; and this overall increases the complexity of the language. Whereas breaking backward compatibility would allow to re-use existing constructs and change their semantics (I am not saying this would be a better path though).

> New methods with simple behaviour don't really make the language more complex through.

I think they do, in some ways. If you have multiple methods or functions which seem to do similar things, as a new-comer it can be incredibly confusing. I remember when I started I never knew what the "best way" to iterate on things was... for loops? for in? for of? foreach? And it was not obvious which one to use or what were the differences. I agree that it might be less ambiguous between "replace" and "replaceAll", but I think over time these things matter.

I have a more meta question related to Python, and only remotely related to this post. With the huge use case of machine learning and deep learning that Python is loved for; I am wondering if other traditional use cases that Python was used for could suffer. What about future developments for the language itself, is ML big enough to influence the direction and priorities of the language? This is far fetched but, if a critical mass of companies and people start using Python for ML exclusively, could it hurt Python as a "general purpose language"?

Really liking these new developments! JavaScript is not the horrible language it used to be anymore (in my very subjective opinion). When I started writing JavaScript in 2016 after a Python background I was frustrated every day... Then after some time and learning about which dark corners to avoid, which tools to use, etc. it became quite an enjoyable experience. Nowadays I mostly use a mix of JavaScript and TypeScript depending on the projects and it's been great.

One thing that I am a bit afraid about though is that the language might become more complex and complicated over time because of backward compatibility (C++-like?). I saw that they will be introducing a new function "replaceAll(...)" to avoid the weird semantics of "replace(...)" (stopping after first occurrence unless a 'global' RegExp is passed as argument). And that's great, but that's a new function, and maybe 'replace(...)' should have had this behavior from the start. I hope to be wrong on this one!

That is a good ideas. If we can do it in a privacy-preserving way, which seems doable, we will most likely do it (although we would be constrained by resources available)

> No references in any of your published source code, and your search engine isn't free software:

You looked at the wrong tab, check in "Code" to find "code" related to Anolysis: https://github.com/search?q=org%3Acliqz-oss+anolysis&type=Co...

> Cliqz has done unsavory things in the past (like the Firefox fiasco a few years back, for example, which I can't fault Cliqz entirely for: Mozilla is just as guilty).

Not sure how this is Cliqz' fuck-up. We are not hiding anything. On the contrary we are very transparent and detailed about how everything we do is designed to not track users. All of this is on our new tech blog: https://0x65.dev, feel free to have a look between two comments on HN and give us some feedback!

> "anolysis" gets around both uBlock Origin and uMatrix, despite both of them automatically blacklisting any URL with "analytics" in it, as an example. Getting around the most popular content filterers on the internet is a pretty strong signal.

It's not called "getting around it" when there is no tracking or ads going on (if you want to see how smart the "most popular content filterers are" check out this link and see that the image is blocked because it contains the substring: "analytics": https://whotracks.me/blog/private_analytics.html. Wicked smart!).

Anolysis is not a typo, it's a project name, people tend to do that when they care and spend a lot of time on projects: give them names. So, at the risk of repeating myself, Anolysis = Analysis + Anonymous (at the time we thought it was a pretty neat name!).

Anolysis does not operate outside of Cliqz products (no websites analytics here and we do not rely on a third-party, we built it in-house for this reason) and we put a lot of work into it to make sure it does not use a unique ID (like virtually every other analytics out there) but allows to by-design not track any single user (in fact the system does not even have the concept of a user). Sure, we did not write extensively about it but I guess we have to start somewhere (in December we are writing on 24 different things we do, we will be sure to consider Anolysis as a good candidate for a technical blog post in the future).

What you attribute to malice is simply a lack of time, as you probably noticed Cliqz is working on solving a lot of very hard problems (search, browsers, antitracking, adblocking, privacy-preserving telemetry and so much more) and writing a paper about the new system you designed and implemented is not always the priority :)

One thing that is not stressed in the article but seems relevant, is that, while being designed to detect and block fraud attempts without de-anonymizing users, the system might very well be super effective against malicious SEO as well. Wondering how techniques to game search engines will evolve in the future with counter-measures such as this one.

This post is part of an "advent series of posts" about search, privacy and security. First ones posts were:

- "The world needs more search engines" - https://news.ycombinator.com/item?id=21676252

- "Privacy or Data, a Convenient False Dichotomy" - https://news.ycombinator.com/item?id=21684708

- "Human Web – Data Collection Without Privacy Side-Effects" - https://news.ycombinator.com/item?id=21694980

There is and it is planned for tomorrow! Stay tuned. As a small spoiler... We experimented with Tor in the past for this proxy network and it worked well but had other constraints (more tomorrow). Currently we rely on a third-party proxy provider (with a custom contract: they don't keep logs, etc.) and heavy crypto to make sure that we do not get any network-level information about users, and the proxies do not get any information about the content of messages. In Cliqz browser it's also possible to enable search through this proxy network for anonymity!

Indeed, the idea is to use each day until Christmas to unveil a new post about search, privacy and security. Today is about opening a discussion about data collection practices. It's often presented as a "all or nothing" (more often than not it's "nothing"). But data is needed to create independent and viable alternatives to the giants so maybe there is a middle ground if it's done with privacy at heart from the ground up.

Edit: For those curious about the details, we've started a new page where we gather research, talks and blog posts from the past about these topics: https://0x65.dev/pages/dissemination-cliqz.html

If you are ok with some extra latency you can enable our “search via proxy” feature which proxies all queries and hide your IP. Maybe that is what you are looking for?

Hey, I work at Cliqz. As mentioned in another thread; we have an advertising model, but it does not rely on any personal data. We have also experimented with paid products and we would love for it to be a viable option. Unfortunately, experience suggests that very few people are willing to pay (we would love to be wrong and will keep considering this possibility). So we are exploring many options; what they all have in common though is privacy by design. We do not, and do not want to collect personal data.

Hey, I work at Cliqz. We have an advertising model, but it does not rely on any personal data. We have also experimented with paid products and we would love for it to be a viable option. Unfortunately, experience suggests that very few people are willing to pay. So we are exploring many options; what they all have in common though is privacy by design. We do not, and do not want to collect personal data.

Thanks for the feedback, if you find something fishy in the requests then that's like a bug and we will gladly fix it ASAP.

Disclaimer: I work at Cliqz.

That is factually incorrect: https://beta.cliqz.com

That is exactly the point of the article, by the way. Cliqz is built in Munich, Europe. And it does not rely on any other US search engine to provide results (contrary to DDG, Qwant and others).

> Does the Cliqz browser still disallow plugins, such as ad blockers?

Hey, I work on Cliqz' adblocker (open-source here: https://github.com/cliqz-oss/adblocker). Cliqz browser comes built-in with adblocking, anti-tracking, anti-phishing and private search built-in so that users are protected by default.

You can also install another one if you prefer since all Firefox extension are also available from Cliqz! But the one built-in is pretty good already[^1] and we're happy to get more feedback about how we can improve it!

Finally, Cliqz search is also available standalone now so feel free to give it a try! https://beta.cliqz.com/

[^1]: https://whotracks.me/blog/adblockers_performance_study.html

Absolutely love this quote from the article:

> Paraphrasing Monty Python: “What has Google ever done for us?”. They have done an awful lot for society, for the web and probably saved it a few times a long time ago. But it is time to also see them as the empire they have become. And empires must fall.

I think people generally do not understand well enough the far-fetched consequences of a monopoly in the search business. We really need more diversity and competition in this space, not just for quality of results and convenience of services; but for society as a whole.

The article rightly points out that Google is a new "Cambridge Analytica" (even on steroids), we should not passively wait for our democracies to break down because information is controlled by a single private entity. Let's step out of our comfort zones, try new products, support the ones we like!

Great point! We wrote about how we're doing it on whotracks.me a while ago, if anyone is wondering how that could work: https://whotracks.me/blog/private_analytics.html

Once Manifestv3 is rolled-out, Chrome will not support uBlock Origin at all.

Doesn’t DNS server only see the hostname instead of the URL? The approach would allow to only scrap home pages then I guess. Also, what about pages behind login?

Chrome did not introduce any limitation yet and it’s gonna take a while before they do it (they only started showing part of Manifest v3 changes in canary versions). So nothing changes for forks of Chrome for now.

If you have some source about the efficiency advantage, could you please share?

I only ever read that hypothetically it could be more efficient but never stumbled upon actual measurements/benchmarks.

Also, extension adblockers are already so efficient that I would doubt there is a perceptible difference.

In principle yes. But in practice, we are talking about nanoseconds; and I would very much like to see benchmarks/measurements showing anything that can be perceived by users. Also, this blocking cost is still orders of magnitude lower than network latency and blocking requests (even with a slow adblocker) will result in a noticeable performance boost while browsing the web.

That’s the goal, but they’re not there yet. Quite a few things are missing to match uBlock Origin (Or other popular adblockers).