oscargrouch

>That's interesting. Are you aware of the absurd-sql project?

No, thanks for pointing that out, will take a look into it.

> Also, how do peers find each other?

The idea is to use the torrents as a common shareable resource where given the peers have the same interest in that torrent, lets say the torrents works as a "meta-database" with just enough immutable metadata info, giving the developers of that application a list of peers that will have the same RPC service you designed, with a interface that suits your purpose according to your application goals (a distributed Youtube for instance), and giving you know and designed that API yourself whatever you want from each node, lets say a piece of data, be it a file or a database key-value range, you can ask your API for it, combining the torrent peers and whatever distribution combo you need.

> And finally, do you intend to open source it?

I've just did

https://github.com/mumba-org/mumba

It's badly documented given i'm on the final touches before a proper launch, but the "documentation" of the storage layer is planned for today, giving how important it is for the whole thing.

The first thing i've tried to get right was the storage layer, and the capacity to use mutable sqlite databases over torrent (together with files which are simpler given they are meant to be immutable).

Given theres no proper doc yet, i can point out to the source at

https://github.com/mumba-org/mumba/tree/main/lib/storage

where:

https://github.com/mumba-org/mumba/tree/main/lib/storage/bac...

is a modified chrome cache storage layer (which is the real underlying disk storage) that abstract the files and databases storages that from the bit-torrent layer perspective are on the disk.

https://github.com/mumba-org/mumba/blob/main/lib/storage/sto...

is the front end

and the:

https://github.com/mumba-org/mumba/blob/main/lib/storage/tor...

is the main abstraction that may be a "fileset"(collection of files as in torrent) or a dataset/database, which in this case you can get the sqlite db handle from the torrent object and deal with it as normal database.

I've take care to enable a key-value store over the sqlite btree, so both form of databases are possible, a key-value and a normal SQL database.

Key-values are important for the distributed case where you may want the nodes to have partial data and abstract a SQL layer (or whatever) on top of the distributed nodes, which is a better solution for distributed storage.

For the database distribution, a 64k SQLite memory page maps to a torrent piece of the same size, which can be synchronized over other peers that knows what's the root of the merkle tree of the given database is (you can use the RPC layer to coordinate this or use the bit-torrent DHT updating your slot with the new merkle root)

BTW This is what i'm using to distribute the applications, the DHT which points to a "database torrent" which in turn is a index to other files and database torrents. The application owner have write permissions over that particular DHT slot and can change the root merkle anytime the application itself changes (a new version, or some of the assets).

What im finishing right now is exactly the higher level layers that automate this whole thing and make it work "under the hood" without the users or applications distributors need to understand how it is implemented.

Of course the developers will have access to iterating over the peers of a giving torrent to group them over RPC interfaces, and the idea is also to give access to the DHT so solutions that need a access to mutable ever-changing merkle root is also possible (even though the torrent based solution cover most if not all of the ground), but the idea is to give the devs the tools to be creative about their solutions.

(The solution is a whole browser-based UI/web applications platform, that use torrents and the torrent DHT for the p2p storage layer)

My current project can expose a sqlite database over torrent wrapped by the Chrome cache filesystem.

The torrent is merkle-tree based, and for each change in the SQLite memory page it also updates the merkle tree giving you a different torrent infohash at the end.

So as databases are published together with the applications the initial torrent info is the same, so theres a swarm of initial peers. Once it gets updated and the torrent "turns into another", the solution is to use the RPC api (that my solution provides) over the initial swarm of peers and use some sort of distributed algorithm like raft or gossip to define how to deal with the subsequent changes over the peers.

The important part is that all other changed torrents from all the peers are available to all peers in the network, and with a RPC working as a abstract interface to organize them, it will work for a lot of different scenarios.

I also have a project to explore this alternative way of peers communication but i have a different answer to this, and i think its better if its a network of peers that expose API's

https://github.com/mumba-org/mumba

It's badly documented as i have just published to github, but i hope it gives a clue of how is supposed to work.

I'm on the final touches over this project, but the main concept is already working as is 90% of it, but i think exposing SQL is too raw, and maybe dont offer the whole picture, as for instance, what is important is not data, but sometimes pure computation.. Eg. suppose you offer a deep leaning inference where you receive and give back tensors..In the middle of it is a different sort of computation, where it doesnt have anything to do with databases.

Or yet, suppose you need to access something in a third-party before giving an answer, or if you want to do it in a distributed fashion without you api consumer even noticing it?

API's are a good answer to that, and in my opinion are superior interfaces, whatever the semantic web of the future will be, it will need this network of API peers to work as a floor to it.

For instance, you can design a Graph API on top of it. Exposing your data layer directly is bad engineering as there's a lot of problems you wont be able to solve, and where leaving clients to talk to "you" over a well-defined API will.

To put it simply, in my point of view the direction the semantic-web is pointing to is cool, but the answer is not the right one, and this idea of exposing SQLite directly while is cooler, yet have the same flaws, or else something as GraphQL would have taken the world as its not much a different answer than the one presented here.

Looking again at it, its quite bigger now. I wonder how it compares to Boost and now also Abseil? (features and performance-wise)

Not the op but, giving this rely in whatever is on the host OS, the DLL's on the host OS will evolve changing its interface or even not ship anymore while with applications that rely the bare minimum like electron this is not a issue.

i've forked chrome to create something like this (meant for native apps in a multi-process environment) and can confirm.

The multi-process architecture and the chrome platform is already a overkill just for a browser (in my opinion), its much more if for each application you have a browser process + gpu process + several renderers.

If you have only one main process, + 1 gpu and a process for each application running you solve that problem, even more if those process are not running javascript as in my case.

It depends of how this will be laid out. If the future generations choose for a more homogeneous sort of intelligence it might trigger the opposite effect, as humanity might loose a sort of balance the natural world struggled to achieve.

But if somehow they manage to understand organic balance and go for a more heterogeneous composition of inteligences, than it might work out as intended.

I guess the best approach is to just reinforce and even boost the genes that are already there in the gene pool of the human being.

Lets get just one intelligence characteristic for instance that will be sort of harmless to boost in everyone: give us more memory..

Except for the fact that forgetting about some things can do wonders to our psychology and for a human being resisting through all the years it must to..

Maybe the suicide levels will rise, and people unable to function in society will also rise, creating the opposite effect.

The thing is, this is a really complex matter, and i hope that when we are ready to do this, we live in a more advanced society (morally and philosophically) than we do now, or else the consequences of people that are unable to understand the consequences of what they are doing, do it without being really prepared for it.

For me, the the nazi-germany of the 30's is a (extreme) cautionary tale of a possible future of ours, where we are technologically advanced, but morally corrupt, and that combination will probably lead to the human kind fallout, if nothing changes..

Our technological progress is good, but we need to rethink other important things, and the movie is on spot over this (they had all the technology to solve the problem, but they had the emotional IQ of toddlers).

I agree with you that one language will take the productivity you've gained in a later stage, but we would need to subtract the time fixing those bugs, as compared to the time spent on code to avoid them in both languages (one before and another later).

I'm pretty sure that a lot of people will be more productive with Rust as compared with C++ as with others will be the opposite. So i guess in by making an average over those two crowds, i doubt they will differ that much.

The sort of bugs Rust will avoid that are kind of hard to debug, giving it doesn't happen that often is the problem of shared state in a multi-thread environment.. the other kinds are often easy to spot in C++, the only difference is that they are caught in runtime.

I think a lot of us with experience in "system languages" would acknowledge where Rust fits in.

But there are a lot of people in IT with experience in other fields that gets on into the marketing sales pitch that is trendy since the Java days, and i'm not blaming the community of doing anything wrong here, giving its the expected behavior of tech communities nowadays. But there is this noise into how Rust is sugared that i dont think its true, in that its the first system language that is as productive as a mid-level normally GCed language like Java or Go while giving more control and performance.

I think there is no language into that spot yet, but i think that something that follows Rust could achieve this (Its something in the middle of Rust and Swift).

The control and performance are there, as happens to C++, but the productivity is basically the same, which is not very good as compared to other languages that target more of being productive than being in control.

The control level pay out in the end in the quality of the final product, but it s not the best option for things that dont require the amount of control that C++ and Rust gives you.

But the thing with Rust, C++, Zig, etc.. is that, when you need it, you need it. There are things that can only be done with those kind of languages, and often are very complex and high-performant piece of software.

Its not for everyone, and you need a sort of commitment that you would only pay if you really need. So writing things like 'web services' on them is not optimal, and you should use languages like Go, Java et al. for this.

Tech people should be able to navigate hype and marketing lingo and see the things for what they are, i think is the same that is happening with 'web3' right now.

Sure. is there a way to contact you?

This is cool.

I'm also working on a decentralized distribution mechanism based on torrents, and while i've being working on a different architecture, the network mechanism are basically the same with a couple of differences on the network level.

In my case i'm working on a very customized version of chrome where the web api is actually available for native applications beyond Javascript.

from the look of it, you will need to have the handshake resolver installed and replaced as your local dns resolver. With that it will return you a the public key address on the bittorrent DHT which can be solved by some torrent client.

There it should have a torrent info payload that your client can turn into a ordinary torrent, where you can proceed to download the files, and can open in your browser on your local filesystem.

Of course, if you want a very efficient application, you are completely right. But the thing is that with Swift you can get a Java like performance (which is pretty good) out-of-the-box even while paying the ref-counting penalty, without the Java memory penalty (it has a low memory footprint) and without having to run it all with a external VM.

Of course, if you want to catch-up with the average speed of C++ you will need some effort, but knowing that it will be pretty fast without much effort and without all the other penalties is reassuring.

Its in the Java, C# family of languages, where you can do pretty powerful things without the complexity of the likes of C++ and (now Rust) with a very good performance.

And when you take the family Swift is in, (now also with Kotlin), i consider it the best language over its peers not only given the design, but the fact that being AOT compiled from the start it's of great value.

Having said that, i feel inclined to invert the question as, why i would not want to use and learn it?

In my experience at least, a lot of IT folks, just don't have the mental energy for the likes of C++ and Rust, but can peak at a language like Java which can already be complex enough.

For instance, i deal mostly with C++ all day, and i don't think i would choose it if i were to create an application with buttons, multi-media, etc.. Nor Rust(same problem as C++) or Go(poor design for that goal). Given there's a language like Swift which combines the design and speed/efficiency to create such a thing.

If i were to create a backend service for instance, i would take Go into consideration, and probably in that case it would be a better fit than Swift or Rust or C++..

So its about being the right tool for the job.

I'm not sure what the real problem is, but we live in a economical system that have a tendency to hijack governance of things that are important to 'them' (whoever the contextual them are) and re-purpose it's goals and direction working as a effective power-grab strategy.

You can see this happening with browsers for instance, where the companies now are the ones who define the standards with things that are important to them, like DRM or showing ads, embellishing the pill with things nerds and hackers also like, where once this governance where in the hands of Tim-Berners-Lee et al.

And that's because we are talking about technologies that born more free of corporation oversight, which is the minority.. (Think Java first with Sun and now with Oracle for instance)

This Rust thing is just because powerful corporations are adopting it, and defining board members, so they can define more the direction of the platform and the technology.

For those corporations corrupt board members that know who owns them and what interests they should defend are more important than the personal character of that board member..

It's always like this and it will always be like this. Capitalism is a amoral system and it will turn everything it touch into its own image..

Until we don't stop and fix the root cause of the system that is corrupting everything that we value its like thinking you can stop a ocean wave by resisting it while on front of it.

Note: I'm not advocating to just destroying capitalism as a solution, but to acknowledge that this is actually systemic and happen all over and instead to try to fix the specific problem caused by it, we should get back some steps and try to fix the problem, which is systemic, for everything while preventing it from happening all over again in the future.

Is this the same one that is in use by Chrome?

Any plans to create more sandboxes like this for other platforms like Windows and Mac?

Also apparently is Javascript only, which are the most well served community in terms of recognition, funding and (comparatively) less complex and smaller codebases.

I wonder how a underfunded and complex crypto library, emulator or VM feel about the effort expended vs. the funds that goes to 3000 line of code javascript libraries that are linked everywhere.

Not to understate the work and effort in those libraries, but the guy who creates a engine (lets say in C++) which is wrapper in a popular python library and do the heavy lifting will be underfunded while the guy that wraps it over python will receive all the funding.

My guess is, well organized, hype-based language communities will win in the end, even if the technology is inferior(not the language, but the libraries), giving they will be able to generate more funding.

Other language communities should keep an eye on this and step up, because in the end "it's the economy, stupid".

Sovereign states are anemic by now and i'm skeptical that they would want to fight where the money and (real) power is, and if they grow some balls, or stop being so corrupt, even than, i'm skeptical that they will manage to win this fight.

We are entering into a new era where big techs are so massive they can fight sovereign states and win, heading us into a new era of neo-feudalism were "civil rights" are only granted if they are not in the middle of our new digital lords lust for profit and power.

So people might think this is unlikely and conspiratory, but its just natural human nature, social dynamics and following the political vectors by the aggregation of the past events.

If they are willing, they know the secrets of everyone, and its just a matter of pressing the right people to bow to their will and that's it, we become the fools of Weimar republic that were not aware of the dangers ahead of us, because there were no historical precedent to make us vigilant and aware, and all the signs were well aligned to a hostile takeover.

I think what i'm working with would definitely help in this endeavor.

Will let this comment here also as a mark, so i can come back latter (2 months from now) and poke the author to see if it does what he was intending..

Of course it not the concrete shell, but the platform to build such a shell, and anything else (applications in general) really..

I think Rust kind of aged C.

It happens when some new band push the needle for a genre, its only when this happens that the "modern" stuff start to look old.

C is getting deprecated while C++ will need to catch up somehow (if there's a way).

This is actually a good effect of Rust, making people think more that safety is important and that its possible to have more of that in languages without a GC.

(Sorry for hijacking the question, that was not directed at me, but this is a good debate so i couldn't resist)

I was trained to understand that illiteracy in English means someone that cant read or write, but if it was translated as-is, illiteracy means "iletrado" -> someone "without letters" which in my mother language doesn't mean that someone cant read, but a broader category of people who cant read together with people who have very a poor language understanding and communication skills.

Some illiterate 'iletrado'(for us) in some cases can read, but might not understand it very well.

That might explain why the OP used the given word because in some languages its the definition that defines people who cant read.

I pity companies that will follow this hype train and rewrite perfectly working software to achieve nothing in the end, from a pragmatic point of view.

You can do an essay for every other vector of software development and claim how a rewrite will do you wonders:

Eg: FP, strong type system, Linux...

The problem is, in real life, we have a bunch of a problems to solve from many vectors, not just (the little square) of memory safety related bugs.

And people from the real world tend to measure those vectors together for their particular problems and often their answers, even if they go for C++, are a better suit to solve their issues than a objective-imperative from a person who looks only from one perspective, and think that is 'the most important thing®' that should be pursuit, risking avoidable financial bankrupts (by threatening financial bankrupt).

Think for a minute if the native access of a giving library, say, LLVM or Tensorflow is much more important for the project and/or the company survival..

Its like Bush's "War on terror" making people feel fear about some boogeyman but actually inflicting the real terror they said you should trust them to avoid.

Specialized, cartesian division of technology tend to turn us all into neurotics, and the problem with neurotics is that they don't see the bigger picture which is much more complex and systemic as things in real world tend to be.

Contrary to the extremists views, modern C++ is a fine option and people should not go for this FUD tactics, making they fear to opt for a perfectly nice platform to develop things, and that has proved itself over and over through time doing amazing things.

Rust might be a fine option too, but this whole urge to rewrite C++ software is pure non-sense.

https://www.joelonsoftware.com/2000/04/06/things-you-should-...

This whole "written in Rust" as if it was a feature BS must stop someday.

Get the project 'A' features and compare with project 'B' features, the strength and weakness of programming languages will be apparent, on the features, runtime speed, bugs, etc..

Choosing a project over another just because its in a given language is not based on logical reasoning, but emotional reasoning and therefore the bad paths will get evident with time (Except of course if the language is really unsuitable to the domain, which is not the case here whether with C or Rust).

This emotional bias for me is a reason to actually look at Rust projects with suspicious if people working on it know its just another tool with some given properties and not some magical wand that will turn everything into unicorns and rainbows.

Note: this is also targeted at "why you would want to choose C over others ?" comment bellow in the same line

I'm finishing a browser based application platform where the applications installed expose a RPC api, so in the end all applications can call others in the same local(or remote) node/s.

The beauty of this is that you also can compose with other nodes and for a distributed service by calling the local service as a proxy and routing the requests to the other nodes of the same api.

It took more time than i've predicted because its also expected to deliver UI and most of the 'HTML5' api to native applications (instead of Javascript), which is a massive platform by now (and the #1 reason why newcomers to browser technology cant compete, giving the feature creep tax imposed to them).

The idea is also to distribute over a DHT so you can just serve your application over torrent without needing to register anything..

The only way to get there is by empowering users and developers and taking some of the control from the cloud platform giants.

In my point of view the only way to break the browser monopoly now is to create a new path forward, a branch.. its not the time to follow the rules, its time to break them or else the future doesn't look so bright in my opinion..

You must remember that each Electron instance will have its own gpu and browser process.

If you do just like Chrome, one host process + one gpu process and all the rest are renderer process as it was meant to, you will use less memory.

Im working on something that are meant to enable native applications with a Chrome-based platform, and it was one of the first thing i've tried to achieve.

Electron was just being sloppy and stick with its first MVP.. because being able to centrally control things through a process like the Chrome browser process does, give you a lot more functionality to explore, compared to what Electron is doing where each host/browser process is isolated and don't know about the other players.

Its akin to use a cannon to kill a fly..

My problem with all this, is that they don't get it where this is taking us all.

Big Tech have a tremendous power all over the world. Lack of regulation and civil rights enforcement is taking us into a era o digital feudalism..

So its not just about their rights to let Apple strip their rights, but the rights of us all.

It's about our future, because governments are getting weaker and big tech companies are start to take over with power that transcend territory.

So its our duty, to not let this happen, because they are not really seeing, being selfish and thinking about their own convenience.

The worst part is that, it will not change anything to them, but the marketing machinery whisper all the time how this is bad for them, making they fear any sort of change in that direction.

Won't comment on your first point because i think its pretty well documented by now, specially for the HN crowd. But if you really want to to go down into this rabbit hole..

> iPad is a consumer device. They don’t make it for “developers.” You aren’t their target audience.

Sure, but the consumer is, a consumer who would want the best experience and the best applications on his device.

Crippling and curbing the developer forcing the hand where he cannot use certain technologies or certain paths only to suit Apple monopolistic, control freak neurosis, will in the end damage the users, as he would not have access to goodies he would like and want to.

This is basically the reason why Apple is getting away with it. The user doesn't even know he is being damage in certain ways so Apple can resort its control, because the application that should reach him, apps he would want if the option was given to him, never will reach him.

Because even if the developer miraculously circumvented an obstacle (that is not there for the final users sake) and published its app, the control over app store publishing would take care of censoring the application through the evaluation process.

Just a sample of where the final user is being damaged, the fact that he cannot really run other browsers that are basically UI shims over Safari, because Apple is afraid of web based applications multi-platform applications to work on your device.

> So yes run your own OS, but if you bought a product and expect its software to do things its creators are clear it doesn’t do, then that’s on you.

That's the point being argued here. They doing all they can to not let this happen. You are saying as if there's still an option to do this.

It gets even worse, because inside the OS, you are their lap dog. They are enforcing options to curb any form of competition. Technology is being forcefully crippled so they can resort monopolistic forms of control.

And with the other hand they teach you to like it in a paternalistic way.

The minute you buy something, what you do with it has to do with your rights and not the rights of the company that created that thing. I don't get it this line of argumentation.

Imagine someone was kidnapped against their will, and once we were to discuss the rights of the person being kidnapped the person kidnapped with stockholm syndrome have argued in favor of the kidnapper, as if it was within his rights to break its own rights.

Because this is what you are advocating here. That a company should have the right even to break your rights because they are a "good willing actor" or that they are doing it thinking in your well being (which you are inclined to think with tons of well crafted and manipulative propaganda towards this goal).

And its even worse because, its not the third party that its at fault according to your point of view, but the government that is trespassing by trying to force with laws, approved in a democratic regime, that your rights have to be respected and there are limits into how they can limit you or forbid what you can do with something that you bought and therefore its your property.

It would be no problem if Apple were just doing their best to deliver the best experience to their users as you are implying they are doing here.

The problem here is that they are deliberately crippling and limiting what the hardware can do in a way that avoid any possibility of competition.

Limiting JIT's and external web engines are an example of not allowing the devs to have an option to develop for web SDK forcing them to stick with the apple SDK.

So, there's no problem in Apple trying to do things better. The real problem is the limitations imposed to curb competition and freedom for the developers to use the hardware the best way they can.

Apple is ur-facism encoded in a private company. Google or Microsoft dont get away with half of what Apple do, so people should look more carefully to what is beneath.