HNHacker News
TopNewBestAskShowJobs

whyrusleeping

476 karma · joined February 14, 2013

submissionscomments
whyrusleeping··on IPFS, Again
So, at a high level you want more than just the backgrounding the pin task, you also need to be able to tell what the progress is, periodically. This probably means we need to do something with the api endpoint output (i was initially thinking 'just throw it away and mark completion').

> in some cases, if I've pinned it at all.

What cases would those be? If the file is pinned it should always show up in an `ipfs pin ls $CID`.

In the short term, I assume your service is a server side app that is making requests to the ipfs node, right? You could set some background task in that application that just waits on the `ipfs pin add --progress` call and keeps track of the progress for that pin, that way, when a customer queries it, and its in progress, you can return that information. Agree this should be built into ipfs at this point, but that seems like a reasonable workaround for now.

whyrusleeping··on IPFS, Again
Doing this right is not simple. It requires an entire task management system internally, which (as you know) i've sketched up here: https://github.com/ipfs/go-ipfs/issues/3114

Sure, we could hack this together quickly, but we're trying hard to avoid adding technical debt at this point. Adding every feature requested would put us in a bad spot.

Prioritizing this over the countless other things people continually ask for is hard, but we hear you. We welcome pull requests, and since you needed this for the service you were building, helping us out here would be fantastic.

whyrusleeping··on IPFS, Again
Yeah, we're watching this. They aren't so much piggybacking as they are just using the open source code for libp2p and our pubsub (which is fully peer to peer, not reliant on any central server that could shut it down). However, the way they are doing it makes it really easy to find infected nodes. Running a scraper we see around 3k peers involved
whyrusleeping··on IPFS, Again
Hey there! I’m one of the first developers to work on IPFS, and was the maintainer of the project for around three years. I’ve since been pulled into working on Filecoin, which has definitely taken away from the time I can spend pushing IPFS forward. That said, the IPFS Project has a really solid team and OSS community still pushing the project forward, and there is an entire team working on just libp2p (the network stack used by IPFS and other projects like Livepeer, Ethereum 2.0, OpenBazaar, and more). There’s been a ton of progress made recently, and the network has grown significantly. We’ve gone from nearly 30k peers last year (this is hard to estimate accurately, but best guess) to around 400k peers on the network today — roughly 13x growth. This massive influx in usage has taken a lot of our attention up, things catching fire that we didn’t expect to catch fire, and other bugs popping up that need to be fixed now. All this detracts our attention from things like docs and website publishing, but we recognize it’s a really important area we need to make time for and step things up. For example, there’s an overhaul of ipfs-cluster docs: https://github.com/ipfs/ipfs-cluster-website/issues/89#issue... We also put a lot of effort into our new docs website: https://docs.ipfs.io/ And we’ve been pushing a lot of specs for everything we build: https://github.com/libp2p/specs https://github.com/ipfs/specs https://github.com/filecoin-project/specs https://github.com/ipld/specs
whyrusleeping··on IPFS, Again
> it's unable to reliably verify that some peer is actually storing data, without a local copy to verify against.

Filecoin can actually do this. I'm planning on doing a blog post about how this works soon (in all that copius free time), but a good summary is here: https://github.com/filecoin-project/specs/issues/155

whyrusleeping··on IPFS, Again
We actually find NAT to be a pretty big problem still. Even using NAT-PMP, upnp, and hole punching, we still see a roughly 70% (sometimes much higher) undialable rate. Especially for people running ipfs in China (though high failure rates are also observed in other countries).

We're pushing hard on getting libp2p relays up and running to help get through this. The idea is that we can use a relay to make the initial connection, then ask the remote peer to try dialing back (assuming that both peers involved arent undialable).

whyrusleeping··on IPFS, Again
Hey there! I'm usually one of the people who likes to engage on HN. I'm actually taking a couple days off to visit family, someone just texted this thread to me. I'm gonna go through and respond to some stuff now.
whyrusleeping··on Obsidian: A safer blockchain programming language
Is there a language spec and examples anywhere? I can only find one example of what the language looks like in their insurance policy case study paper.
whyrusleeping··on Show HN: WAPM – Package Manager for WebAssembly
Signed packages are useful, but still don't solve the problem.

One fairly simple thing you can do to improve this would be to include build scripts as part of the package, and allow people to run those build scripts through wapm. The exact versions of any involved tools will need to be recorded too, but since there arent too many ways to generate wasm blobs yet, this shouldnt be too out there.

Eventually it would be great if everyone had fully reproducible build formulae (kinda like https://github.com/polydawn/repeatr).

whyrusleeping··on Show HN: WAPM – Package Manager for WebAssembly
Are all the 'packages' here reproducible? I'd love to be able to run something like `wapm verify <package>`, or `wapm verify --all`. Running arbitrary code from random people on my computer is sketchy even when I can look at the code. Its far more sketchy when I can't.
whyrusleeping··on IPFS Project Roadmap
Yeah! That's a thing we're working towards. We're currently looking into apt and npm, both efforts are coming along pretty well and driving development towards fixing the bottlenecks preventing it from "just working".

https://github.com/ipfs-shipyard/npm-on-ipfs https://github.com/ipfs-shipyard/apt-on-ipfs

whyrusleeping··on Opening the Filecoin Project Repos
Take a read through Ben's PoRep paper: https://eprint.iacr.org/2018/702
whyrusleeping··on Opening the Filecoin Project Repos
To generate a response to the verifier, you have to have your replica of the data in question. A replica (from proof of replication) is a unique encoding of the data that is quite slow to generate. If the prover isn't storing the replica, they will have to regenerate it before they can respond to the challenge, which takes a noticeable amount of time.
whyrusleeping··on Opening the Filecoin Project Repos
Yep! He gave the talk just before me
whyrusleeping··on Opening the Filecoin Project Repos
The proof of replication provides assurance that for some given data D, a unique replica R is created for it. What you’re proving with the Proof of SpaceTime isn’t that you have some data, you’re proving that you have the replicas. Generating the replica is slow, so you can’t do it on the fly. I gave a talk recently that goes over some of this briefly: https://youtu.be/GZZ2G9bPXsM
whyrusleeping··on Show HN: Next Browser native on Linux
We should have more people who can't wait to replace chrome.
whyrusleeping··on Libp2p – A modular network stack
Hey Mark, pretty sure you're talking about floodsub, which is a naive implementation of pubsub of top of libp2p, not actually 'libp2p'. Libp2p is quite a bit more than a simple p2p pubsub library.

In any case, floodsub is just one option you can use for pubsub on top of libp2p (one with a notably high level of reliability and minimal cross network latency for infrequent messaging). We also have a tree pruning gossiping implementation called gossipsub. Take a look at the spec: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub...

whyrusleeping··on Announcing gRPC Support in Nginx
last i checked grpc could only technically support bidirectional streams. None of the libraries I looked at actually implemented it.
whyrusleeping··on IPFS Pubsub: Distributed named pubsub channels based in the IPFS Cloud
Not yet, its somewhere in our giant pile of things to do. If anyone is interested in helping out on this I can definitely help out and point people in the right direction.
whyrusleeping··on IPFS Pubsub: Distributed named pubsub channels based in the IPFS Cloud
Hey Everyone! IPFS dev here, If anyone is interested in the future of ipfs pubsub you can participate in the discussions in our research repo: https://github.com/libp2p/research-pubsub . One of the next big things we're planning to do is to implement a new routing algorithm, one proposal for this is here: https://github.com/libp2p/research-pubsub/pull/17 (feedback and review very welcome)

We are also planning on implementing various different modes for authentication and encryption. Our pubsub currently is plaintext and open for anyone to publish to any topic

whyrusleeping··on Git hash function transition plan
Just a quick note, while we still would really love to have git use multihash. You can already serve git objects natively over ipfs via: https://github.com/magik6k/git-remote-ipld

Which uses our new plugin system: https://github.com/ipfs/go-ipfs/blob/master/docs/plugins.md

whyrusleeping··on Is Filecoin a $257 million Ponzi scheme? [pdf]
Thank you! that means a lot :)
whyrusleeping··on Is Filecoin a $257 million Ponzi scheme? [pdf]
Hey, ipfs dev here.

In filecoin (as in sia, and most other decentralized storage networks) the client who is storing data has control over who stores their data. It's a market. So for example, if a well known company decides to offer storage through filecoin, they can publicly attest to their miner IDs, and you can choose to make deals with them for data storage (even at a slightly higher price if they want to charge it).

Separately, we are working on schemes to allow miners to only serve a clients data in situations they approve of (authorization/authentication). This would address the problem of the data being 'public'. If you can control who stores your data and who can access it, it has the same attack surface as any traditional cloud storage provider.

whyrusleeping··on Is Filecoin a $257 million Ponzi scheme? [pdf]
> Cryptocurency integration and identity within IPFS could have been done without a new coin

I don't think thats a fair claim to make. There are many attacks on 'pay for distributed storage' that Filecoin addresses. Take a look at [the paper](https://filecoin.io/filecoin.pdf), in particular section 3.1 where we discuss sybil attacks, Outsourcing attacks, and generation attacks. We cover a lot of cases, and build in incentives to continually increase the amount of available storage and push down the price-per-byte. If we thought it was possible to properly incentivize data storage on ipfs without building a blockchain, we would have done so.

whyrusleeping··on Is Filecoin a $257 million Ponzi scheme? [pdf]
Hey, ipfs dev here. The amount of work being done on ipfs has been increasing continually. Work on filecoin is not detracting from ipfs development, and we are bringing on more people for ipfs, libp2p and filecoin.
whyrusleeping··on The decentralized WeTransfer
I wouldn't disagree with that.
whyrusleeping··on Uncensorable Wikipedia on IPFS
:)
whyrusleeping··on Uncensorable Wikipedia on IPFS
Yes, DHT, mdns (for local nets), and manual connections via `ipfs swarm connect`. You can get a peers address through any other open means of communication, connect to them, and then through them, get other peers information. All it takes is one good node
whyrusleeping··on Uncensorable Wikipedia on IPFS
I also recommend the `--progress` flag to get a bit of feedback.
whyrusleeping··on Uncensorable Wikipedia on IPFS
You can write to the same IPNS entry with multiple different computers, but i'm not sure that would be the desired way of updating this. One way i would imagine it could be done is using crdts in combination with some global immutable log (not necessarily a blockchain). Then there needs to be some sort of concept of identity to help avoid spam and trolls, I really like the idea of a web of trust, and that could very likely work here. I'd love to see more widespread applications of CRDTs
← PreviousPage 3 of 5Next →