HNHacker News
TopNewBestAskShowJobs

lgierth

796 karma · joined January 7, 2012

I work on libp2p and IPFS -- https://github.com/lgierth

[ my public key: https://keybase.io/lgierth; my proof: https://keybase.io/lgierth/sigs/96UTewVcgE6mn2Hh1Tg3WCHUDRP_jWSFookoCcfruAg ]

submissionscomments
lgierth··on Team Discussions
So, my colleague just started a discussion in one of our teams, it showed up in my email inbox with a subject indistinguishable from an issue comment, and with its own number counting (#1). That's kind of a deal breaker for me unfortunately.

I'll give it some more time to mature before considering it :)

lgierth··on Some problems of URLs
Very interesting concerns about URLs! I've looked into better ways of addressing recently too, but coming from different problem spaces:

1. how to decouple content from location, i.e. location-addressing vs. content-addressing 2. how to accept more network connection constructions than what URLs permit (proxying, domain fronting, tunnels, etc.)

> We could also imagine a new URL syntax, with fewer and less ambiguous syntactic meta-characters.

That's part of what we're attempting for network addressing with the multiaddr format: https://github.com/multiformats/multiaddr - If the author of the article is reading here, I'd love to hear your thoughts!

We're going with encapsulation and a posix filesystem compatible plan9-ish syntax and approach there. multiaddr describes the semantics of networking namespaces, and then various content systems bring their own namespaces. IPFS brings the /ipfs and /ipns namespaces for its content.

lgierth··on Chrome breaks the Web
> Does anyone know any mainstream browserd planning to implement, I dunno, IPFS?

It's actually not too far off - IPFS is already capable of running a full js-ipfs node in a WebExtension's background page and expose its API to content pages. What's missing is minor things like streaming to/from the background page, and WebRTC in the background page.

For proper ipfs:// protocol support in the address bar and in links, a better protocol handlers API in WebExtensions is required though, which will take some more time. Basically right now with the ipfs-companion extension, ipfs:// URLs get rewritten to http://.

lgierth··on PeerPad – A realtime P2P collaborative editing tool powered by IPFS
Note that https://peerpad.net is hosted right out of a go-ipfs gateway. You can instead also use https://ipfs.io/ipns/peerpad.net or http://localhost:8080/ipns/peerpad.net, or use a specific version of it:

    > open https://ipfs.io$(ipfs resolve /ipns/peerpad.net)
lgierth··on PeerPad – A realtime P2P collaborative editing tool powered by IPFS
These websockets node you see aren't servers. They're nodes like any other node, except they have a well-known domain name. They don't do any work that a browser js-ipfs node wouldn't be capable of.
lgierth··on Orchid: a new surveillance-free layer on top of the existing Internet
The build system is written in nodejs -- it turned out way faster than both CMake and classic Makefiles.
lgierth··on Orchid: a new surveillance-free layer on top of the existing Internet
Caleb James Delisle's Networking Suite
lgierth··on Judge Recommends ISP and Search Engine Blocking of Sci-Hub in the US
You're not giving me much to work with :) It's all about the data, how you write/read it, and how you structure it. IPFS gives you a networked database for immutable graphs, IPNS for mutable links, and PubSub plus CRDTs for multi-writer scenarios.
lgierth··on Judge Recommends ISP and Search Engine Blocking of Sci-Hub in the US
And thanks for the feedback! :) We're actually thinking through the documentation overhaul right now (today and tomorrow) and large datasets are a top priority use case, so your post is really valuable.

I'd say at this point private networks and the filestore are unlikely to change significantly, but we're not 100% satisfied with the respective test suites (and the docs, oh my) so it'll be a bit longer until they'll be elevated out of experimental status.

About the object patch command, you can also call it through the HTTP API if that makes it easier. The CLI is essentially a 1:1 mapping of the HTTP API to a console command.

lgierth··on Judge Recommends ISP and Search Engine Blocking of Sci-Hub in the US
Hey, IPFS dev here, happy to help :) I promise we'll improve our documentation around storing and providing large datasets.

1) You can have individual files automatically be wrapped in a directory: `ipfs add -w dog.jpg` will result in `/ipfs/QmSomeHash/dog.jpg`

2) There's commands for modifying objects: e.g. `ipfs object patch add-link QmDirHash the-link-name QmLinkHash`

3) You can host multiple IPNS names from the same node since 0.4.9 or so -- see `ipfs key` and `ipfs name publish -k`. You can also publish updates for the same IPNS name from multiple nodes, although admittedly that's a bit hacky right now, since it involves an additional tool: https://github.com/whyrusleeping/ipns-pub

4) What kind of directory structure are you looking for here? We've successfully and repeatedly used IPFS with large datasets of different kinds so I'm curious how we can improve this.

lgierth··on ZeroNet: Decentralized websites using Bitcoin crypto and the BitTorrent network
There's no such limitation in IPFS -- currently the largest known dataset is about 3.8 TB, a mirror of https://cdn.media.ccc.de
lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
You got it the wrong way around - Filecoin will be built on top of IPFS, but IPFS itself works fine without Filecoin.
lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
See PubSub: https://ipfs.io/blog/29-js-ipfs-pubsub/

You can use it as the base for CRDT structures: https://ipfs.io/blog/30-js-ipfs-crdts.md

lgierth··on How the Catalan government uses IPFS to sidestep Spain's legal block
We are working on extensions/addons for all major browsers -- you can see an early version here: https://github.com/ipfs/ipfs-companion
lgierth··on How the Catalan government uses IPFS to sidestep Spain's legal block
You're likely missing gx, the package manager used by IPFS and many projects in its community. Try:

    go get github.com/whyrusleeping/gx
    
    go get github.com/whyrusleeping/gx-go
    
    cd your-project/
    
    gx install
    
    go build // or whatever you were doing :)
lgierth··on How the Catalan government uses IPFS to sidestep Spain's legal block
Thanks for mentioning domain fronting! It's also been used by Signal to circumvent communications censorship in Egypt [1]. We want to bring domain fronting to IPFS by making libp2p's websockets transport capable of it. [2]

Fun fact: advanced networking setups like domain fronting are impossible to address in a URL/URI scheme. Check out multiaddr :) [3] A domain-fronted service could be addressed as something like `/dns4/google.com/tcp/443/tls/sni/google.com/http/example.com/ws`

[1] https://signal.org/blog/doodles-stickers-censorship/

[2] https://github.com/libp2p/libp2p/issues/18

[3] https://github.com/multiformats/go-multiaddr

lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
One very important vector for adoption that often gets overlooked is interoperability. The cost of adoption can be significantly reduced by making sure the new thing nicely interoperates with the existing deployments. We're attempting to do this well with IPFS and libp2p.
lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
Content-addressing doesn't alleviate the problem 100%, since content can still fall off the network - but it improves the structure of the network in a way that makes it tremendously easier to keep content around. You don't need control over the original domain name to help in distributing a certain piece of content.

My colleague Matt addressed this beautifully in a recent talk at the NSDR Symposium: https://archive.org/download/ndsr-dc-2017/04_Speaker_3_Matt_...

// edited to fix URL

lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
> Last I checked IPFS will not tolerate minute long latencies and requires a bandwidth above several kilobits per second which would immediately disqualify it for anything farther than the moon. > >And I'm not sure it would work on the moon since that is a 2 second latency and I had issues with it when I used it on a mobile phone network with 800ms latency.

Fair points :) We'll be addressing this in the coming months with increased work on the network stack (libp2p).

lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
I agree that HTTP doesn't have to die -- for many use cases it has serious flaws though, the most important being its coupling to location-addressing. You always need a domain name and path to construct a URL. If the host decides to change the path, or loses control over the domain, you're out of luck.

Fun fact: when URLs were being defined, some people expressed the opinion that location-addressing is a huge mistake to begin with. Almost all of the links and references in the respective mailing list archives are now: broken.

lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
The one huge advantage of content-addressing over location-addressing (= URL with domain name and path) is that the original source isn't solely responsible for keeping the content online and the link working.

Anyone can help out by keeping a copy. With location-addressing additional copies of the content aren't just largely hidden, they also get into a weird mode of competition with the original URL. With content-addressing, additional copies instead forge stronger resilience.

lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
Noms is also a great example of a peer-to-peer database: https://noms.io
lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
You're right - you can model any data as append-only though. Granted, in many cases it will require you to seriously sit down and remodel your data. Nobody's claiming it's going to be SQL and ACID transactions :) It's more likely going to be collaborative append-only logs based on CRDTs.

There are examples of the use cases you mention being built with decentralized technologies. [1][2]

[1] The various cryptocurrency wallets and exchanges

[2] https://openbazaar.org

lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
Ask any non-technical person whether they've ever been bitten by link-rot! :)

Content-addressing doesn't alleviate the problem 100%, since content can still fall off the network - but it improves the structure of the network in a way that makes it tremendously easier to keep content around. It's not up the original source of the content (owner of the domain name) to keep the content around - anyone can help out by keeping a copy.

My colleague Matt addressed this beautifully in a recent talk at the NSDR Symposium: https://archive.org/download/ndsr-dc-2017/04_Speaker_3_Matt_...

lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
Another important factor for adoption/adoptability is how well the new system integrates with existing deployments of older systems. Ideally it completely interoperates with the older systems, while providing you with additional value right from the start.
lgierth··on HTTP is obsolete. It's time for the Distributed Web (2015)
IPFS can discover other nodes in the same local network via mDNS -- you don't need to have an internet connection at all to share data locally.

Combine this with the recent work on PubSub [1] and CRDTs [2] and you can make many applications work locally, that are otherwise annoyingly strongly coupled to internet services (thing Etherpad, Google Docs, Skype, Github, etc.)

> For a truly distributed network we should look into ways to make internet work more like a mesh.

Yes! We'll be putting more work into the network stack (libp2p) in the coming months. IPFS itself has done a ton to rework how content is defined and moved around, and libp2p will do the same for the network connections underneath. Think overlay networks, cryptokey routing, packet switching.

[1] https://ipfs.io/blog/29-js-ipfs-pubsub/

[2] https://ipfs.io/blog/30-js-ipfs-crdts.md

lgierth··on HTTP Immutable Responses
Nice to see this has made it through the standards process -- the `immutable` keyword is tremendously useful for systems that store and provide actual immutable data, e.g. content-addressed distributed systems.
lgierth··on Deploying a global, private CDN
And Vultr gives you BGP Anycast :)
lgierth··on Burger King launches WhopperCoin crypto-cash in Russia
Are whoppers a scarce resource?
lgierth··on I Bought a Book About the Internet from 1994 and None of the Links Worked
I've had to deal with Hetzner and IPFS too -- my conclusion is that it's Hetzner who are aggressive here. In of the cases I had fixed the dialing-local-networks behaviour, and then Hetzner still continued to block the server for about a week. They blocked it on 25-Dec and released it on 31-Dec.
← PreviousPage 5 of 10Next →