Creating Expressive C++ Smart Pointers for Remote Memory | Hacker News Reader

Creating Expressive C++ Smart Pointers for Remote Memory

people.eecs.berkeley.edu

67 pointsxiii140811 comments

IMO a stronger and more flexible concept is a stream. A stream takes a buffer reader/writer of some kind (mem, socket, file, etc..) and does the usual I/O & encoding/decoding. It's similar to read()/write() vs fread()/fwrite() in stdio.

But this doesn’t seem to solve the issue the author had to provide typed access to remote memory.

How so? If the stream's buf is HTTP and you do a seek(x); read(y); that can be converted to a an HTTP POST + Range: x-y; and the stream can decode/cast to the type required.

This doesn't really fit the random-access primitives that remote memory provides, and you still don't really get type safety.

For parallel programs, streams are problematic because they enforce a strict order to communication. (I assume you're talking about creating distributed istream/ostream classes.)

Also, stream communication is explicitly two-sided; if you want to pull in a value on an istream, there needs to be a corresponding process on the other side inserting something into an ostream. I've found pointers to be a much better abstraction for data structures, which often require irregular access that two-sided communication can't satisfy easily.

Streams also have the problem that, while the underlying interface (a socket, file, etc.) is untyped, the interface is typed. This leads to the unfortunate situation where you can attempt to pull an std::string in on an istream and get junk because you're using the wrong type.

This does depend a lot on the kinds of programs you're writing and what your assumptions are--in the HPC world, we don't generally care about fault tolerance, and we actually have hardware that gives us fast RDMA read/write. In a distributed systems setting where processes are communicating with messages and need to be fault tolerant and reach consensus using algorithms like Paxos or something, I could see abstractions like streams being really useful.

> Also, stream communication is explicitly two-sided; if you want to pull in a value on an istream, there needs to be a corresponding process on the other side inserting something into an ostream.

That's not always the case is it, though? stdin on a standard computer for one, and in embedded world you might have an ostream to speak UART and give yourself a nice printing function, but you probably won't have a corresponding istream.

So why can't this be similarly exceptional?

That's true--you can add the ability to seek, in which case the stream becomes like a file.

Whether or not that's desirable depends on what kind of programs you're writing. For programs like databases that traditionally store things using the filesystem, that's probably a very useful abstraction, and some people at VMware have actually implemented an "RDMA over the filesystem" abstraction, which is super cool: https://256fd102-a-62cb3a1a-s-sites.googlegroups.com/site/mk....

If you're looking for an abstraction with which to directly implement programs that use data structures, however, I still think pointers are much more valuable than files because files are not strongly typed, and it's easy to read junk due to type errors and lack of structure.

Pretty neat! Would like to see if/how to handle field offsets/getters/setters that avoids loading/storing the whole encapsulated object. (Is that something the C++ templates can do without special annotations?) E.g.

  struct coord_t {
    float x;
    float y;
  };

  remote_ptr<coord_t> p = getRemoteCoord();
  p->x = 42.f; // as described by OP, only modifies local copy?

Almost! This is something I've looked into, and hopefully I'll have time to write a whole blog post about it in the future.

There are some fundamental syntax limitations due to the way that operator-> works (and that operator. does not exist).

The best thing I have is a `pointerto()` macro that gets you a remote pointer to a member of a remotely stored object. This uses C++'s `offsetof` macro and a whole bunch of template voodoo to make the types work.

  struct coord_t {
    float x;
    float y;
  };

  remote_ptr<coord_t> p = getRemoteCoord();
  remote_ptr<float> p_x = pointerto(x, p);

  float x = rget(p_x);
  rput(p_x, x+1);

You can make operator-> work as well, for both methods and members, but unfortunately right now it has to involve pulling the whole object, storing it inside a proxy object, accessing the member of the local object, and then storing it back in remote memory in the proxy object's destructor. It's a cool piece of code (I'm open-sourcing this library as soon as I have a free weekend to clean it up and document it).

I hope that metaclasses or reflection in a future version of C++ will allow the syntax you wrote above, but right now this is the only way I've come up with.

While there is a possible solution for this (as the author describes in the previous comment), this is something you should not really consider a useful pattern in your application. Prefer a struct of arrays instead. This makes the code way cleaner results in better cache utilization.

Cache utilization... of a remote system access? What cache? This isn't RAM, Dogg.

Reply on news.ycombinator.com