DTE

Love this. Elegant and powerful. Stateful streams are surprisingly difficult to DIY and as everything becomes a stream of tokens this is super useful tool to have in the toolbox.

Hey HN! We built Paperspace (YCW15) and we are super excited to announce Autonomous. We just announced our $15M fundraise to build the future of financial intelligence.

Why do we think now is the time for Autonomous? Millions of people already ask AI about their finances, but there's no Cursor-style "apply" button and it doesn't run continuously. The technology finally exists to deliver institutional wealth management through AI.

Wealth management generates ~$250B/year in advisory revenue, with human intermediaries extracting massive fees without creating proportional value. It feels like a pre-Robinhood brokerage world: entrenched, expensive, structurally misaligned.

We are following the neobank playbook that companies like Mercury and Ramp used in traditional banking: replace brick-and-mortar with elegant digital experiences.

Also we are hiring across multiple roles in NYC and SF —> https://atg.science/

Arxiv: https://arxiv.org/abs/2408.04619

Dillon here (CEO @ Paperspace, YCW15). I want to give a huge thanks to the YC community and all the support over the years. We have always admired DO and couldn't be happier to join forces!

We (Paperspace, YCW15) also give you the ability to run a high-performance workstation in the cloud (and soon on any remote machine!). Check it out https://www.paperspace.com

Disclaimer: I am the CEO/co-founder

The "Society of the Spectacle" is a wonderfully enigmatic take on media and tech. Highly recommended.

Another really exciting co in this space is Runway (https://runwayml.com/). Primarily focused on ML-enhanced media production workflows.

Also worth looking at ReactFlow (https://reactflow.dev) if you are looking for this type of tool.

We (Paperspace YCW15) have something similar at https://ml-showcase.paperspace.com/ where you can launch and run some popular Jupyter notebooks and run for free :)

Disclaimer: I’m the CEO/co-founder

Just a quick note: Morgan Funtowicz from HuggingFace did an awesome webinar overview of the transformers library here for anyone who is interested https://www.youtube.com/watch?reload=9&v=SUqi_E_Lyjs

CEO of Paperspace here. I'm really sorry about this. That is not the experience that we are striving for and FWIW, since leaving beta, Gradient is much more mature at this point (many millions of hours of runtime and lots of developer work). We have been aggressively stabilizing (and building out new features) over the past few years and continue to improve the product every release. My sincerest apologies for your negative experience early on I hope you will give it another try.

Dillon from Paperspace here. If anyone is interested, this is actually part of a longer multi-part series covering GauGAN in-depth. Here's the link to the full series: https://blog.paperspace.com/tag/series-gaugan/

We (https://www.paperspace.com) have full Linux and Windows desktops with web-based streaming and web-based terminal access. Might be what you are looking for :)

I respectfully disagree :) I think that the notebook environment is a nice entry-point for lots of applications. We actually use it pretty regularly to launch more sophisticated experiments / multi-node training jobs, etc using our python SDK.

You are right that versioning is still an issue and we largely punt on it by using the docker container (with layer commits on each notebook teardown) as the versioning mechanism. Maybe not the best solution but it does have it's advantages.

I should also mention you can just as easily run these on CPU-backed instances as well. The GPU is not a hard requirement.

As for checkpointing data, that is still a relatively difficult problem to solve and our current recommendation is to use a combination of the persitent /storage directory and the notebook home directory. There are definitely issues with doing 100K+ of small files and committing those to the primary docker layer.

When you get to testing it out don't hesitate to reach out to use and we can try to see what the best solution is for your particular project. To date there isn't a "one size fits all" solution but we are working hard on making more intelligent choices behind the scenes to unblock some of these IO constraints.

Because everything is running in a docker container behind the scenes we support any kernel you would like. We have a handful of pre-built containers and you can also add a custom container very easily or build one off of a base template such as the Jupyter R stack. Here is a list of some of the container we provide by default https://docs.paperspace.com/gradient/notebooks/notebook-cont...

The 5GB of persistent data is available for by default for all free accounts running on the free instances. You can easily upgrade your storage up to 1TB by upgrading you subscription within the console. We can also provide up to 4TB by opening a ticket.

Yes, one of the differences in how we handle notebooks is that everything is actually run in a container behind the scenes. This means that is isn't just the .ipynb that we are hosting and if you install and dependencies, libraries, etc it will actually persist all of it inside of a container. This makes it much easier to share your work with others so that i.e. I could fork your notebook if you made it public and get all of the installed libraries and compiled dependencies by default. Hope that helps!

Edit: we also have another tool called GradientCI (https://docs.paperspace.com/gradient/projects/gradientci) that might also be of interest. Basically it lets you connect a GitHub repo directly to a project and you can use it to build your container automatically.

Hey there ! I'm one of the co-founders of Paperspace. We currently have a couple of options for data ingest (and more coming soon!). The current system provides a single persistent mount that you can access from any notebook/experiment/etc in the /storage directory. We also mount a special directory called /artifacts where you can pipe out any models, files, etc and they will be pushed to an S3-compatible object store.

Checkout the docs here -> https://docs.paperspace.com/gradient/data/storage

And happy to answer any other questions

Congrats to Aneesh and team! We (Paperspace, YCW15) are big fans and have been following these guys for a while now!

Very cool! We (Paperspace) just posted a tutorial on Detecting Pneumonia from X-ray scans using PyTorch for anyone interested in building some more intuitions around this type of Deep Learning: https://blog.paperspace.com/detecting-and-localizing-pneumon...

Congrats, Jeremy and crew!

I have been researching data orchestration/versioning tools for a long time and have been following the Quilt guys closely. It is definitely one of the more powerful tools in the ML/AI engineer's toolbox and solves a huge problem that almost everyone runs in to right out the gate. It's still early days in this space but Quilt gets a lot of things right and I'm super excited to see this product develop.

Full disclosure: I run Paperspace (https://www.paperspace.com) and am working with the Quilt team to integrate their tools in to our platform.

Good video on how they built out Radar. https://www.infoq.com/presentations/stripe-machine-learning

Really great breakdown! Another important thing to think about when building out a production or even dev environment are the other associated costs (i.e. bandwidth, storage, egress, etc). We think that these costs can be pretty obscene in the public cloud and we try to optimize as much as possible to keep the total cost down.

Disclosure: I'm one of the founders of Paperspace (https://www.paperspace.com) and we spend a lot of time thinking about GPU compute and pricing. Happy to answer any questions here

Great write up! I would be remiss to not mention Paperspace https://www.paperspace.com where we also offer cloud GPU infrastructure that is less expensive and more powerful than most of the larger clouds.

We also offer a suite of tools that makes setting up cloud AI pipelines a bit easier to set up and manage.

If this is of interest to anyone here's a $5 promo code to try us out : HNGPU5

full disclosure: I'm one of the co-founders :)

For ML/deep learning tasks you should definitely use a dedicated GPU. I would recommend our GPU+ (NVIDIA Quadro M4000/P4000) which has 8GB of VRAM and 1664 CUDA cores.

We have definitely been thinking a lot about what that would look like (i.e. is it more of a job architecture, an API, clustering, etc). Would love to hear your thoughts on what GPU Lambda might look like. Feel free to hit me up directly dillon [@] paperspace [dot] com if you want to continue the conversation :)

Hi guys, Dillon here from Paperspace (https://www.paperspace.com). We are a cloud that specializes in GPU infrastructure and software. We launched V100 instances a few days ago in our NY and CA regions and its much less expensive than AWS.

Think of us as the DigitalOcean for GPUs with a simple, transparent pricing and effortless setup & configuration:

AWS: $3.06/hr V100*

Paperspace: $2.30 /hr or $980/month for dedicated (effective hourly is only $1.3/hr)

Learn more here: https://www.paperspace.com/pricing

[Disclosure: I am one of the founders]

Dillon here (co-founder of Paperspace). Thanks for the article! Just a quick note that our billing is actually done down to the millisecond not the minute. I'll update our docs to make that more clear :)