Ask HN: Does anybody store files in S3 for their SaaS? | Hacker News Reader

Ask HN: Does anybody store files in S3 for their SaaS?

5 pointsBeefin8 comments

Is anybody storing non-text files in S3 for their software? I'm referring to pdf, video, audio, images, etc.

If so, it would mean the world if I could talk to you and just ask about some of your experience with uploading/retrieving these files within your software.

To give context: I have a free managed API that offers full-text-search for your S3 bucket to integrate into your software. The website is https://mixpeek.com

Let me know if this matches you and if you're willing to answer some quick Qs in exchange to be involved in the free pilot :)

View on HN

What you describe is part of the AWS "Well Architected Framework". Static files should, as much as possible, be hosted on S3. This takes the pressure off the web/app server, is cheaper, increases resiliency to DDOS attacks, and allows much faster loading, particularly of video files, when one integrates the S3 storage with a CDN (such as CloudFront).

What I would find interesting is anyone running a SaaS project and NOT using S3/object storage for a significant amount of their files.

I think you misunderstand. The non-text files remain in S3, my service just parses them for text, then stores it in a Lucene index to become searchable.

I think there's a typo in the code example for the Python client that would result in a NameError if run

  from mixpeek import mixpeek

Should be

  from mixpeek import Mixpeek

Given that it is the class instantiated below.

One point in the Healthcare and Insurance use case reads as follows

>Ensure the are up to date Insurance Documents

Do you mean "Keep Insurance Documents up to date"?

We use S3 for https://iko.ai and give a filesystem like experience for notebook users. They can 'cd', 'ls', etc on S3 buckets right from a Jupyter notebook.

We both integrate with external buckets, private or public, but also use an S3 compatible object storage (MinIO), for which I wrote this library: https://pypi.org/project/bmc/

One thing I find will hinder your product's adoption is the "/upload" route. If I were to use your tool, I'd want to use it to search through already existing objects in my bucket, not to upload to the bucket.

That was one objection we faced, and then we added the ability to mount S3 buckets because people had datasets of 1TB or so in existing buckets.

Oh good catch !

Thanks for the feedback, you're not actually uploading the file. you're just sending it to be parsed and then the metadata is stored in a Lucene index.

Yes, we store the generated PNG images for CDN links at Bruzu.com

This seems like an interesting use case, since Bruzu pictures are all text. How are users searching these today?

no search feature. and these are PNG images not text files.

sounds like a great use case tbh, if you ever want to make these searchable for free shoot me an email and we can setup a conversation: info@mixpeek.com

Reply on news.ycombinator.com