pencildiver

Agreed on the large margin of error. Working on a bot to store and convert the images to webp to improve performance. Having the bot do a check for any images that don't exist and removing those listing. Will likely also need to triangulate this with a 404 check. Recently added an option for users to mark a product as "sold out" on the search results which will help as well.

Unrelated, but what was the "weird criteria" for the secret santa exchange? Half joking but also helps with figuring out filters :)

Thanks! Laughed audibly at your comment in the other thread about Mongo haha

Looking into all options right now. A tradeoff between current stability, price, and performance.

Can't believe I missed this. Taking a look at both repos now. The further I get into this space, the less I feel like I know. Appreciate you sharing, will reach out!

Found a list on a site called "Built With". Mentioned this in another comment but I think it's meant for building sales outreach email lists.

Once you have the store URL, you can get general store information at [Base URL]/meta.json

Here's an example: https://wildfox.com/meta.json

Thanks for sharing this! If you're up for it, I'd love to talk more about your experience, especially the technical tooling. Working as fast as I can to understand the right way to approach the tech, as there are tradeoffs with performance and price. I'm at support @ searchagora .com

OP here. Yup, I am in the process of starting a holding company LLC for my software products and small investments. Just went ahead and deleted 2 from the Investments page that are not launched yet but still in-development (just had landing pages up for those). Wasn't planning on releasing the Jaggi site yet, as I'm still wrapping my head around the holding company structure / it's new to me.

Agora has been a side project of mine. TBH in retrospect, I wish I would have given this post more thought as the servers / search performance wasn't prepared for any significant traffic. So definitely didn't game HN.

You’re live on Agora:

https://www.searchagora.com/products/invisible-calendar-6266...

Thinking that we should have a page where store owners can submit their URL to be crawled.

Absolutely. Currently on the search results page, you can click on the title / price area to open the product URL in a new tab. Open to any suggestions as well. So the product URL is accessible from the search results page or if you click on the product image to open the product listing and click on 'visit product'. Let me know if that makes sense and if you have any suggestions to make it better!

Will do. Thanks for the heads up!

Awesome, thanks for the suggestion / link! Actually left another comment about potentially doing semantic image search to improve results so wrapping my head around it now.

Not yet! Definitely something to consider as I upgrade the architecture. Would it be helpful to have API access to all products on Agora for your own app?

I definitely need to watch it now.

Right now, charging Shopify store owners $99 / product / month to give them a 'verified' tag and boost their product in search results. Currently not making money on affiliate fees.

I wanted to first prove that people would actually use this / find value in it. Fortunately a few merchants have reached out already via email to talk through the business model so this will likely evolve as we learn more.

I've definitely noticed that already. Any advice on how to spot that?

Another challenge is that there are products sold on the original site and third-party marketplaces, both of which could sell on Shopify. So need to find a way to automatically detect the type of store.

Just upgraded the storage and put in a few fixes so it's working a bit faster now. Working on caching some responses locally as we speak. Great idea.

Amazing, glad you were able to find it. I also just learned about what a "Dala" is :)

Thanks for sharing this! Definitely wasn't expecting this level of traffic so didn't account for some front-end loading experiences. Implementing these now.

For 3, thinking to let the back button work the same as the "x", that way a user can return to where they are in a search result regardless of what they click on.

Great suggestions, looking into this right now. First time building something like this so definitely new to some of these tools.

For scraping: Found that every Shopify store has a public JSON file that is available in the same route. The JSON file appears on the [Base URL]/products.json. For example, the store for Wild Fox has their JSON file available here: https://www.wildfox.com/products.json.

Built a crawler in simple Javascript to run through a list that I bought on a site called "Built With", access their JSON file with the product listing data, and scrape the exact data we want for Agora. Then storing it in Mongo and, currently, using Mongo Atlas Search (i.e. saw they released Vector Search but haven't looked at it). It has been a process of trial and error to pick the right data fields that are required for the front-end experience but not wanting to increase the size of the data set drastically. And after initially using React, switched to NextJS to make it easier to structure URLs of each product listing page.

Mongo will run me about $1,500 / month at the current CPU level. AWS all in will be about $700. I'm currently not storing the image files, so that reduces the cost as well.

A few improvement that has helped so far:

- Having 2 separate Search Indexes, one for the 'brand' and on for the 'product'. There's a second public JSON file that is available on all Shopify stores with relevant store data at [Base URL]/meta.json For example: https://wildfox.com/meta.json

- Removing the "tags" that are provided by store owners on Shopify. I believe these are placed for SEO reasons. These were 1 - 50 words / product so removing these reduced the data size we're dealing with. The tradeoff is that they can't be used to improve the search experience now.

Hope this helps. Still wrapping my head around all of this.

OP here. Yup, haven't launched the holding company yet but idea is to have an LLC for all my software projects. Still a work-in-progress in both thinking and execution. Agora specifically came from a personal need and is obviously still an MVP project.

Spun up a Merchant Page and Affiliate Program page in a few hours on Webflow using a template. There is a merchant dashboard built but the 'affiliate program' is a test.

Enter key should work, but the loading speed is very slow right now. Fixing that right now :)

You can reach out at support @ searchagora.com. Would love to talk!

Absolutely. Working on a "ships to" filter and enhancing the 'price' filter.

Also fixing the loading experience as we speak. Wasn't expecting this level of traffic so didn't account for slow server speed with the front-end experience.

Super cool project (especially as a coffee lover myself)!

Yes. There's a very basic price range filter right now. Working on adding a ships to, location, and a few others. Open to any ideas that would help in the shopping experience.

There are 'reviews' now and made the decision to only let authenticated users leave users so they are more trustworthy (i.e. thinking is that adding more friction will lead to higher quality reviews).

Thank you, that means a lot. It has definitely been a whirlwind of emotions since posting on HN but glad I did. It's definitely an MVP so going to work fast to improve it.

Yeah absolutely. I hadn't heard of Shop until today but the value proposition is definitely similar. In the next week, I'll add other e-commerce platforms like BigCommerce, WooCommerce, support for custom built sites, etc. to really differentiate the user experience.

Great call! I am doing back-ups on Mongo and this is a good use-case for tracking changes. Also trying to figure out how to detect is a product is sold out or not being sold anymore.

Really happy to hear this. I'll do my best to keep this around :)

Bought an initial list of 2m stores for a few hundred dollars from a website called "Built With". Think they are used for building sales outreach lists. Then narrowed down the focus to stores to US only and between $100k - $1m in revenue to keep the initial data set manageable (and the CPU / Storage costs reasonable).

Definitely have not solved the problem yet! The search algorithm prioritizes the brand called "Red Wing Shoe" so still figuring out ways to show real 'red shoes'. Have been thinking about passing the images through a detection tool and tag them to enhance the search experience.

Re: Value Proposition. Absolutely, I think focusing on the SMB-angle and 'local shopping' will help direct users better. I'll definitely take this into account.

Super cool product! I'm currently using a list of Shopify stores, so it's still limited (i.e. wanted to start with a relatively small list to focus on the search experience). I'll submit your URL to the crawler now. If you want to reach out to support @ searchagora.com , I'd love to get your feedback as a Shopify store owner.