mariarmestre

Hmmm... this is not eligible for their zero data retention policy anymore. Not sure how this will go down.

Are you going to keep maintaining the package?

I just tried it on a AI newsletter: https://a.tldrnewsletter.com/web-version?ep=1&lc=f08d3180-da... and it turns out I'm not keeping up with AI news as well as I thought.

Sorry, this is the link: https://community.intel.com/t5/Blogs/Tech-Innovation/Cloud/T...

Thank you! Our goal is to make production-ready code, so we believe that good documentation and stability of the code are paramount. I'll pass the feedback along the rest of the team :)

This represents a major rewrite of the package with more powerful features than ever.

We have introduced the new concept of components. Components are composable and customisable and can be connected into pipelines. Pipelines are dynamic execution graphs that support a range of flows from simple linear chains to more complex execution flows containing loops. This means, you can get started easily with a few lines of code, but have room for extending and customising the logic of the pipeline.

This restructuring of the package paves the way to building truly extensible and composable AI systems ready for production.

The team will be around for questions!

Thank you for this! Congrats for the beta release. I guess this is not totally battle-tested yet then?

I think a fundamental issue with search, and the reason why many companies do not invest in tuning a good search experience, is that the main metric usually is to minimise embarrassing/irrelevant results, rather than get the best possible set of results. How can you even know what is the best answer to your query? Systematic evaluation is very hard.

The only issue here is what I mentioned in the comment above: how easy is it to read and parse the content of said website and is it legal to read the content programmatically? Do you have any website(s) in mind?

Thanks so much for your comment! You're right that this annotation tool can be used on any form of free-form documents found online. I tackled Wikipedia first because it was an obvious first choice and they have an API to read the html. This could be opened to other sources of data, but I also do not want this to become a scraping tool, so we would need to weigh costs/benefits of adding new data sources. The additional cost of adding a new source is mostly about how difficult it is to read and parse the content. In the future, I could integrate with some paying sources (e.g. news publications), where people have to pay for the content they scrape & label.

I have a pitch deck and I'm looking for all the things you mentioned :-). I can send the pitch deck to anyone interested.

Wikidata is a very complete knowledge base, but I think there is still room for a tool like this to be used on Wikipedia data. There might be missing information on Wikidata which is still found on Wikipedia (e.g. list of Bytedance products or investors is incomplete, or the number of employees is also missing from Wikidata). This tool can be used to uncover these relationships for your application, or to feed back into Wikidata if it's of public interest.

It could also be that you are trying to extract data to train a named entity recognition model. In that case, you want to extract the paragraph or sentence that has the information and the label.

This is to build your own knowledge base. In many cases, Wikidata might not have the data you're looking for. For example, in the tutorial I have linked, the task is to come up with all the products released by a list of companies. Toutiao would be a product of Bytedance. This is a relation that might not exist on Wikidata (I tried to search for it but could not find it https://www.wikidata.org/wiki/Q24835387).

Hi! Thanks for your feedback. I had not come across this, but it looks quite similar :) (at least conceptually). I don't have a video, but there is a short gif in the repository. I am planning to make a video at some point though!