Comment by freehunter | Hacker News Reader

Comment by freehunter | Hacker News Reader

My side project is a local news outlet for two small cities near me. They are small enough cities that I am the only game in town when it comes to digital media, the only competitor is the local newspaper who has no online presence. Plenty of our stories are actual news articles talking about local politics and new businesses/restaurants and local interest type stuff, all written by humans.

But filling in the gaps is plenty of machine-generated content. One of our big draws is the event calendar, and fortunately events give me enough data to feed an AI that I've developed. There is enough structured data and the information is so routine that this system can churn out several articles per day that sounds hand-written and every article sounds custom tailored to the business and the event itself. This fills in the routine work of keeping content flowing in between the stuff that takes a lot longer to research and write, and makes it viable for us (one full time employee and one part time employee) to run two popular small-town news websites where otherwise no one would even bother trying.

This is also a great case where classical AI is still relevant to modern problems. Neural networks just are not good at writing English in a way that humans would enjoy reading. At some point I plan on packaging it up and doing a Show HN, but for the time being, this article is spot on. Machine-generated news content breathes whole new life into an area that's increasingly hard to turn a profit.

Id be really interested in your technical approach to this. How did you implement this in a high level?

The core of it is a decision tree tied to a pretty in-depth database that either knows everything about the city or seeks to learn everything about the city. The system reads an event on the calendar that says "Van Halen is playing at American Brew Pub on February 12th at 7pm" and starts running running through the decision tree pulling information out of the database to fill in the blanks on the phrases it's picked.

Then it strings them all together to write something that looks something like:

>Do you have plans on Tuesday? Well now you do! Van Halen is playing at American Brewpub at 7pm! Van Halen is a local rock band that we love. They have been to our city before, but they played at Next Door Music Venue (link to the previous article for Van Halen at Next Door). This time they're playing at our favorite brewery, so you can watch the show while drinking Local IPA! Tickets are required, and you can purchase them here (link to tickets). We'll see you there!

People love these articles but they're not super fun to write every day (sometimes twice a day for the two cities), it's just routine stuff.

The concept seems pretty solid, but reading the output I feel like this is bordering on unethical. Maybe this is just a bad example, but it seems you’ve gone past programmatically giving people information, and into giving the impression you’re actually endorsing things. Unless you actually add metadata to the artist saying you love them, or rate the venue to indicate it’s good for live music, you’ve got no way of going beyond “Van Halen, a local rock band, are playing at American Brewpub at 7pm Thursday. You can drink local IPA while watching them, and buy tickets here.”

Maybe the AI also considers ratings of the band. Then it would be no more unethical than online shops displaying customer satisfaction ratings?

In any case the AI could make several educated guesses on how good the band is. Quality of the venue would be another indicator.

This is a very simplified example written by human hands just for this post (not machine written). The database is immense and has far more fields than you might imagine. It takes into account how many clicks content with Van Halen has had in the past and how many clicks content with American Brewpub has had etc to figure out how likely this article is to resonate with our audience, plus how many human-written articles have been published about these subjects and sentiment analysis to figure out if our articles were positive or neutral (we don't write negative articles, if we don't like a business/event we just don't talk about it), which impacts how the machine-generated article is phrased or even if it gets written at all.

We have about 90 events on our calendar per city in any given month, and this AI writes about three or four articles per city per week even though it's running constantly. It's pretty selective and very rarely writes something that our two human writers wouldn't have mentioned otherwise. Our personal preferences and those of our audience are certainly taken into account.

That’s incredibly cool, and definitely covers my concerns there. Good work on giving it some restraint rather than going for sheer volume as well.

I've been working on something along the same lines for the past couple of years. I would love to exchange some ideas with you!

Send me an email at moura at my the domain in my profile

Reply on news.ycombinator.com