Perhaps HN isn't a significant threat, but it still poses a threat; each new service likely eats away at Digg's share of the market, assuming that people have limited time to use social media.
As an anecdote, I was using Digg substantially up until the 2008 elections. I haven't used the service since then. Instead, I'm spending time on Hacker News (+ Techmeme, Twitter, Facebook).